SwadhyAI Notes

All Python notes

K-Nearest Neighbors (KNN) 🤝

You Are Like Your Neighbors

KNN is a simple algorithm: "Tell me who your neighbors are, and I'll tell you who you are." It classifies a new data point based on the majority class of its K nearest neighbors.

✨ How it Works

  1. Choose K (e.g., K=3).
  2. Calculate the distance from the new point to all existing points.
  3. Select the K closest points.
  4. Classification: Vote! The most common class wins.
  5. Regression: Average the values.

💻 Implementation

python
from sklearn.neighbors import KNeighborsClassifier

x = [4, 5, 10, 4, 3, 11, 14 , 8, 10, 12]
y = [21, 19, 24, 17, 16, 25, 24, 22, 21, 21]
classes = [0, 0, 1, 0, 0, 1, 1, 0, 1, 1]

data = list(zip(x, y))

# K=3 means look at 3 nearest neighbors
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(data, classes)

# Predict for a new point
new_point = [(8, 21)]
prediction = knn.predict(new_point)

print("Predicted Class:", prediction[0])

✨ Choosing K

Small K (e.g., 1): Sensitive to noise. Large K: May blur boundaries. Try odd numbers (3, 5, 7) to avoid ties.