▶Neighborhood
Rank training points by distance to the query and keep the closest k.
K-Nearest Neighbors
A training-free classifier built from nearby examples
KNN stores the training set and labels a new point by majority vote among its k nearest examples.
watchone idea
→manipulateone example
→leave withone intuition
The essential idea: small k follows local detail; large k smooths the decision rule.
One Concept · One Example
What to notice
- Prediction requires distances to stored data.
- The neighborhood changes with the query point.
- k controls overfitting versus underfitting.
Prediction by local vote
Majority vote
The most frequent label inside that local neighborhood becomes the prediction.
From one local vote to model complexity
First inspect the neighbors behind one prediction. Then use the noisy half-moon example from the slides to see how k controls overfitting and underfitting.
Example 2 · Model complexity
When does KNN overfit or underfit?
The slides use two noisy interlocking half-moons. Change the only hyperparameter, k, and watch the decision boundary move from noise-sensitive to overly smooth.
KNN makes no parametric fit; its complexity is controlled mainly by the neighborhood size k.