k-NN Explained Visually
Quick answer
The k-nearest-neighbors algorithm needs no training at all — it just remembers. See why small k means wiggly boundaries and large k means smooth ones.
What is k-nearest neighbors?
k-nearest neighbors (k-NN) is the simplest classification algorithm in machine learning: to classify a new point, look at the k closest labeled points and take a majority vote. No training, no parameters fitted — the "model" is the dataset itself.
Think of moving to a new city and asking your k closest neighbors which football team they support. If k is 3, your answer is whatever the three nearest people say. If k is 300, you get a city-wide consensus.
Why k matters so much
Small k produces flexible, wiggly decision boundaries that chase every outlier — high variance, low bias. Large k smooths everything toward the majority class — low variance, high bias. Set k to 1 and you memorize noise; set k to the size of the dataset and you predict the same class every time.
The sweet spot is usually between 3 and 15 for datasets of a few thousand points, and odd values avoid tied votes in binary problems. Always standardize your features first — k-NN measures distances, and raw scales let big-number features dominate.
The catch: cost at prediction time
k-NN inverts the usual training/prediction trade. Training is instant, but every prediction compares the query against every stored point — O(n) distance computations. Approximate nearest-neighbor indexes (like those in vector databases) exist precisely to fix this.
Try it in the AIalgo.si demo: drag the k slider from 1 to 20 and watch the decision boundary snap from jagged islands to a smooth curve. That single visual teaches more about the bias-variance tradeoff than a lecture.
Cite this page
AIalgo.si Editorial (2025). k-NN Explained Visually. AIalgo.si Resources. https://aialgo.si/resources/k-nn-explained-visually