AIalgo.si
Glossary

k-NN Explained Visually

DMDr. Maja NovakJune 25, 20255 min read

Quick answer

The k-nearest-neighbors algorithm needs no training at all — it just remembers. See why small k means wiggly boundaries and large k means smooth ones.

What is k-nearest neighbors?

k-nearest neighbors (k-NN) is the simplest classification algorithm in machine learning: to classify a new point, look at the k closest labeled points and take a majority vote. No training, no parameters fitted — the "model" is the dataset itself.

Think of moving to a new city and asking your k closest neighbors which football team they support. If k is 3, your answer is whatever the three nearest people say. If k is 300, you get a city-wide consensus.

Why k matters so much

Small k produces flexible, wiggly decision boundaries that chase every outlier — high variance, low bias. Large k smooths everything toward the majority class — low variance, high bias. Set k to 1 and you memorize noise; set k to the size of the dataset and you predict the same class every time.

The sweet spot is usually between 3 and 15 for datasets of a few thousand points, and odd values avoid tied votes in binary problems. Always standardize your features first — k-NN measures distances, and raw scales let big-number features dominate.

The catch: cost at prediction time

k-NN inverts the usual training/prediction trade. Training is instant, but every prediction compares the query against every stored point — O(n) distance computations. Approximate nearest-neighbor indexes (like those in vector databases) exist precisely to fix this.

Try it in the AIalgo.si demo: drag the k slider from 1 to 20 and watch the decision boundary snap from jagged islands to a smooth curve. That single visual teaches more about the bias-variance tradeoff than a lecture.

Cite this page

AIalgo.si Editorial (2025). k-NN Explained Visually. AIalgo.si Resources. https://aialgo.si/resources/k-nn-explained-visually

#knn#classification#fundamentals

Keep reading