Learning from Examples

Nearest Neighbours

You are the teacher! Put apples, lemons and pears on the table. Then drag the mystery fruit around: the robot guesses it by asking the fruit that sit closest to it.

I can't taste or smell. I only know two things about a fruit: how round and how yellow it is. So every fruit is a spot on my table. Drag the big ? around and I'll guess what lives there!

Tap the table to drop the fruit you picked · drag any fruit to move it · drag the big ? to ask the robot

Snapshots while learning

A photo of my map after 1, 2, 4, 8… examples. More examples = a smarter map. Tap one to look back.

How it works

Examples with answers

Every fruit you drop comes with a name tag. The name tag is the "answer".

Turn fruit into spots

Roundness goes left-right, yellowness goes up-down. Every fruit becomes a spot on the table.

Ask the neighbours

For a new fruit, find the K fruit sitting closest to it.

Most votes wins

Each neighbour votes for its own name. The name with the most votes is the guess!

What just happened?

For grown-ups: what's really going on

This is k-Nearest Neighbours (k-NN) classification with two features (roundness, yellowness) and three labels.

  • k-NN is a lazy / instance-based learner: "training" is just storing examples. Prediction finds the k closest stored points by Euclidean distance √((x₁−x₂)² + (y₁−y₂)²) and takes a majority vote.
  • The tinted background is the decision boundary; colour strength shows how unanimous the vote is.
  • k trades variance for bias: k = 1 memorises noise (jagged islands), large k smooths everything and can out-vote small classes. Odd k avoids ties with two classes.
  • The test score is accuracy on a held-out test set drawn from the same distribution; the chart is a learning curve (accuracy vs. training-set size).
  • In real use, features must be on comparable scales (normalise them), and k-NN gets slow and weak in high dimensions (the curse of dimensionality).

Further reading: An Introduction to Statistical Learning, §2.2.3 · scikit-learn "Nearest Neighbors" guide.