Data & Features

Same Size, Fair Fight

Age goes up to 80 but income goes up to 200,000. Watch the robot ignore age completely, then fix it by putting both on the same size.

Click the picture to place a new person. I will find the 5 people nearest to them. Are they really the most alike? Then try the scaling buttons!

Who is alike? (click or use arrow keys to move the star)

likes the gadgetdoes notstar = new personring = 5 nearest

The picture always shows age sideways and income upwards. The ringed people are the ones the robot thinks are nearest, using the setting you picked.

Challenges

See the problem: with no scaling and income in dollars, make age supply under 5% of the distance.
Fair fight: reach at least 85% test accuracy with scaling on, from a start under 75%.
Units matter: with no scaling, switch income to thousands of dollars and watch accuracy jump.
Faster learning: pick a scaling that gets the learning error below 1% of its start within 60 steps.

What just happened?

For grown-ups: what's really going on

The classifier is k-nearest neighbours (k = 5, majority vote, Euclidean distance) and the data is two features with labels from age_norm + income_norm + noise > 1. Accuracy is measured on 300 held-out people; scalers are fitted on the 200 training people only to avoid data leakage.

  • Min-max scaling: x' = (x − min) / (max − min), mapping to [0, 1] but sensitive to outliers.
  • Standardisation (z-score): x' = (x − μ) / σ, giving mean 0 and standard deviation 1.
  • Euclidean distance is √Σ(aᵢ − bᵢ)², so a feature with a larger range dominates the sum. The "distance share" bars show Σ(Δage)² / Σ(Δ²) over the neighbours.
  • The learning-speed chart fits spend = w₁·age + w₂·income + b by gradient descent on mean squared error, always using the largest stable step 1/λmax of the Hessian (2/n)XᵀX (found by power iteration). The loss surface is a long thin valley when features differ in scale (a large condition number), so progress along the flat direction is very slow. Scaling reduces the condition number.
  • Used with scaling: k-NN, k-means, SVMs, PCA, neural networks, regularised regression. Mostly unaffected: decision trees and random forests.