Same Size, Fair Fight
Age goes up to 80 but income goes up to 200,000. Watch the robot ignore age completely, then fix it by putting both on the same size.
Click the picture to place a new person. I will find the 5 people nearest to them. Are they really the most alike? Then try the scaling buttons!
Who is alike? (click or use arrow keys to move the star)
likes the gadgetdoes notstar = new personring = 5 nearest
The picture always shows age sideways and income upwards. The ringed people are the ones the robot thinks are nearest, using the setting you picked.
Challenges
See the problem: with no scaling and income in dollars, make age supply under 5% of the distance.
Fair fight: reach at least 85% test accuracy with scaling on, from a start under 75%.
Units matter: with no scaling, switch income to thousands of dollars and watch accuracy jump.
Faster learning: pick a scaling that gets the learning error below 1% of its start within 60 steps.
What just happened?
- The robot measures how far apart two people are by adding up the gaps in each clue. A gap of 50,000 dollars is huge next to a gap of 30 years, so income wins and age hardly counts.
- Scaling squeezes every clue onto a similar size. Then age and income both get a vote and the nearest people really are the most alike.
- Writing income in thousands of dollars helped too. The units you pick quietly change what the robot sees.
- Learning by small steps (gradient descent) has the same trouble: with clues of very different sizes it creeps along. Scaled clues learn much faster.
- Not every robot cares. Decision trees ignore sizes. But distance-based and step-by-step learners need fair sizes.
For grown-ups: what's really going on
The classifier is k-nearest neighbours (k = 5, majority vote, Euclidean distance) and the data is two features with labels from age_norm + income_norm + noise > 1. Accuracy is measured on 300 held-out people; scalers are fitted on the 200 training people only to avoid data leakage.
- Min-max scaling:
x' = (x − min) / (max − min), mapping to [0, 1] but sensitive to outliers. - Standardisation (z-score):
x' = (x − μ) / σ, giving mean 0 and standard deviation 1. - Euclidean distance is
√Σ(aᵢ − bᵢ)², so a feature with a larger range dominates the sum. The "distance share" bars showΣ(Δage)² / Σ(Δ²)over the neighbours. - The learning-speed chart fits
spend = w₁·age + w₂·income + bby gradient descent on mean squared error, always using the largest stable step1/λmaxof the Hessian(2/n)XᵀX(found by power iteration). The loss surface is a long thin valley when features differ in scale (a large condition number), so progress along the flat direction is very slow. Scaling reduces the condition number. - Used with scaling: k-NN, k-means, SVMs, PCA, neural networks, regularised regression. Mostly unaffected: decision trees and random forests.