Learn from Mistakes
Each tiny helper makes just one straight cut. After each round, the dots it got wrong grow bigger so the next helper pays more attention.
Purple dots are inside a circle, orange are outside. One straight cut cannot do it. Press Add one helper and watch the mistakes grow!
Round 0
Bigger dot = the learner cares more about it. Thick ring = the newest helper got it wrong. Dashed line = the newest helper's cut. Background = what all helpers say together.
Challenges
What just happened?
- The first helper makes one straight cut, the best it can. It is not great, maybe 65% right.
- Dots it got wrong grow bigger; dots it got right shrink.
- The next helper sees the bigger dots as more important, so it picks a cut that fixes those.
- Each helper gets a say in the final vote, louder if it was accurate. Together a pile of weak cuts draws a shape no single cut could.
How it works
- Give every dot the same weight.
- Find the single cut (a stump) with the lowest weighted mistake rate.
- Give that helper a vote size from its error: fewer mistakes means a louder vote.
- Multiply the weights of the dots it got wrong up, and the right ones down, then rescale.
- Repeat. The final answer is the sign of the sum of all votes.
For grown-ups: what's really going on
This is discrete AdaBoost (Freund and Schapire) with depth-1 decision stumps as weak learners. For round t with weighted error εt: αt = ½ ln((1 − εt) / εt), then wi ← wi · exp(−αt yi ht(xi)), normalised to sum 1. The final model is H(x) = sign(Σ αt ht(x)).
- Boosting reduces bias: it builds a complex boundary out of simple pieces. Each stump looks at one axis, so the final model is a sum of one-feature functions. A circle (x² + y²) is exactly such a sum, which is why this puzzle suits stumps.
- It is a stagewise additive model minimising exponential loss
Σ exp(−y F(x)), which is why misclassified points get exponentially growing weight. - Training error can reach zero while test error keeps improving on clean data, a famous observation about margins. With noisy labels, though, the growing weights chase the wrong labels; the next lesson shows this.
- Errors are clamped strictly between 0 and ½ so α stays finite.
Data: 90 random dots in the unit square, class 1 inside a circle of radius 0.33 around the centre, no label noise. Tested on 500 fresh dots.