Twenty Questions Trees
A decision tree plays twenty questions: "Is it left of this line? Is it above that one?" Grow one question at a time, then plant a whole forest of trees that vote together.
See the red, blue and yellow shapes? I want to sort them with as few yes/no questions as I can. Press Ask one question and watch me draw my first line!
Tap to add shapes · point at any spot to see how the trees decide · hollow shapes are test shapes the trees never learn from
The question tree
The forest
Snapshots while learning
A photo of the map after 1, 2, 4, 8… questions. Tap one to look back.
How it works
Try every question
The tree tries every possible straight line: "left or right of here?", "above or below here?"
Pick the tidiest split
It keeps the question that leaves each side the least mixed up.
Ask again inside
Each side gets its own new question, until every box is tidy or we run out of questions.
Forest votes
Many slightly different trees each guess. The most popular answer wins — fewer silly mistakes!
What just happened?
- The tree split the map with yes/no questions, one at a time.
- Each question was chosen to make the boxes less mixed.
- A very deep tree gets every practice shape right but trips on new test shapes — it memorised instead of learning.
- A forest of different trees voting together makes smoother, safer guesses.
For grown-ups: what's really going on
This is a CART decision tree and a random forest on two features.
- Each split is an axis-aligned threshold chosen to minimise weighted Gini impurity
G = 1 − Σ pₖ²of the two children ("how mixed is this box"). The tree here grows best-first: each click splits the leaf with the biggest impurity reduction. - Max depth limits complexity. Deep trees have low bias but high variance: practice (training) accuracy → 100% while test accuracy drops — overfitting.
- A random forest trains each tree on a bootstrap sample (draw n points with replacement) and considers a random feature subset at each split (here: one of the two axes). Predictions average the trees' class probabilities (bagging), which cuts variance.
- The small multiples show each tree's own decision map; the big map is the averaged vote.
Further reading: Breiman (2001) "Random Forests" · R2D3 "A visual introduction to machine learning" · ISLR ch. 8.