Wisdom of the Crowd
One person guessing jellybeans is usually wrong. A whole crowd averaged together is often amazingly close.
The jar has a secret number of jellybeans. Everyone guesses a bit too high or too low. Slide the crowd size up and watch the average settle on the truth!
Everybody's guesses
Green line = the true number. Purple arrow = the crowd's average. Each dot is one person.
Challenges
Big crowd: use 100 or more people, and make the crowd at least twice as close as a typical person.
Fair fight: ask 5 random people. The crowd should beat at least 4 of them.
Fooled together: set the shared bias to 20% or more, use 100+ people, and see the crowd stay wrong by at least 10%.
What just happened?
- Each person guesses with their own random wobble, some high and some low.
- When you average many guesses, the highs and lows cancel out. The crowd error shrinks fast at first, then slowly (about one over the square root of the crowd size).
- The one person who happened to be closest is often closer than the average, but you cannot know who they are before the answer is revealed.
- If everybody leans the same way, averaging does nothing about it. A crowd can fix random mistakes, but not a mistake everyone shares.
How it works
- Everyone guesses the jar. Each guess is the truth, plus a shared lean, plus personal noise.
- We average the first N guesses.
- We compare that average with the truth, and with how far off a typical single person is.
- The chart repeats the whole experiment 300 times with fake crowds, so the line is a fair average and not one lucky jar.
For grown-ups: what's really going on
Each guess is modelled as gi = T(1 + b + σ·εi) with εi ~ N(0,1), σ = 0.3 and bias b shared by everyone. The crowd mean has bias T·b and variance (Tσ)²/n, so its typical error falls like 1/√n until it hits the floor set by the bias.
- This is the idea behind variance reduction by averaging, used by bagging and random forests: average many noisy but roughly unbiased models.
- It needs the errors to be independent. If errors are correlated with correlation ρ, the variance floors at
ρσ²instead of going to zero. A shared bias is the extreme case. - The jellybean story is the classic Galton ox-weight example (1907) and is why ensembles work: diversity matters as much as numbers.
The chart is a Monte Carlo estimate (300 simulated crowds). The jar and crowd come from a seeded random generator, so a seed always gives the same crowd.