How Sure Is Sure?
A robot guesses if fruit is ripe, and says how sure it is. Is "95% sure" really right 95% of the time? Let's check, then teach it to say "I don't know".
If I say "90% sure" I should be right 9 times in 10. Look at the chart: the dots should sit on the dashed line. They don't yet!
Is its confidence honest?
Bottom: how sure it said it was. Side: how often it was really right. Bars on the dashed line mean honest. Bars below it mean overconfident.
Challenges
What just happened?
- Trained on only a few fruit, the robot learned to sound very sure. Many of its "I'm 95% sure" answers were wrong.
- Calibrating did not make it smarter. Its accuracy stayed about the same, but its confidence became a number you can trust.
- With honest confidence, "I don't know" works: you give up some answers and the ones left are right more often.
- This is like a chatbot that states a made-up fact in a confident voice. Knowing when to be unsure is a safety feature.
For grown-ups: what's really going on
A logistic regression with 14 features (3 informative, 11 noise) is trained on 50 examples with no regularisation for many steps, so it fits noise and produces extreme logits. This is overconfidence, common in modern neural networks too.
- The reliability diagram bins predictions by confidence (the probability of the chosen answer) and plots the bin's real accuracy. Expected calibration error is the weighted average gap:
ECE = Σ (nb/N) · |accb − confb|. - Temperature scaling divides the logit by one number T fitted on a separate validation set by minimising log loss:
p = σ(z / T). It never changes which answer is picked, only how sure it sounds. - Platt scaling fits
p = σ(a·z + c)on the validation set, which can also shift the model if it is biased towards one answer. - Selective prediction (abstention) answers only above a confidence threshold, producing a risk-coverage trade-off. It works only if confidence ranks correct above incorrect answers and is well calibrated.
- Language models can state false claims fluently (hallucination). Calibrated uncertainty, retrieval, and abstention are active ways to reduce harm, but token probability is not truth.
Further reading: Guo et al., On Calibration of Modern Neural Networks (2017) · Platt, Probabilistic Outputs for Support Vector Machines (1999) · Geifman and El-Yaniv, Selective Classification for Deep Neural Networks (2017).