Is It Fair?
A pretend club robot learned from old decisions that were unfair to Group B. Check who it says yes to, then try to fix it.
Everything here is made up. Look at the two bars: the robot copied the old unfairness. Try switching things off and see what changes!
100 new applicants, who gets in?
Each dot is a person. Coloured = the robot says yes. A red ring = really a good member, but the robot said no.
Challenges
What just happened?
- The robot learned from old choices that marked Group B down, so it copied that unfairness. Nobody told it to.
- Deleting the Group column did not fix it. The neighbourhood code is a stand-in (a proxy) for the group, so the robot found the group again.
- Fixes help in different ways: drop proxy clues, reweight the examples, or set different bars. Each one trades something.
- There is no single button for "fair". People have to choose what fair means, and check each group's results.
For grown-ups: what's really going on
Applicants are synthetic. A hidden ground truth q depends only on skill and effort. The "historical" label subtracts a penalty for group B (label bias), and group B also has a slightly lower mean score and lives mostly in area 1 (proxy variable, ~85% vs ~15%). A logistic regression is trained by weighted gradient descent on the historical labels only.
- Demographic parity gap: difference in approval rates between groups. Equal opportunity gap: difference in true positive rate (
P(yes | really qualified)). Both are shown; they can disagree, and known results show that several fairness criteria cannot all hold at once when base rates differ. - Fairness through unawareness (dropping the protected column) fails when other features correlate with it. Look at the weight on neighbourhood code after you drop Group.
- Reweighing (Kamiran and Calders) gives each example weight
P(g)P(y) / P(g, y)so group and label look independent in training. - Group-specific thresholds post-process the scores. They can equalise a metric but treat people differently by group, which can be legally or ethically contested.
- "Right about real skill" uses the hidden truth, which real systems never see. Real accuracy is measured against the same biased labels, which makes biased models look good.
Further reading: Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning (2016) · Barocas, Hardt and Narayanan, Fairness and Machine Learning.