Ethics, Safety & Alignment

Is It Fair?

A pretend club robot learned from old decisions that were unfair to Group B. Check who it says yes to, then try to fix it.

Everything here is made up. Look at the two bars: the robot copied the old unfairness. Try switching things off and see what changes!

100 new applicants, who gets in?

Each dot is a person. Coloured = the robot says yes. A red ring = really a good member, but the robot said no.

Challenges

See it: with everything switched on, find a gap of 20 points or more between the groups.
Sneaky clue: switch off the Group column, and notice the gap is still 10 points or more.
Reweight: turn on reweighting and see the gap shrink compared with before.
Fairer: get the good-member gap to 5 points or less, while staying 80% right about real skill.
Try the bar: move Group B's bar and watch both the gap and the score change.

What just happened?

For grown-ups: what's really going on

Applicants are synthetic. A hidden ground truth q depends only on skill and effort. The "historical" label subtracts a penalty for group B (label bias), and group B also has a slightly lower mean score and lives mostly in area 1 (proxy variable, ~85% vs ~15%). A logistic regression is trained by weighted gradient descent on the historical labels only.

  • Demographic parity gap: difference in approval rates between groups. Equal opportunity gap: difference in true positive rate (P(yes | really qualified)). Both are shown; they can disagree, and known results show that several fairness criteria cannot all hold at once when base rates differ.
  • Fairness through unawareness (dropping the protected column) fails when other features correlate with it. Look at the weight on neighbourhood code after you drop Group.
  • Reweighing (Kamiran and Calders) gives each example weight P(g)P(y) / P(g, y) so group and label look independent in training.
  • Group-specific thresholds post-process the scores. They can equalise a metric but treat people differently by group, which can be legally or ethically contested.
  • "Right about real skill" uses the hidden truth, which real systems never see. Real accuracy is measured against the same biased labels, which makes biased models look good.

Further reading: Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning (2016) · Barocas, Hardt and Narayanan, Fairness and Machine Learning.