Predict Responsibly: Increasing Fairness by Learning to Defer David - PowerPoint PPT Presentation

Predict Responsibly: Increasing Fairness by Learning to Defer David Madras , Toniann Pitassi, Richard Zemel University of Toronto, Vector Institute December 8, 2017 David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 1 / 15

The Judge and the Black-Box David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 2 / 15

The Judge and the Black-Box David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 2 / 15

The Judge and the Black-Box “0.6” David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 2 / 15

The Judge and the Black-Box “0.6” What does the prediction “0.6” mean? What qualities should it have? David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 2 / 15

What We Want From Black Box Predictions David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 3 / 15

What We Want From Black Box Predictions 1 Accuracy David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 3 / 15

What We Want From Black Box Predictions 1 Accuracy 2 Fairness David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 3 / 15

What We Want From Black Box Predictions 1 Accuracy 2 Fairness 3 Responsibility — Ability to say “I Don’t Know” David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 3 / 15

Why Say IDK? Judge is external decision maker (DM) - may have more knowledge Can seek out extra information on difficult cases Can assess qualitative or difficult-to-codify features Can access privacy-sensitive information David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 4 / 15

Learning to Punt “Positive”, “Negative”, and “IDK” Learn two thresholds: t 0 , t 1 At test time, punt to DM if t 0 < x i < t 1 ; else, output prediction David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 5 / 15

Results - Punting Trained our model (2-layer NN) with fair regularization L fair = Accuracy + α · Fairness Simulated external DM by training separate (unfair) model This DM received some extra attributes in training, simulating a possible real-life imbalance between DM and model David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 6 / 15

Results - COMPAS 0.20 baseline-acc DM punt-fair punt-unfair 0.15 binary-fair Disparate Impact 0.10 0.05 0.00 0.22 0.24 0.26 0.28 0.30 0.32 0.34 Error Rate David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 7 / 15

Results - Heritage Health 0.45 baseline-acc DM 0.40 punt-fair punt-unfair 0.35 binary-fair 0.30 Disparate Impact 0.25 0.20 0.15 0.10 0.05 0.00 0.16 0.18 0.20 0.22 0.24 Error Rate David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 8 / 15

DM-Aware Learning What if judge has access to extra info on some defendants? Detailed written analysis, classified info, further inquiry What if judge is biased towards some types of defendants? Unfairness may be concentrated on a few examples By using info about the DM during learning, we could punt more intelligently This is learning to defer David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 9 / 15

Learning to Defer Modify our model to take DM scores Y DM on training set Use IDK output as a mixing parameter π i Can describe system output Y sys as function of s ∼ Bernoulli ( π i ) , Y DM , and Y model Y sys = s · Y DM + (1 − s ) · Y model s ∈{ 0 , 1 } ; Y sys , Y DM , Y model ∈ [0 , 1] David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 10 / 15

Learning to Defer Suppose we are optimizing some loss function L ( Y , Y sys ) over ground truth labels Y and system output Y sys We can then define a new loss function L Defer L Defer ( Y , Y sys ) = E s L ( Y , Y sys ) = E s L ( Y , s · Y DM + (1 − s ) · Y model ) Penalty for IDK ≈ DM loss on that example David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 11 / 15

Results (Learning to Defer) - COMPAS 0.20 defer-fair punt-fair binary-fair DM 0.15 baseline-acc Disparate Impact 0.10 0.05 0.00 0.22 0.24 0.26 0.28 0.30 0.32 0.34 Error Rate David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 12 / 15

Results (Learning to Defer) - Heritage Health 0.45 defer-fair punt-fair 0.40 binary-fair DM 0.35 baseline-acc 0.30 Disparate Impact 0.25 0.20 0.15 0.10 0.05 0.00 0.16 0.18 0.20 0.22 0.24 Error Rate David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 13 / 15

Conclusion We argue that it is important to consider IDK models as part of a larger pipeline We demonstrate that learning to defer can provide benefits above and beyond learning to punt Deferring intelligently can improve the entire pipeline in both accuracy and fairness David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 14 / 15

Thank you! David Madras , Toniann Pitassi, Richard Zemel (University of Toronto, Vector Institute) Predict Responsibly December 8, 2017 15 / 15