fairsquare
play

FairSquare: Probabilistic Verification of Program Fairness Aws - PowerPoint PPT Presentation

Oct 02, 2023 •255 likes •654 views

FairSquare: Probabilistic Verification of Program Fairness Aws Albarghouthi Loris DAntoni Samuel Drews University of Wisconsin-Madison Aditya V. Nori Microsoft Research Machine Data Model Learning Machine Data Model Learning small

  1. FairSquare: Probabilistic Verification of Program Fairness Aws Albarghouthi Loris D’Antoni Samuel Drews University of Wisconsin-Madison Aditya V. Nori Microsoft Research

  2. Machine Data Model Learning

  3. Machine Data Model Learning small error

  4. Machine Data Model Learning small error racist? sexist?

  6. Group Fairness decision- making program

  7. Group Fairness sensitive feature (e.g. minority)

  8. Group Fairness sensitive feature (e.g. minority)

  9. Group Fairness population model

  10. Individual Fairness similarity

  11. Q: What are good definitions of fairness?

  12. Q: What are good definitions of fairness? A: Someone else’s problem

  13. Contributions ● Formalize fairness definitions as probabilistic verification problems ● Decision procedure for evaluating probabilistic postconditions

  14. FairSquare

  15. Population Model Decision Program Fairness FairSquare Definition

  16. Population Model Unfairness proof Decision Program Fairness proof Fairness FairSquare Definition

  18. population decision-making model program

  19. population decision-making model program

  22. represent assignments as a region (LRA formula) colRank “weighted volume” ethnicity yExp

  23. define dec(colRank, yExp): define popModel(): expRank ← yExp – colRank ethnicity ~ gauss(0,10) if colRank <= 5: colRank ~ gauss(25,10) hire ← true yExp ~ gauss(10,5) elif expRank > -5: if ethnicity > 10: hire ← true colRank ← colRank + 5 else: return colRank, yExp hire ← false return hire popModel dec Pr

  24. define dec(colRank, yExp): define popModel(): expRank ← yExp – colRank ethnicity ~ gauss(0,10) if colRank <= 5: colRank ~ gauss(25,10) hire ← true yExp ~ gauss(10,5) elif expRank > -5: if ethnicity > 10: hire ← true colRank ← colRank + 5 else: return colRank, yExp hire ← false return hire popModel dec Pr

  25. define dec(colRank, yExp): define popModel(): expRank ← yExp – colRank ethnicity ~ gauss(0,10) if colRank <= 5: colRank ~ gauss(25,10) hire ← true yExp ~ gauss(10,5) elif expRank > -5: if ethnicity > 10: hire ← true colRank ← colRank + 5 else: return colRank, yExp hire ← false return hire popModel dec Pr

  33. Evaluating post ● Obtain formula for each probability in post ● Underapproximate the weighted volume of ● Overapproximate by doing the same for eventually, approximations are good enough

  34. “Live” demo

  35. Case Study Bayesian Population Networks Model Income Dataset [1] Decision Trees, Decision Linear SVMs, Program RLU Neural Nets Group Fairness Fairness for Women Definition [1] https://archive.ics.uci.edu/ml/datasets/adult

  36. Fairness Verification Problems 14 12 NN 10 SVM 8 DT 6 4 2 0 Total FairSquare PSI [1] VolComp [2] [1] Gehr et al. CAV 2016 [2] Sankaranarayanan et al. PLDI 2013

  37. Population Model Unfairness proof Decision Program Fairness proof Fairness FairSquare Definition

Recommend

More recommend