biases in nlp models and what it takes to control them
play

Biases in NLP Models and What It Takes to Control them Kai-Wei - PowerPoint PPT Presentation

Feb 27, 2024 •182 likes •917 views

Biases in NLP Models and What It Takes to Control them Kai-Wei Chang 1 A carton of ML (NLP) pipeline Prediction Evaluation (Structured) Inference Auxiliary Corpus/Models Representation (e.g, word embedding) Data Kai-Wei Chang

  1. Biases in NLP Models and What It Takes to Control them Kai-Wei Chang 1

  2. A carton of ML (NLP) pipeline Prediction Evaluation (Structured) Inference Auxiliary Corpus/Models Representation (e.g, word embedding) Data Kai-Wei Chang (kw@kwchang.net) 2

  3. Motivate Example: Coreference Resolution • Coreference resolution is biased 1,2 • Model fails for female when given same context Semantics Only w/ Syntactic Cues his ⇒ her 1 Zhao et al. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. NAACL 2018. 2 Rudinger et al. Gender Bias in Coreference Resolution. NAACL 2018 Kai-Wei Chang (kw@kwchang.net) 3

  4. Wino-bias data v Stereotypical dataset v Anti-stereotypical dataset Kai-Wei Chang (kw@kwchang.net) 4

  5. Gender bias in Coref System 78 73 68 63 58 53 48 Neural Coref Model E2E E2E (Debiased WE) E2E (Full model) Steoetype Anti-Steoretype Avg Kai-Wei Chang (kw@kwchang.net) 5

  6. Gender bias in Coref System 78 73 68 63 58 53 48 Neural Coref Model E2E E2E (Debiased WE) E2E (Full model) Steoetype Anti-Steoretype Avg Kai-Wei Chang (kw@kwchang.net) 6

  7. Gender bias in Coref System 78 73 68 63 58 53 48 Neural Coref Model Mitigate WE Bias Mitigate Data Bias E2E E2E (Debiased WE) E2E (Full model) Steoetype Anti-Steoretype Avg Kai-Wei Chang (kw@kwchang.net) 7

  8. Misrepresentation and Bias Kai-Wei Chang (kw@kwchang.net) 8

  9. Stereotypes Which word is more likely to be used by a female ? Giggle – Laugh (Preotiuc-Pietro et al. ‘16) Credit: Yulia Tsvetkov Kai-Wei Chang (kw@kwchang.net) 9

  10. Stereotypes Which word is more likely to be used by a female ? Giggle – Laugh (Preotiuc-Pietro et al. ‘16) Credit: Yulia Tsvetkov Kai-Wei Chang (kw@kwchang.net) 10

  11. Stereotypes Which word is more likely to be used by a older person ? Impressive – Amazing (Preotiuc-Pietro et al. ‘16) Credit: Yulia Tsvetkov Kai-Wei Chang (kw@kwchang.net) 11

  12. Stereotypes Which word is more likely to be used by a older person ? Impressive – Amazing (Preotiuc-Pietro et al. ‘16) Credit: Yulia Tsvetkov Kai-Wei Chang (kw@kwchang.net) 12

  13. Why do we intuitively recognize a default social group? Credit: Yulia Tsvetkov 13

  14. Why do we intuitively recognize a default social group? Implicit Bias Credit: Yulia Tsvetkov 14

  15. BIASED AI Data is riddled with Implicit Bias Modified from Yulia Tsvetkov’s slide 15

  16. Bias in Wikipedia v Only small portion of editors are female v Have less extensive articles about women v Have fewer topics important to women. (Ruediger et al., 2010) Kai-Wei Chang (kw@kwchang.net) 16

  17. BIASED AI Consequence: models are biased Credit: Yulia Tsvetkov 17

  18. Bias in Language Generation The Woman Worked as a Babysitter: On Biases in Language Generation (Sheng EMNLP 2019) • Language generation is biased (GPT-2) Kai-Wei Chang (kw@kwchang.net) 18

  19. Where’s Biases? Kai-Wei Chang (kw@kwchang.net) 19

  20. A carton of ML (NLP) pipeline Prediction Evaluation (Structured) Inference Auxiliary Corpus/Models Representation (e.g, word embedding) Data Kai-Wei Chang (kw@kwchang.net) 20

  21. Representational Harm in NLP: Word Embeddings can be Sexist Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings [ Bolukbasi et al. NeurIPS16] Given gender direction ( 𝑤 #$ − 𝑤 &#$ ) , find word pairs with parallel direction by cos(𝑤 , − 𝑤 - , 𝑤 #$ − 𝑤 &#$ ) he: _______ she:_______ brother sister she beer cocktail he physician registered_nurse professor associate professor Google w2v embedding trained from the news Kai-Wei Chang (kw@kwchang.net) 21

  22. Implicit association test (IAT) v Greenwald et al. 1998 v Detect the strength of a person's subconscious association between mental representations of objects (concepts) Boy Math Girl Reading https://en.wikipedia.org/wiki/Implicit-association_test https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 22

  23. Implicit association test (IAT) Boy Girl https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 23

  24. Implicit association test (IAT) Boy Girl Emily https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 24

  25. Implicit association test (IAT) Boy Girl Tom https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 25

  26. Implicit association test (IAT) Math Reading https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 26

  27. Implicit association test (IAT) Math Reading number https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 27

  28. Implicit association test (IAT) Boy Girl Math Reading https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 28

  29. Implicit association test (IAT) Boy Girl Math Reading Algebra https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 29

  30. Implicit association test (IAT) Boy Girl Math Reading Julia https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 30

  31. Implicit association test (IAT) Boy Girl Reading Math https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 31

  32. Implicit association test (IAT) Boy Girl Reading Math Literature https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 32

  33. Implicit association test (IAT) Boy Girl Reading Math Dan https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 33

  34. Implicit association test (IAT) https://implicit.harvard.edu Kai-Wei Chang (kw@kwchang.net) 34

  35. Word Embedding Association Test (WEAT) • X : “mathematics”, “science”; Y : “arts”, “design” • A : “male”, “boy”; B : “female”, “girl” “mathematics” “male”, “boy” “female”, “girl” Caliskan et al. Semantics derived automatically from language corpora contain human-like biases Science. 2017 35 Kai-Wei Chang (kw@kwchang.net)

  36. Word Embedding Association Test (WEAT) • X : “mathematics”, “science”; Y : “arts”, “design” • A : “male”, “boy”; B : “female”, “girl” Differential association of the two sets of words with the Aggregate the target words attributes Caliskan et al. Semantics derived automatically from language corpora contain human-like biases Science. 2017 36 Kai-Wei Chang (kw@kwchang.net)

  37. Word Embedding Association Test (WEAT) • X : “mathematics”, “science”; Y : “arts”, “design” • A : “male”, “boy”; B : “female”, “girl” The effect size of bias: Caliskan et al. Semantics derived automatically from language corpora contain human-like biases Science. 2017 37 Kai-Wei Chang (kw@kwchang.net)

  38. Word Embedding Association Test Caliskan et al. (2017) IAT WEAT Kai-Wei Chang (kw@kwchang.net) 38

  39. Word Embedding Association Test Caliskan et al. (2017) Kai-Wei Chang (kw@kwchang.net) 39 WEAT finds similar biases in Word Embeddings as IAT did for humans

  40. she he father mother king queen Kai-Wei Chang (kw@kwchang.net) 40

  41. Kai-Wei Chang (kw@kwchang.net) 41

  42. Can we Extend the Analysis beyond Binary Gender? Kai-Wei Chang (kw@kwchang.net) 42

  43. Beyond Gender & Race/Ethnicity Bias Manzini et al. NAACL 2019 Biases in word embeddings trained on Kai-Wei Chang (kw@kwchang.net) 43 the Reddit data from US users.

  44. How about other Embedding? Kai-Wei Chang (kw@kwchang.net) 44

  45. Bias Only in English? v Language with grammatical gender v Morphological agreement (Zhou et al, EMNLP 2019) Kai-Wei Chang (kw@kwchang.net) 45

  46. v Linear Discriminative Analysis (LDA) v Identify grammatical gender direction feminine words masculine words Kai-Wei Chang (kw@kwchang.net) 46

  47. masculine Female Male feminine Kai-Wei Chang (kw@kwchang.net) 47

  48. masculine Female Male feminine Kai-Wei Chang (kw@kwchang.net) 48

  49. masculine Female Male feminine Kai-Wei Chang (kw@kwchang.net) 49

  50. How about bilingual embedding? [Zhou et al. EMNLP19] Female doctor in Spanish male doctor in Spanish Kai-Wei Chang (kw@kwchang.net) 50

  51. How about Contextualized Representation? Gender Bias in Contextualized Word Embeddings Zhao et al. NAACL 19 v First two components explain more variance than others (Feminine) The driver stopped the car at the hospital because she was paid to do so (Masculine) The driver stopped the car at the hospital because he was paid to do so gender direction: ELMo(driver) – ELMo(driver) Kai-Wei Chang (kw@kwchang.net) 51

Recommend

Errors, biases and Quality control in Next Gen Sequencing Dr David Humphreys

Errors, biases and Quality control in Next Gen Sequencing Dr David Humphreys

Errors, biases and Quality control in Next Gen Sequencing Dr David Humphreys d.humphreys@victorchang.edu.au - Lab scientist : Bioinformatician - RNA biologist - small RNAs (miRNA) Victor Chang Cardiac Research Institute, Sydney, Australia

423 views • 26 slides
Behavioural models Cognitive biases Marcus Bendtsen Department of Computer and Information

Behavioural models Cognitive biases Marcus Bendtsen Department of Computer and Information

Behavioural models Cognitive biases Marcus Bendtsen Department of Computer and Information Science (IDA) Division for Database and Information Techniques (ADIT) Judgement under uncertainty Humans are not necessarily very good at estimating

435 views • 28 slides
Agenda Problems at building level What to do Hands on Who takes care of the control

Agenda Problems at building level What to do Hands on Who takes care of the control

Agenda Problems at building level What to do Hands on Who takes care of the control equipment? Who takes care of the substation? Some people have knowledge some don't. Some property owners have their own staff taking care of

574 views • 23 slides
Capital Budgeting: Biases (Welch, Chapter 13-5) Ivo Welch More Biases Overconfidence Are you

Capital Budgeting: Biases (Welch, Chapter 13-5) Ivo Welch More Biases Overconfidence Are you

Capital Budgeting: Biases (Welch, Chapter 13-5) Ivo Welch More Biases Overconfidence Are you overconfident? (Note that this can also affect proper volatility estimates in real options.) 90% Confidence Interval Give a 90% confidence interval

517 views • 13 slides
Access Control and Protection Overview Access control: What and Why Abstract Models of

Access Control and Protection Overview Access control: What and Why Abstract Models of

Access Control and Protection Overview Access control: What and Why Abstract Models of Access Control Discretionary acces control Mandatory access control Real systems: Unix Access Control Model Access control Access

817 views • 47 slides
Unconscious Bias 1 Questions to Start: Are we aware of our unconscious biases? Do we accept

Unconscious Bias 1 Questions to Start: Are we aware of our unconscious biases? Do we accept

Unconscious Bias 1 Questions to Start: Are we aware of our unconscious biases? Do we accept them? What types of biases are you aware of? Do you have experience with your own or seen biases in others? Activity: You are working

289 views • 14 slides
Breakout for small group discussions Topics Understanding your biases Awareness of

Breakout for small group discussions Topics Understanding your biases Awareness of

8/18/2020 Listen and Learn about Racial Justice 1 Presentation then Q&A Breakout for small group discussions Topics Understanding your biases Awareness of how biases affect behaviors Discerning your voice and your role

465 views • 13 slides
Investigating Potential Investigating Potential Biases in Aerosol Light Biases in Aerosol Light

Investigating Potential Investigating Potential Biases in Aerosol Light Biases in Aerosol Light

Investigating Potential Investigating Potential Biases in Aerosol Light Biases in Aerosol Light Absorption Absorption Measurements Measurements Christine Walsh Lund University (Lund, Sweden) NOAA ERSL (Aerosol Research Group) Overview of

354 views • 17 slides
Oil Market Outlook Whatever it takes - OPEC will do whatever it takes to get oil

Oil Market Outlook Whatever it takes - OPEC will do whatever it takes to get oil

Oil Market Outlook Whatever it takes - OPEC will do whatever it takes to get oil inventories down to the 5-year average DNB oil story in pictures & graphs June 2017 - Torbjrn Kjus Saudi Arabia Has Drawn Down 33% Of Its Foreign

277 views • 26 slides
Heuristics and biases Tina Nane 2 Heuristics and biases Lotto Icon by Dapete is

Heuristics and biases Tina Nane 2 Heuristics and biases Lotto Icon by Dapete is

1 Heuristics and biases Tina Nane 2 Heuristics and biases Lotto Icon by Dapete is licensed under CC BY 2.5 2 Heuristics and biases Heuristics Lotto Icon by Dapete is licensed under CC BY 2.5 Tricks, rules of thumb,

372 views • 23 slides
TEXT AND TEXT AND AUTOMATED BIASES AUTOMATED BIASES NATURAL LANGUAGES ARE THE NATURAL

TEXT AND TEXT AND AUTOMATED BIASES AUTOMATED BIASES NATURAL LANGUAGES ARE THE NATURAL

TEXT AND TEXT AND AUTOMATED BIASES AUTOMATED BIASES NATURAL LANGUAGES ARE THE NATURAL LANGUAGES ARE THE BASE HUMAN COMMUNICATION BASE HUMAN COMMUNICATION we learn from books of all kinds about complex topics and keep ourself updated

532 views • 51 slides
Biases in Decision Making Alexander Felfernig alexander.felfernig@ist.tugraz.at Decision Biases

Biases in Decision Making Alexander Felfernig alexander.felfernig@ist.tugraz.at Decision Biases

Institute for Software Technology International Workshop on Decision Making and Recommender Systems, Bolzano, 2014 Biases in Decision Making Alexander Felfernig alexander.felfernig@ist.tugraz.at Decision Biases & Recommender Systems 1

1.11k views • 53 slides
Spectral and Radiometric Issues for Level 1C L. Larrabee Strow and Scott Hannon Atmospheric

Spectral and Radiometric Issues for Level 1C L. Larrabee Strow and Scott Hannon Atmospheric

Introduction Scan Angle Biases Doppler Effect Biases IASI vs AIRS Biases Conclusions Spectral and Radiometric Issues for Level 1C L. Larrabee Strow and Scott Hannon Atmospheric Spectroscopy Laboratory (ASL) Physics Department and Joint

437 views • 20 slides
It Takes One Line (of SAS) to Analyze Your Data, but it Takes Many Lifelines to See it (Through)

It Takes One Line (of SAS) to Analyze Your Data, but it Takes Many Lifelines to See it (Through)

It Takes One Line (of SAS) to Analyze Your Data, but it Takes Many Lifelines to See it (Through) Shawn Murphy, Victor Castro, Wannapa Mahamaneerat, and Taowei David Wang Partners HealthCare October 30 th , 2012 Does Glargine Increase Cancer

211 views • 20 slides
Assessing Social and Intersectional Biases in Contextualized Word Representations Yi Chern Tan,

Assessing Social and Intersectional Biases in Contextualized Word Representations Yi Chern Tan,

Assessing Social and Intersectional Biases in Contextualized Word Representations Yi Chern Tan, L. Elisa Celis Yale University {yichern.tan, elisa.celis}@yale.edu Social Bias in Contextual Word Models Key Objectives: Do embedding

272 views • 16 slides
Why Data Flow Models? Models from Chapter 5 emphasized control Control flow graph, call

Why Data Flow Models? Models from Chapter 5 emphasized control Control flow graph, call

Why Data Flow Models? Models from Chapter 5 emphasized control Control flow graph, call graph, finite state machines We also need to reason about dependence Dependence and Data Flow Models Where does this value of x come from?

369 views • 9 slides
Pedestrian Models Based on Rational Behaviour CROWDS: Models and Control Centre International de

Pedestrian Models Based on Rational Behaviour CROWDS: Models and Control Centre International de

Pedestrian Models Based on Rational Behaviour CROWDS: Models and Control Centre International de Rencontres Mathmatiques Rafael Bailo , Jos A. Carrillo, Pierre Degond 3 rd June 2019 Department of Mathematics, Imperial College London 1

622 views • 27 slides
Two models for the control of sea lice infections using chemical treatments and biological

Two models for the control of sea lice infections using chemical treatments and biological

Population Approach Group in Europe (www.page-meeting.org) Glasgow 11-14 June 201 3 Two models for the control of sea lice infections using chemical treatments and biological control on farmed salmon populations George Gettinby Maya Groner

447 views • 26 slides
Control of redundant robots using learned models: an operational space control approach Presented

Control of redundant robots using learned models: an operational space control approach Presented

Control of redundant robots using learned models: an operational space control approach Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems October 12, 2009 / St Louis - USA by Camille Salan, Vincent Padois

827 views • 53 slides
EE3CL4: Models Feedback Introduction to Linear Control Systems systems Control

EE3CL4: Models Feedback Introduction to Linear Control Systems systems Control

EE 3CL4, PRWCR 1 / 14 Tim Davidson Instinct-based Design EE3CL4: Models Feedback Introduction to Linear Control Systems systems Control Post-Reading-Week Conceptual Review System Design Tim Davidson McMaster University Winter 2020

366 views • 10 slides
It Takes a Village to Heal Warriors- and It Takes the Warrior to Teach the Village How

It Takes a Village to Heal Warriors- and It Takes the Warrior to Teach the Village How

It Takes a Village to Heal Warriors- and It Takes the Warrior to Teach the Village How Understanding, Engaging and Providing Services to Our OEF/OIF Veteran Population Sherrill Valdes, LCSW, BCD Goals and Objectives Increase awareness of

789 views • 49 slides
Ge Getting tting Re Real al Ab About out Bia Biases ses an and Issues d Issues Re

Ge Getting tting Re Real al Ab About out Bia Biases ses an and Issues d Issues Re

Ge Getting tting Re Real al Ab About out Bia Biases ses an and Issues d Issues Re Related lated to to Bi Bias as Kenne neth h Roach, ch, Ed.D .D., ., LCMHC HC Wendy dy Thomas, mas, MS, , ACMHC MHC The Universi versity

269 views • 26 slides
11/27/2006 Massachusetts Institute of Technology Context Optimal predictive control of

11/27/2006 Massachusetts Institute of Technology Context Optimal predictive control of

11/27/2006 Massachusetts Institute of Technology Context Optimal predictive control of dynamic systems What is best sequence of control inputs that takes system state from Optimal, Robust Predictive the start to the goal? Control

407 views • 13 slides
Lecture 5 First Order Transfer Function Models Process Control Prof. Kannan M. Moudgalya IIT

Lecture 5 First Order Transfer Function Models Process Control Prof. Kannan M. Moudgalya IIT

Lecture 5 First Order Transfer Function Models Process Control Prof. Kannan M. Moudgalya IIT Bombay Tuesday, 30 July 2013 1/38 Process Control First Order Models Outline 1. Thermal sensor as a first order system 2. Scilab demonstration of

739 views • 38 slides

More recommend