how black box use of imputation can cause bias
play

How black-box use of imputation can cause bias Nicole Erler Erasmus - PowerPoint PPT Presentation

Jul 20, 2023 •579 likes •1.03k views

How black-box use of imputation can cause bias Nicole Erler Erasmus Medical Center, Rotterdam Nicole Erler, FGME 2019, Kiel 1 Handling Missing Values is Easy! Functions automatically exclude missing values: ## [...] ## Residual standard

  1. How black-box use of imputation can cause bias Nicole Erler Erasmus Medical Center, Rotterdam Nicole Erler, FGME 2019, Kiel 1

  2. Handling Missing Values is Easy! Functions automatically exclude missing values: ## [...] ## Residual standard error: 2.305 on 69 degrees of freedom ## (25 observations deleted due to missingness) ## Multiple R-squared: 0.09255, Adjusted R-squared: 0.02679 ## F-statistic: 1.407 on 5 and 69 DF, p-value: 0.2325 Nicole Erler, FGME 2019, Kiel 2

  3. Handling Missing Values is Easy! Functions automatically exclude missing values: ## [...] ## Residual standard error: 2.305 on 69 degrees of freedom ## (25 observations deleted due to missingness) ## Multiple R-squared: 0.09255, Adjusted R-squared: 0.02679 ## F-statistic: 1.407 on 5 and 69 DF, p-value: 0.2325 Imputation is super easy: library ("mice") imp <- mice (mydata) However ... Nicole Erler, FGME 2019, Kiel 2

  4. Handling Missing Values Correctly is Not So Easy! (Imputation) methods makes certain assumptions , e.g.: missingness is M(C)AR Nicole Erler, FGME 2019, Kiel 3

  5. Handling Missing Values Correctly is Not So Easy! (Imputation) methods makes certain assumptions , e.g.: missingness is M(C)AR the incomplete variable has a certain conditional distribution (e.g. normal) Nicole Erler, FGME 2019, Kiel 3

  6. Handling Missing Values Correctly is Not So Easy! (Imputation) methods makes certain assumptions , e.g.: missingness is M(C)AR the incomplete variable has a certain conditional distribution (e.g. normal) all associations are linear no interactions no non-linear effects no transformations Nicole Erler, FGME 2019, Kiel 3

  7. Handling Missing Values Correctly is Not So Easy! (Imputation) methods makes certain assumptions , e.g.: missingness is M(C)AR the incomplete variable has a certain conditional distribution (e.g. normal) all associations are linear no interactions no non-linear effects no transformations compatibility of the imputation models congeniality (compatibility between analysis and imputation models) Nicole Erler, FGME 2019, Kiel 3

  8. Handling Missing Values Correctly is Not So Easy! (Imputation) methods makes certain assumptions , e.g.: missingness is M(C)AR the incomplete variable has a certain conditional distribution (e.g. normal) all associations are linear no interactions no non-linear effects no transformations compatibility of the imputation models congeniality (compatibility between analysis and imputation models) violation ➡ bias Nicole Erler, FGME 2019, Kiel 3

  9. Literature: mis-specification in Multiple Imputation Several authors have investigated robustness to mis-specification (of distribution) in MI using FCS / MICE in joint model MI and/or proposed to use Tukey’s gh distribution Fleishman polynomials GAMs (in FCS) Doubly-robust weighted estimating equations (instead of MI) Nicole Erler, FGME 2019, Kiel 4

  10. Fully Bayesian Analysis & Imputation Joint distribution p ( y | X , b , θ ) p ( X | θ ) p ( b | θ ) p ( θ ) � �� � � �� � � �� � ���� imputation priors analysis random model part effects Nicole Erler, FGME 2019, Kiel 5

  11. Fully Bayesian Analysis & Imputation Joint distribution p ( y | X , b , θ ) p ( X | θ ) p ( b | θ ) p ( θ ) � �� � � �� � � �� � ���� imputation priors analysis random model part effects Imputation part X � �� � p ( x 1 , . . . , x p , X compl . | θ ) = p ( x 1 | X compl . , θ ) p ( x 2 | X compl . , x 1 , θ ) p ( x 3 | X compl . , x 1 , x 2 , θ ) . . . Nicole Erler, FGME 2019, Kiel 5

  12. Fully Bayesian Analysis & Imputation Joint distribution p ( y | X , b , θ ) p ( X | θ ) p ( b | θ ) p ( θ ) � �� � � �� � � �� � ���� imputation priors analysis random model part effects Imputation part X � �� � p ( x 1 , . . . , x p , X compl . | θ ) = p ( x 1 | X compl . , θ ) p ( x 2 | X compl . , x 1 , θ ) p ( x 3 | X compl . , x 1 , x 2 , θ ) . . . Software Implemented in the R package JointAI Nicole Erler, FGME 2019, Kiel 5

  13. MICE vs JointAI Imputation in MICE Imputation in JointAI p ( y | X compl . , x 1 , x 2 , x 3 , . . . , θ ) p ( x 1 | y , X compl . , x 2 , x 3 , x 4 , . . . , θ ) p ( x 1 | X compl . , θ ) p ( x 2 | y , X compl . , x 1 , x 3 , x 4 , . . . , θ ) p ( x 2 | X compl . , x 1 , θ ) p ( x 3 | y , X compl . , x 1 , x 2 , x 4 , . . . , θ ) p ( x 3 | X compl . , x 1 , x 2 , θ ) . . . . . . Nicole Erler, FGME 2019, Kiel 6

  14. MICE vs JointAI Imputation in MICE Imputation in JointAI p ( y | X compl . , x 1 , x 2 , x 3 , . . . , θ ) p ( x 1 | y , X compl . , x 2 , x 3 , x 4 , . . . , θ ) p ( x 1 | X compl . , θ ) p ( x 2 | y , X compl . , x 1 , x 3 , x 4 , . . . , θ ) p ( x 2 | X compl . , x 1 , θ ) p ( x 3 | y , X compl . , x 1 , x 2 , x 4 , . . . , θ ) p ( x 3 | X compl . , x 1 , x 2 , θ ) . . . . . . No issues with complex outcomes, e.g.: multi-level survival congeniality compatibility Nicole Erler, FGME 2019, Kiel 6

  15. MICE vs JointAI Imputation in MICE Imputation in JointAI p ( y | X compl . , x 1 , x 2 , x 3 , . . . , θ ) p ( x 1 | y , X compl . , x 2 , x 3 , x 4 , . . . , θ ) p ( x 1 | X compl . , θ ) p ( x 2 | y , X compl . , x 1 , x 3 , x 4 , . . . , θ ) p ( x 2 | X compl . , x 1 , θ ) p ( x 3 | y , X compl . , x 1 , x 2 , x 4 , . . . , θ ) p ( x 3 | X compl . , x 1 , x 2 , θ ) . . . . . . ➡ ➡ Potential mis-specification of association structure conditional distribution M(C)AR Nicole Erler, FGME 2019, Kiel 7

  16. Simulation Study: Quadratic Effect Analysis model: y ∼ β 0 + β 1 x 1 + β 2 x 2 + . . . Quadratic association between covariates: x 1 ∼ α 0 + α 1 x 2 + ❍❍ α 2 x 2 2 + . . . ❍ x 1 (incomplete covariate) 30 20 10 0 −10 −2 0 2 −2 0 2 −2 0 2 x 2 (complete covariate) Nicole Erler, FGME 2019, Kiel 8

  17. Simulation Study: Quadratic Effect Analysis model: y ∼ β 0 + β 1 x 1 + β 2 x 2 + . . . Quadratic association between covariates: x 1 ∼ α 0 + α 1 x 2 + ❍❍ α 2 x 2 2 + . . . ❍ x 1 (incomplete covariate) 30 20 10 0 −10 −2 0 2 −2 0 2 −2 0 2 x 2 (complete covariate) 2.0 10% NA 1.8 30% NA 1.6 relative bias 50% NA 1.4 1.2 1.0 0.8 0.6 mice JointAI mice JointAI mice JointAI Nicole Erler, FGME 2019, Kiel 8

  18. Simulation Study: Logarithmic Effect Analysis model: y ∼ β 0 + β 1 x 1 + β 2 x 2 + . . . Log-association between covariates: x 1 ∼ α 0 + α 1 ❩ log( x 2 ) + . . . ❩ x 1 (incomplete covariate) 4 0 −4 −8 −12 0.4 0.8 1.2 0.4 0.8 1.2 0.4 0.8 1.2 x 2 (complete covariate) 1.8 1.6 1.4 relative bias 1.2 1.0 0.8 0.6 10% NA 0.4 30% NA 0.2 50% NA mice JointAI mice JointAI mice JointAI Nicole Erler, FGME 2019, Kiel 9

  19. Simulation Study: Gamma distribution Analysis model: y ∼ β 0 + β 1 x 1 + β 2 x 2 + . . . Gamma-distributed covariate: x 1 | x 2 , x 3 , . . . ∼ Ga () conditional distribution 2.0 1.5 1.0 0.5 0.0 0 2 4 6 0 2 4 6 0 2 4 6 x 1 (incomplete covariate) 2.0 10% NA 1.8 30% NA relative bias 1.6 50% NA 1.4 1.2 1.0 0.8 mice JointAI mice JointAI mice JointAI Nicole Erler, FGME 2019, Kiel 10

  20. Flexible Bayesian Models We need more flexible imputation models! Ideally: models that fit (almost) any distribution / association structure. Nicole Erler, FGME 2019, Kiel 11

  21. Flexible Bayesian Models We need more flexible imputation models! Ideally: models that fit (almost) any distribution / association structure. Ideas: flexible association structure: penalized splines flexible residual distribution : mixture of Polya-Trees Nicole Erler, FGME 2019, Kiel 11

  22. Bayesian P-Splines d � Instead of β 1 x 2 we use β ℓ B ℓ ( x 2 ): ℓ =1 ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● x 1 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● x 2 Nicole Erler, FGME 2019, Kiel 12

Recommend

BIAS What Is Bias? Bias can be defined as favoring one side, position, or belief being

BIAS What Is Bias? Bias can be defined as favoring one side, position, or belief being

BIAS What Is Bias? Bias can be defined as favoring one side, position, or belief being partial or prejudiced Where have you seen bias? Bias vs. Propaganda Bias is prejudice; a preconceived judgment or an opinion formed

717 views • 29 slides
Overview Multiple Imputation for Multilevel Data Bayesian estimation for MLMs Univariate

Overview Multiple Imputation for Multilevel Data Bayesian estimation for MLMs Univariate

Overview Multiple Imputation for Multilevel Data Bayesian estimation for MLMs Univariate multiple imputation Session 1 Craig K. Enders Brian T. Keller Joint model imputation University of California - Los Angeles Fully conditional

901 views • 45 slides
Black Infant-Maternal Health: Focusing on Implicit Bias in Healthcare By Lydia King Advisor:

Black Infant-Maternal Health: Focusing on Implicit Bias in Healthcare By Lydia King Advisor:

Black Infant-Maternal Health: Focusing on Implicit Bias in Healthcare By Lydia King Advisor: Christine Stevens Research Question &amp; Thesis Question: Why is there such a high instance of maternal-infant mortality for black women? =

421 views • 10 slides
Method for the imputation of the earnings variable in the Belgian LFS Workshop on LFS

Method for the imputation of the earnings variable in the Belgian LFS Workshop on LFS

Method for the imputation of the earnings variable in the Belgian LFS Workshop on LFS methodology, Madrid 2012, May 10-11 Astrid Depickere, Anja Termote, Pieter Vermeulen http://economie.fgov.be Outline 1. Introduction 2. Imputation 3.

291 views • 11 slides
Equity &amp; Excellence: Hidden Bias Implicit Bias Inherent Bias

Equity &amp; Excellence: Hidden Bias Implicit Bias Inherent Bias

Equity &amp; Excellence: Hidden Bias Implicit Bias Inherent Bias ____________________________________________ Vincent R. De Lucia Educator-in-Residence Director of Mandatory Training &amp; PD New Jersey School Boards Association Anderson

426 views • 41 slides
Is there bias in the estimated climate forcing by black carbon aerosols? John Ogren 1 Elisabeth

Is there bias in the estimated climate forcing by black carbon aerosols? John Ogren 1 Elisabeth

Is there bias in the estimated climate forcing by black carbon aerosols? John Ogren 1 Elisabeth Andrews 1,2 1 NOAA Earth System Research Laboratory 2 Univ. of Colorado Boulder, Colorado, USA Global Monitoring Annual Conference Boulder, CO, May

664 views • 35 slides
bad decision bias? 2 5 6 Affinity Bias: What can we do? 2. 3. 1. Seek

bad decision bias? 2 5 6 Affinity Bias: What can we do? 2. 3. 1. Seek

bad decision bias? 2 5 6 Affinity Bias: What can we do? 2. 3. 1. Seek outputs Compare rather than Be aware performance compentenci not people es 7 9 10 11 12 Unconscious bias refers to a

292 views • 15 slides
Consistent Variance Estimates for Multiple Multiple imputation Imputation in R MI alternative

Consistent Variance Estimates for Multiple Multiple imputation Imputation in R MI alternative

Consistent MI Variances in R James Reilly Consistent Variance Estimates for Multiple Multiple imputation Imputation in R MI alternative R package Summary James Reilly University of Auckland 8 July 2009 James Reilly Consistent MI

600 views • 11 slides
Genotype imputation accuracy with different reference panels Guan-Hua Huang and Yi-Chi Tseng

Genotype imputation accuracy with different reference panels Guan-Hua Huang and Yi-Chi Tseng

Genotype imputation accuracy with different reference panels Guan-Hua Huang and Yi-Chi Tseng National Chiao Tung University TAIWAN Genotype imputation Background GWAS based on common SNPs have only identified a small fraction of the

836 views • 17 slides
BIAS BIAS LIGHT LIGHT &amp; &amp; MEDIUM MEDIUM TR TRUCK UCK TIRES TIRES Bias Bias Ligh

BIAS BIAS LIGHT LIGHT &amp; &amp; MEDIUM MEDIUM TR TRUCK UCK TIRES TIRES Bias Bias Ligh

BIAS BIAS LIGHT LIGHT &amp; &amp; MEDIUM MEDIUM TR TRUCK UCK TIRES TIRES Bias Bias Ligh ight T t Truck / / Trail iler r Ti Tire Si Size No Nome mencla latu ture 9.50-16 9.50 16.5L .5LT/D T/D ST ST 7.00 7.00-15 15 1 10

121 views • 11 slides
Bias in, Bias out: Gender Equality and the Fourth Industrial Revolution Debra Howcroft and

Bias in, Bias out: Gender Equality and the Fourth Industrial Revolution Debra Howcroft and

Bias in, Bias out: Gender Equality and the Fourth Industrial Revolution Debra Howcroft and Jill Rubery Work and Equalities Institute University of Manchester Bias in, bias out: an overview Two parts Identify the methodology

706 views • 21 slides
Genotype Imputation in Genome-wide Association Studies Fernando Rivadeneira 1,2 1 Department of

Genotype Imputation in Genome-wide Association Studies Fernando Rivadeneira 1,2 1 Department of

Genotype Imputation in Genome-wide Association Studies Fernando Rivadeneira 1,2 1 Department of Internal Medicine 2 Department of Epidemiology Course SNPs and Human Diseases Rotterdam November 12 th , 2018 Imputation Facilitates

735 views • 45 slides
Expectancy bias and Bias and forensic evidence Bias and speech research forensic speech

Expectancy bias and Bias and forensic evidence Bias and speech research forensic speech

Overview Bias effects Expectancy bias and Bias and forensic evidence Bias and speech research forensic speech research Blind forensic speech research Maartje Schreuder TMFI / Maastricht University I AFPA 2011 Bundeskrim

180 views • 7 slides
Implicit Bias Implicit bias Implicit bias refers to attitudes or stereotypes that affect our

Implicit Bias Implicit bias Implicit bias refers to attitudes or stereotypes that affect our

Implicit Bias Implicit bias Implicit bias refers to attitudes or stereotypes that affect our understanding, actions and decisions in an unconscious manner. It's different from suppressed thoughts we might conceal to keep the peace; it's

634 views • 16 slides
Bias Busting Across the Center: A Model to Interrupt Bias and Promote Inclusion Modeled on

Bias Busting Across the Center: A Model to Interrupt Bias and Promote Inclusion Modeled on

Bias Busting Across the Center: A Model to Interrupt Bias and Promote Inclusion Modeled on Googles BiasBusting@Work and Carnegie Mellons Bias Busters @ CMU Learning Outcomes Understand the role that biases play, positively &amp;

637 views • 34 slides
Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff

Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff

Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff 1 n=1 Cramr-Rao bound n=4 0.8 n=30 0.6 CDF of max of n samples of 0.4 N( =2, 2 =1) 0.2 [representing error estimates for n

517 views • 28 slides
Unconscious Bias: From Awareness to Action! Did you know that we all have unconscious bias, and

Unconscious Bias: From Awareness to Action! Did you know that we all have unconscious bias, and

Unconscious Bias: From Awareness to Action! Did you know that we all have unconscious bias, and that we each have a responsibility to prevent its unintended consequences? In this course, we will explore unconscious bias in the workplace and the

213 views • 5 slides
Handling missing data in Stata: Imputation and likelihood-based approaches Rose Medeiros

Handling missing data in Stata: Imputation and likelihood-based approaches Rose Medeiros

Introduction Multiple Imputation Full information maximum likelihood Conclusion Handling missing data in Stata: Imputation and likelihood-based approaches Rose Medeiros StataCorp LP 2016 Swiss Stata Users Group meeting Medeiros Handling

841 views • 33 slides
Bias and Equity Implicit Bias &amp; Health Care Disparities Two sides of the same coin Clinical

Bias and Equity Implicit Bias &amp; Health Care Disparities Two sides of the same coin Clinical

3/2/2018 Implicit Bias, Race and Medical Care We have nothing to disclose Kate Lupton, MD Sarah Schaeffer, MD, MPH University of California, San Francisco Bias and Equity Implicit Bias &amp; Health Care Disparities Two sides of the same

394 views • 11 slides
Reference based multiple imputation; for sensitivity analysis of clinical trials with missing

Reference based multiple imputation; for sensitivity analysis of clinical trials with missing

Reference based multiple imputation; for sensitivity analysis of clinical trials with missing data Suzie Cro MRC Clinical Trials Unit at UCL The London School of Hygiene and Tropical Medicine Outline Reference based multiple imputation;

586 views • 39 slides
Transistor bias circuits 1 Objectives Discuss the concept of dc biasing of a transistor for

Transistor bias circuits 1 Objectives Discuss the concept of dc biasing of a transistor for

Chapter 5 Transistor bias circuits 1 Objectives Discuss the concept of dc biasing of a transistor for linear operation Analyze voltage-divider bias, base bias, emitter bias and collector-feedback bias circuits. and collector-feedback

588 views • 31 slides
PVMD Miro Zeman Delft University of Technology P-N Junction under Bias - + V a I I A P N

PVMD Miro Zeman Delft University of Technology P-N Junction under Bias - + V a I I A P N

The Dark J-V Curve PVMD Miro Zeman Delft University of Technology P-N Junction under Bias - + V a I I A P N C I d Anode Cathode Bias Conditions Forward bias Reverse bias J-V Characteristic Shockley equation: Dark J-V

347 views • 5 slides
Hindsight Bias of Juries in Hindsight Bias of Juries in Personal Injury Actions Courtroom

Hindsight Bias of Juries in Hindsight Bias of Juries in Personal Injury Actions Courtroom

Presenting a live 90 minute webinar with interactive Q&amp;A Hindsight Bias of Juries in Hindsight Bias of Juries in Personal Injury Actions Courtroom Strategies to Minimize Negative Effects of Hindsight Bias THURS DAY, AUGUS T 25, 2011 1pm

861 views • 83 slides
Billion Word Imputation Guide: Prof. Amitabha Mukherjee Group 20: Aayush Mudgal [12008] Shruti

Billion Word Imputation Guide: Prof. Amitabha Mukherjee Group 20: Aayush Mudgal [12008] Shruti

CS365 Course Project Billion Word Imputation Guide: Prof. Amitabha Mukherjee Group 20: Aayush Mudgal [12008] Shruti Bhargava [13671] Problem Statement Problem Description : https://www.kaggle.com/c/billion-word-imputation Examples : 1.

473 views • 15 slides

More recommend