Unpacking the Black-Box of Causality: Learning about Causal - PowerPoint PPT Presentation

Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies Kosuke Imai Princeton University November 2, 2011 Joint work with L. Keele (Penn State) D. Tingley (Harvard) T. Yamamoto (MIT) Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 1 / 27

Quantitative Research and Causal Mechanisms Causal inference is a central goal of scientific research Scientists care about causal mechanisms , not just causal effects Randomized experiments often only determine whether the treatment causes changes in the outcome Not how and why the treatment affects the outcome Common criticism of experiments and statistics: black box view of causality Qualitative research uses process tracing Question: How can quantitative research be used to identify causal mechanisms? Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 2 / 27

Overview of the Talk Goal: Convince you that statistics can be useful for learning about causal mechanisms Method: Causal Mediation Analysis Mediator, M Treatment, T Outcome, Y Direct and indirect effects; intermediate and intervening variables New tools: framework, estimation algorithm, sensitivity analysis, research designs, easy-to-use software Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 3 / 27

Causal Mediation Analysis in American Politics The political psychology literature on media framing Nelson et al . ( APSR , 1998) Popular in social psychology Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 4 / 27

Causal Mediation Analysis in Comparative Politics Resource curse thesis Authoritarian government civil war Natural Slow growth resources Causes of civil war: Fearon and Laitin ( APSR , 2003) Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 5 / 27

Causal Mediation Analysis in International Relations The literature on international regimes and institutions Krasner ( International Organization , 1982) Power and interests are mediated by regimes Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 6 / 27

Current Practice in Political Science Regression: Y i = α + β T i + γ M i + δ X i + ǫ i Each coefficient is interpreted as a causal effect Sometimes, it’s called marginal effect Idea: increase T i by one unit while holding M i and X i constant But, if you change T i , that may also change M i The Problem: Post-treatment bias Usual advice: only include causally prior (or pre-treatment) variables But, then you lose causal mechanisms! Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 7 / 27

Formal Statistical Framework of Causal Inference Units: i = 1 , . . . , n “Treatment”: T i = 1 if treated, T i = 0 otherwise Pre-treatment covariates: X i Potential outcomes: Y i ( 1 ) and Y i ( 0 ) Observed outcome: Y i = Y i ( T i ) Voters Contact Turnout Age Party ID i T i Y i ( 1 ) Y i ( 0 ) X i X i 1 1 1 ? 20 D 2 0 ? 0 55 R . . . . . . . . . . . . . . . . . . n 1 0 ? 62 D Causal effect: Y i ( 1 ) − Y i ( 0 ) Problem: only one potential outcome can be observed per unit Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 8 / 27

Potential Outcomes Framework for Mediation Binary treatment: T i Pre-treatment covariates: X i Potential mediators: M i ( t ) Observed mediator: M i = M i ( T i ) Potential outcomes: Y i ( t , m ) Observed outcome: Y i = Y i ( T i , M i ( T i )) Again, only one potential outcome can be observed per unit Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 9 / 27

Causal Mediation Effects Total causal effect: τ i ≡ Y i ( 1 , M i ( 1 )) − Y i ( 0 , M i ( 0 )) Causal mediation (Indirect) effects: δ i ( t ) ≡ Y i ( t , M i ( 1 )) − Y i ( t , M i ( 0 )) Causal effect of the treatment-induced change in M i on Y i Change the mediator from M i ( 0 ) to M i ( 1 ) while holding the treatment constant at t Represents the mechanism through M i Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 10 / 27

Total Effect = Indirect Effect + Direct Effect Direct effects: ζ i ( t ) ≡ Y i ( 1 , M i ( t )) − Y i ( 0 , M i ( t )) Causal effect of T i on Y i , holding mediator constant at its potential value that would be realized when T i = t Change the treatment from 0 to 1 while holding the mediator constant at M i ( t ) Represents all mechanisms other than through M i Total effect = mediation (indirect) effect + direct effect: τ i = δ i ( t ) + ζ i ( 1 − t ) = 1 2 { δ i ( 0 ) + δ i ( 1 ) + ζ i ( 0 ) + ζ i ( 1 ) } Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 11 / 27

What Does the Observed Data Tell Us? Quantity of Interest: Average causal mediation effects (ACME) ¯ δ ( t ) ≡ E ( δ i ( t )) = E { Y i ( t , M i ( 1 )) − Y i ( t , M i ( 0 )) } Average direct effects ( ¯ ζ ( t ) ) are defined similarly Y i ( t , M i ( t )) is observed but Y i ( t , M i ( t ′ )) can never be observed We have an identification problem = ⇒ Need additional assumptions to make progress Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 12 / 27

Identification under Sequential Ignorability Proposed identification assumption: Sequential Ignorability (SI) { Y i ( t ′ , m ) , M i ( t ) } ⊥ ⊥ T i | X i = x , (1) Y i ( t ′ , m ) ⊥ ⊥ M i ( t ) | T i = t , X i = x (2) (1) is guaranteed to hold in a standard experiment (2) does not hold unless X i includes all confounders Limitation: X i cannot include post-treatment confounders Under SI, ACME is nonparametrically identified: � � E ( Y i | M i , T i = t , X i ) { dP ( M i | T i = 1 , X i ) − dP ( M i | T i = 0 , X i ) } dP ( X i ) Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 13 / 27

Example: Anxiety, Group Cues and Immigration Brader, Valentino & Suhat (2008, AJPS) How and why do ethnic cues affect immigration attitudes? Theory: Anxiety transmits the effect of cues on attitudes Anxiety, M Immigration Attitudes, Y Media Cue, T ACME = Average difference in immigration attitudes due to the change in anxiety induced by the media cue treatment Sequential ignorability = No unobserved covariate affecting both anxiety and immigration attitudes Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 14 / 27

Traditional Estimation Method Linear structural equation model (LSEM): α 2 + β 2 T i + ξ ⊤ = 2 X i + ǫ i 2 , M i α 3 + β 3 T i + γ M i + ξ ⊤ Y i = 3 X i + ǫ i 3 . Fit two least squares regressions separately Use product of coefficients ( ˆ β 2 ˆ γ ) to estimate ACME The method is valid under SI Can be extended to LSEM with interaction terms Problem: Only valid for the simplest LSEMs Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 15 / 27

Proposed General Estimation Algorithm Model outcome and mediator 1 Outcome model: p ( Y i | T i , M i , X i ) Mediator model: p ( M i | T i , X i ) These models can be of any form (linear or nonlinear, semi- or nonparametric, with or without interactions) Predict mediator for both treatment values ( M i ( 1 ) , M i ( 0 ) ) 2 Predict outcome by first setting T i = 1 and M i = M i ( 0 ) , and then 3 T i = 1 and M i = M i ( 1 ) Compute the average difference between two outcomes to obtain 4 a consistent estimate of ACME Monte Carlo or bootstrap to estimate uncertainty 5 Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 16 / 27

Example: Estimation under Sequential Ignorability Original method: Product of coefficients with the Sobel test — Valid only when both models are linear w/o T – M interaction (which they are not) Our method: Calculate ACME using our general algorithm Product of Average Causal Outcome variables Coefficients Mediation Effect ( δ ) Decrease Immigration . 347 . 105 ¯ δ ( 1 ) [ 0 . 146 , 0 . 548 ] [ 0 . 048 , 0 . 170 ] Support English Only Laws . 204 . 074 ¯ δ ( 1 ) [ 0 . 069 , 0 . 339 ] [ 0 . 027 , 0 . 132 ] Request Anti-Immigration Information . 277 . 029 ¯ δ ( 1 ) [ 0 . 084 , 0 . 469 ] [ 0 . 007 , 0 . 063 ] Send Anti-Immigration Message . 276 . 086 ¯ δ ( 1 ) [ 0 . 102 , 0 . 450 ] [ 0 . 035 , 0 . 144 ] Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 17 / 27

Need for Sensitivity Analysis Even in experiments, SI is required to identify mechanisms SI is often too strong and yet not testable Need to assess the robustness of findings via sensitivity analysis Question: How large a departure from the key assumption must occur for the conclusions to no longer hold? Sensitivity analysis by assuming { Y i ( t ′ , m ) , M i ( t ) } ⊥ ⊥ T i | X i = x but not Y i ( t ′ , m ) ⊥ ⊥ M i ( t ) | T i = t , X i = x Possible existence of unobserved pre-treatment confounder Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 18 / 27

Parametric Sensitivity Analysis Sensitivity parameter: ρ ≡ Corr ( ǫ i 2 , ǫ i 3 ) Sequential ignorability implies ρ = 0 Set ρ to different values and see how ACME changes When do my results go away completely? ¯ δ ( t ) = 0 if and only if ρ = Corr ( ǫ i 1 , ǫ i 2 ) where Y i = α 1 + β 1 T i + ǫ i 1 Easy to estimate from the regression of Y i on T i : Alternative interpretation based on R 2 : How big does the effects of unobserved confounders have to be in order for my results to go away? Kosuke Imai (Princeton) Causal Mechanisms UVA (November 2, 2011) 19 / 27