Comparing Causal Inference Estimators for Average Treatment Effect - PowerPoint PPT Presentation

Comparing Causal Inference Estimators for Average Treatment Effect of Treated Units in Observational Studies Kip Brown kibrown@siue.edu Department of Mathematics and Statistics Southern Illinois University Edwardsville March 6, 2018 Kip Brown (SIUE) Causal Inference March 6, 2018 1 / 55

Overview Problem Set Up 1 Definitions and Assumptions Propensity Score Framework Estimating the Propensity Score Covariate Balancing Propensity Score Causal Inference Methods 2 Matching Methods Stratification Inverse Probability of Treatment Weighting Entropy Balancing Simulations 3 Empirical Study 4 Kip Brown (SIUE) Causal Inference March 6, 2018 2 / 55

Problem Set Up Randomized vs. Observational Studies Randomized studies balance covariate distribution by design Observational studies may have unbalanced covariate distributions This leads to biased estimates Kip Brown (SIUE) Causal Inference March 6, 2018 3 / 55

Problem Set Up Definitions and Assumptions The Response Function Definition Suppose we have a random sample of size n from a population. For the i th unit in the sample, let T i denote which treatment was received, where T i = 0 denotes the i th unit receiving the control treatment, and T i = 1 denote the i th unit receiving the treatment of interest. Let Y i (0) and Y i (1) denote the outcomes of the control treatment and the treatment of interest, respectively. Let Y i = T i Y i (1) + (1 − T i ) Y i (0) (1) denote the response of the i th unit. Kip Brown (SIUE) Causal Inference March 6, 2018 4 / 55

Problem Set Up Definitions and Assumptions Treatment Effects Definition Let the average treatment effect ( ATE ) be defined as τ = E [ Y i (1) − Y i (0)] . Let the average treatment effect for the treated ( ATT ) be defined as, τ t = E [ Y i (1) − Y i (0) | T i = 1] . (2) Kip Brown (SIUE) Causal Inference March 6, 2018 5 / 55

Problem Set Up Definitions and Assumptions Major Assumptions Assumption (Unconfoundedness) For any unit i = 1 , . . . , n , P ( T i = 1 | Y i (0) , Y i (1) , X i ) = P ( T i = 1 | X i ) (3) or, using conditional independence notation T i ⊥ ⊥ ( Y i (0) , Y i (1)) | X i Assumption (Probabilistic Assignment) For any unit i = 1 , . . . , n , 0 < P ( T i = 1 | X i ) < 1 Kip Brown (SIUE) Causal Inference March 6, 2018 6 / 55

Problem Set Up Definitions and Assumptions Major Assumptions Continued Assumption (Individualistic) For any unit i = 1 , . . . , n , the probability of treatment assignment can be written as a common function of the i th ’s unit potential outcome and observed covariates. Kip Brown (SIUE) Causal Inference March 6, 2018 7 / 55

Problem Set Up Propensity Score Framework Balancing Scores Definition (Balancing Score) A balancing score b ( x ) is a function of the covariates such that T i ⊥ ⊥ X i | b ( Xi ) . This can also be represented as a probability, P ( T i = 1 | X i , b ( X i )) = P ( T i = 1 | b ( X i )) . (4) ie: X i is a balancing score Kip Brown (SIUE) Causal Inference March 6, 2018 8 / 55

Problem Set Up Propensity Score Framework A Better Balancing Score Definition (Propensity Score) The Propensity Score is the conditional probability that a unit with observed covariates, x , will be in treatment group 1. The Propensity Score π ( X i ) is then, π ( X i ) = P ( T i = 1 | X i = x ) . (5) Kip Brown (SIUE) Causal Inference March 6, 2018 9 / 55

Problem Set Up Propensity Score Framework Propensity Score Theorems Theorem (Propensity Score is a balancing score) The propensity score π ( X i ) = P ( T = 1 | X i = x ) is a balancing score. Kip Brown (SIUE) Causal Inference March 6, 2018 10 / 55

Problem Set Up Propensity Score Framework Theorem 1 Proof Proof. We must show that the propensity score is a balancing score, which by equation (4) , P ( T i = 1 | X i , π ( X i )) = P ( T i = 1 | π ( X i )) . (6) Starting with the left side of (6), we have P ( T i = 1 | X i , π ( X i )) = P ( T i = 1 | X i ) = π ( X i ) . Kip Brown (SIUE) Causal Inference March 6, 2018 11 / 55

Problem Set Up Propensity Score Framework Proof Continued Proof. Now with the right side of (6), we have P ( T i = 1 | π ( X i )) = 1 · P ( T i = 1 | π ( X i )) + 0 = 1 · P ( T i = 1 | π ( X i )) + 0 · P ( T i = 0 | π ( X i )) = E T [ T i | π ( X i )] � � = E X E T [ T i | X i , π ( X i )] | π ( X i ) � � = E X P ( T i | X i , π ( X i )) | π ( X i ) � � = E X π ( X i ) | π ( X i ) = π ( X i ) . Thus, π ( X i ) is a balancing score. Kip Brown (SIUE) Causal Inference March 6, 2018 12 / 55

Problem Set Up Propensity Score Framework Propensity Score Theorems Theorem (Unconfoundedness given any balancing score) Suppose Assumption 1 is true. Then, treatment assignment is unconfounded given any balancing score, P ( T i = 1 | Y i (0) , Y i (1) , b ( X i )) = P ( T i = 1 | b ( X i )) (7) or, using conditional independence notation T i ⊥ ⊥ ( Y i (0) , Y i (1)) | b ( X i ) . Kip Brown (SIUE) Causal Inference March 6, 2018 13 / 55

Problem Set Up Propensity Score Framework Theorem 2 Proof Proof. Let Assumption 1 be true. We will start with the left side of (7) , and show the right. P ( T i = 1 | Y i (0) , Y i (1) , b ( X i )) = E T [ T i | Y i (0) , Y i (1) , b ( X i )] � � = E X E T [ T i | Y i (0) , Y i (1) , X i , b ( X i )] | Y i (0) , Y i (1) , b ( X i ) � � = E X E T [ T i | X i , b ( X i )] | Y i (0) , Y i (1) , b ( X i ) � � = E X E T [ T i | b ( X i )] | Y i (0) , Y i (1) , b ( X i ) = E X [ T i | b ( X i )] = 1 · P ( T i = 1 | b ( X i )) + 0 · P ( T i = 0 | b ( X i )) = P ( T i = 1 | b ( X i )) . Kip Brown (SIUE) Causal Inference March 6, 2018 14 / 55

Problem Set Up Estimating the Propensity Score Estimating the Propensity Score The true propensity score is unknown π ( X i ) = P ( T i = 1 | X i ) can be modeled with logistic regression Definition The binary logistic regression response function is exp( X ′ i β ) π ( X i ) = i β ) , (8) 1 + exp( X ′ where X i is vector of covariates for the i th unit, and β is the vector of parameters. Kip Brown (SIUE) Causal Inference March 6, 2018 15 / 55

Problem Set Up Estimating the Propensity Score Likelihood Function T i is a Bernoulli random variable n � π ( X i ) T i · (1 − π ( X i )) 1 − T i L ( β ) = i =1 � n � π ( X i ) T i · (1 − π ( X i )) 1 − T i � l = ln( L ( β )) = ln i =1 n � � π ( X i ) T i · (1 − π ( X i )) 1 − T i � = ln i =1 n � � � = T i · ln( π ( X i )) + (1 − T i ) · ln(1 − π ( X i )) . i =1 Kip Brown (SIUE) Causal Inference March 6, 2018 16 / 55

Problem Set Up Estimating the Propensity Score Estimated Propensity Score Using the MLE method to estimate the parameters, exp( X ′ i b ) π ( X i ) = ˆ 1 + exp( X ′ i b ) or equivalently, � π ( X i ) ˆ � = X i b ln 1 − ˆ π ( X i ) Kip Brown (SIUE) Causal Inference March 6, 2018 17 / 55

Problem Set Up Estimating the Propensity Score Measures of Model Accuracy Definition Let l r and l p be the log-likelihood functions for the reduced model and the proposed model respectively. Then the likelihood ratio statistic, D, is D = 2[ l p − l r ] Kip Brown (SIUE) Causal Inference March 6, 2018 18 / 55

Problem Set Up Estimating the Propensity Score Estimating the Propensity Score in R Include all scientifically significant predictors 1 Include all statistically significant first order terms 2 Include all statistically significant second order terms 3 Kip Brown (SIUE) Causal Inference March 6, 2018 19 / 55

Problem Set Up Estimating the Propensity Score Choosing Statistically Significant Terms Suppose there are p variables in the data set, Fit a base model with all a scientifically significant predictors 1 Fit p - a new models, each with the scientifically significant 2 predictors, plus 1 of the remaining variables. Calculate the likelihood ratio statistic, D, for each model 3 If any D ≥ 1 , add that predictor variable to the base model, and 4 repeat steps 2 - 4 When all D < 1 , add no more first order predictor variables 5 Kip Brown (SIUE) Causal Inference March 6, 2018 20 / 55

Problem Set Up Estimating the Propensity Score Choosing Statistically Significant Terms Choosing second order terms follows a very similar logic Only consider second order terms that include variables already in the model If any D ≥ 2 . 71 , add it to the model, if not move on Kip Brown (SIUE) Causal Inference March 6, 2018 21 / 55

Problem Set Up Estimating the Propensity Score Issues with the Propensity Score The propensity score must be correctly modeled The goal of a logistic regression model is accurate prediction of P ( T i = 1) Iterative processes can be lengthy and difficult Kip Brown (SIUE) Causal Inference March 6, 2018 22 / 55

Problem Set Up Covariate Balancing Propensity Score Covariate Balancing Propensity Score The goal is to balance the covariates This is not an iterative process Parameter estimates are derived by the MLE method with the following balancing condition n 1 T i − (1 − T i ) π ( X i ) � � � ˜ X i = 0 n 1 1 − π ( X i ) i =1 i ) T ) T X i = X i or ˜ ˜ i ( X 2 X i = ( X T Kip Brown (SIUE) Causal Inference March 6, 2018 23 / 55

Causal Inference Methods Estimation Methods Matching Methods 1 Stratification Methods 2 Inverse Probability Methods 3 Entropy Balancing Methods 4 Kip Brown (SIUE) Causal Inference March 6, 2018 24 / 55