Primal-dual Covariate Balance and Minimal Double Robustness via Entropy Balancing Qingyuan Zhao (Joint work with Daniel Percival) Department of Statistics, Stanford University JSM, August 9, 2015
Outline Entropy Balancing Qingyuan Zhao Background Background 1 Results for EB Equivalence of PS and OR References Results for EB 2 Equivalence of PS and OR 3 1/18
Setting Entropy Balancing Qingyuan Zhao Rubin’s causal model Background Consider an observational study: Results for EB Equivalence of Treatment assignment: T ∈ { 0 , 1 } ; PS and OR References Potential outcomes: Y (0), Y (1); Pre-treatment covariates: X ; No hidden bias: ( Y (0) , Y (1)) ⊥ ⊥ T | X . Overlap: 0 < P ( T = 1 | X ) = e ( X ) < 1. 2/18
Covariate balance and propensity score Entropy Balancing Covariate balance plays a crucial rule in observational study: Qingyuan Zhao � e ( X ) � � � E [ c ( X ) | T = 1] = E 1 − e ( X ) c ( X ) � T = 0 , ∀ c ( X ) . Background � Results for EB Equivalence of Rosenbaum and Rubin (1983): any balancing score is a PS and OR function of propensity score (PS). References 3/18
Covariate balance and propensity score Entropy Balancing Covariate balance plays a crucial rule in observational study: Qingyuan Zhao � e ( X ) � � � E [ c ( X ) | T = 1] = E 1 − e ( X ) c ( X ) � T = 0 , ∀ c ( X ) . Background � Results for EB Equivalence of Rosenbaum and Rubin (1983): any balancing score is a PS and OR function of propensity score (PS). References In practice, PS model is subject to misspecification. Propensity score tautology (Imai et al., 2008) Iterate between 1 Modeling propensity score 2 Checking covariate balance. 3/18
Entropy Balancing (Hainmueller, 2011) Entropy balancing (EB) is a one-step solution of the tautology. Entropy Balancing Qingyuan Zhao � − w i log w i maximize w T i =0 Background c j (1) = 1 Results for EB � � subject to w i c j ( X i ) = ¯ c j ( X i ) , j = 1 , . . . , p , n 1 Equivalence of T i =0 T i =1 PS and OR � References w i = 1 , T i =0 w i > 0 , i = 1 , . . . , n . EB estimates the average treatment effect on the treated γ = E [ Y (1) | T = 1] − E [ Y (0) | T = 1] by Y i γ EB = � � w EB ˆ − Y i . i n 1 T i =1 T i =0 4/18
This talk Entropy Balancing Qingyuan Zhao Background EB was proposed purely from an applied perspective and is Results for EB very easy to interpret, but is it actually safe to use EB? Equivalence of PS and OR We give theoretical justifications for entropy balancing: References EB has a “minimal” double robustness property. Elegant correspondence between primal-dual optimization and double robustness. 5/18
Outline Entropy Balancing Qingyuan Zhao Background Background 1 Results for EB Equivalence of PS and OR References Results for EB 2 Equivalence of PS and OR 3 6/18
Heuristics Entropy Balancing Let m ( x ) be the density of X for the control population. Qingyuan Zhao Minimum relative entropy principle Background Estimate the density of the treatment population by Results for EB Equivalence of H ( ˜ m � m ) s . t . E ˜ m [ c ( X )] = ¯ c (1) . (1) PS and OR maximize m ˜ References where H ( ˜ m � m ) = E ˜ m [log( ˜ m ( X ) / m ( X ))] is the relative entropy. 7/18
Heuristics Entropy Balancing Let m ( x ) be the density of X for the control population. Qingyuan Zhao Minimum relative entropy principle Background Estimate the density of the treatment population by Results for EB Equivalence of H ( ˜ m � m ) s . t . E ˜ m [ c ( X )] = ¯ c (1) . (1) PS and OR maximize m ˜ References where H ( ˜ m � m ) = E ˜ m [log( ˜ m ( X ) / m ( X ))] is the relative entropy. The optimization (1) is equivalent to maximize E m [ w ( X ) log w ( X )] s . t . E m [ w ( X ) c ( X )] = ¯ c (1) . w EB is the finite sample version of this problem. 7/18
Exponential tilting Entropy Balancing Qingyuan Zhao The solution to (1) belongs to the family of exponential titled distributions of m (Cover and Thomas, 2012): Background Results for EB m θ ( x ) = m ( x ) exp( θ T c ( x ) − ψ ( θ )) . Equivalence of PS and OR References By Bayes’ formula, this implies a logistic PS model P ( T = 1 | X = x ) P ( T = 0 | X = x ) = w ( x ) = exp( α + θ T c ( x )) Intuitively, EB solves the logistic regression by a criterion different than the MLE. 8/18
“Minimal” double robustness Entropy Balancing Qingyuan Zhao Theorem (Zhao and Percival, 2015) Background Assume there is no hidden bias, the expectation of c ( X ) exists Results for EB and Var ( Y (0)) < ∞ . Let e ( X ) = P ( T = 1 | X ) and Equivalence of PS and OR g t ( X ) = E [ Y ( t ) | X ]. Then References 1 If logit ( e ( X )) or g 0 ( X ) is linear in c j ( X ) , j = 1 , . . . , p , γ EB is statistically consistent. then ˆ 2 Moreover, if logit ( e ( X )), g 0 ( X ) and g 1 ( X ) are all linear in γ EB reaches the semiparametric c j ( X ) , j = 1 , . . . , p , then ˆ variance bound of γ derived in Hahn (1998). 9/18
Proof: outcome regression ← → primal problem Entropy p Balancing � If the true OR model is linear: Y i (0) = β j c j ( X i ) + ǫ i , then Qingyuan Zhao j =1 Background � w i Y i − E [ Y (0) | T = 1] Results for EB T i =0 Equivalence of PS and OR p References � � + � = β j w i c j ( X i ) − E [ c j ( X ) | T = 1] w i ǫ i . j =1 T i =0 T i =0 In the primal problem of EB, moment balancing constraints: n w i c j ( X i ) = 1 � � c j ( X i ) . n 1 i =1 T i =1 10/18
Proof: propensity scoring ← → dual problem Entropy Balancing Qingyuan Zhao The dual problem of EB is Background Results for EB � p p � Equivalence of � � � − PS and OR minimize log exp θ j c j ( X i ) θ j ¯ c j (1) , θ References T i =0 j =1 j =1 Intuitively, EB uses “exponential loss” instead of logistic loss. Consistency under logistic PS model can be rigorously proved by M-estimation theory. 11/18
Asymptotic efficiency of EB Entropy Balancing Qingyuan A natural competitor is the inverse probability weighting Zhao estimator (PS model: logistic regression solved by MLE). Background Results for EB When the logistic PS model is correctly specified, Theorem 3 in Equivalence of our paper provides formulas for the asymptotic variance. PS and OR References When Y (0) is correlated with c ( X ), EB is more efficient than MLE When the true OR model is linear in c ( X ), EB reaches the semiparametric variance bound. Conclusion: EB should be preferred over IPW+MLE. 12/18
Outline Entropy Balancing Qingyuan Zhao Background Background 1 Results for EB Equivalence of PS and OR References Results for EB 2 Equivalence of PS and OR 3 13/18
Balancing PS weights − → OR model Entropy Balancing Qingyuan Doubly robustify an OR estimator: given an OR model ˆ g 0 ( X ), Zhao Background 1 Results for EB γ EB-DR = � � w EB ˆ ( Y i − ˆ g 0 ( X i )) − ( Y i − ˆ g 0 ( X i )) . i n 1 Equivalence of PS and OR T i =1 T i =0 References Theorem (the role of balancing PS weights) p � ˆ If the fitted OR is ˆ g 0 ( X ) = β j c j ( X ), whether or not this j =1 γ EB − DR = ˆ γ EB . model is correctly specified, ˆ 14/18
OR model − → balancing weights Entropy Let X t denote the matrix X t ij = c j ( X i ) and Y t denote the Balancing Qingyuan vector of outcomes for i in the group t = 0 or 1 Zhao p � Background For linear OR model E [ Y (0) | X ] = β j c j ( X ), the OLS Results for EB j =1 Equivalence of estimator of E [ Y (0) | T = 1] is PS and OR References 1 β ) = 1 1 T ( X 1 ˆ 1 T � X 1 [( X 0 ) T X 0 ] − 1 ( X 0 ) T � Y 0 . n 1 n 1 This is a weighted average of Y 0 ! Moreover, they are balancing weights: 1 X 0 = 1 1 T � X 1 [( X 0 ) T X 0 ] − 1 ( X 0 ) T � 1 T X 1 . n 1 n 1 15/18
The role of covariate balance Entropy Balancing Our analysis of EB reveals an interesting equivalence between Qingyuan Zhao PS and OR. Background Results for EB Propensity Score Outcome Regression Equivalence of Covariate Balance Modeling Modeling PS and OR References Bias Reduction/Model Robustness Figure : Dashed arrows: conventional understanding of double robustness. Solid arrows: our understanding of double robustness revealed by entropy balancing. 16/18
Thank you Entropy Balancing Qingyuan Zhao Background Results for EB Equivalence of PS and OR References 17/18
Recommend
More recommend
