Estimating the ATE of an endogenously assigned treatment from a - PowerPoint PPT Presentation

Estimating the ATE of an endogenously assigned treatment from a sample with endogenous selection by regression adjustment using an extended regression models David M. Drukker Executive Director of Econometrics Stata 2018 Italian Stata Users Group meeting 15 November 2018

Fictional data on wellness program from large company . use wprogram2 . describe Contains data from wprogram2.dta obs: 3,000 vars: 8 28 Jul 2017 07:13 size: 96,000 storage display value variable name type format label variable label wchange float %9.0g changel Weight change level age float %9.0g Years over 50 over float %9.0g Overweight (tens of pounds) phealth float %9.0g Prior health score prog float %9.0g yesno Participate in wellness program wtprog float %9.0g yesno Offered work time to participate in program wtsamp float %9.0g Offered work time to participate in sample insamp float %9.0g In sample: attended initial and final weigh in Sorted by: 1 / 41

Three levels of wchange . tabulate wchange prog Weight Participate in change wellness program level No Yes Total Loss 154 960 1,114 No change 251 299 550 Gain 184 36 220 Total 589 1,295 1,884 2 / 41

Three levels of wchange . tabulate wchange prog Weight Participate in change wellness program level No Yes Total Loss 154 960 1,114 No change 251 299 550 Gain 184 36 220 Total 589 1,295 1,884 Data are observational 2 / 41

Dealing with observational data . tabulate wchange prog Weight Participate in change wellness program level No Yes Total Loss 154 960 1,114 No change 251 299 550 Gain 184 36 220 Total 589 1,295 1,884 Table does not account for how observed covariates that affect program participation also affect the potential outcome variables Assume the treatment is as good as random after conditioning on covariates Conditional mean independence Exogenous treatment assignment teffects 3 / 41

Dealing with observational data . tabulate wchange prog Weight Participate in change wellness program level No Yes Total Loss 154 960 1,114 No change 251 299 550 Gain 184 36 220 Total 589 1,295 1,884 Table does not account for how observed unobserved error that affect program participation also affect the potential outcome variables Endogenous treatment assignment etefffects and etregress for continuous outcomes etpoisson for count outcomes Need Stata command for ordinal outcome 4 / 41

Dealing with observational data . tabulate wchange prog Weight Participate in change wellness program level No Yes Total Loss 154 960 1,114 No change 251 299 550 Gain 184 36 220 Total 589 1,295 1,884 Table does not account for the possibility that unobserved errors in the process that caused some of 3,000 individuals not to show for the final weigh in may also affect the potential outcome variables Endogenous loss to follow up Endogenous sample selection 5 / 41

Ordinal Potential outcomes Because the outcome wchange is ordinal, there are really three binary outcomes wchange ==“Loss”, wchange ==“No Change”, and wchange ==“Gain” 6 / 41

Ordinal Potential outcomes In the potential outcome framework, there is an outcome for each person when they participate and when the do not participate 7 / 41

Ordinal Potential outcomes In the potential outcome framework, there is an outcome for each person when they participate and when the do not participate Thus, there are really three binary outcomes for each potential outcome Participate Not participate == “Loss” == “Loss” wchange p wchange np wchange p == “No change” wchange np == “No change” == “Gain” == “Gain” wchange p wchange np 7 / 41

Potential outcome framework For each outcome (Loss, No change, and Gain), we only observe one of these two potential outcomes for each individual 8 / 41

Potential outcome framework For each outcome (Loss, No change, and Gain), we only observe one of these two potential outcomes for each individual We estimate the parameters of a model and use the estimated parameters to predict what each person does in the unobserved potential outcome Regression adjustment 8 / 41

Average treatment effects In the case of one outcome, the average treatment effect (ATE) is E [ y p − y np ] 9 / 41

Average treatment effects In the case of one outcome, the average treatment effect (ATE) is E [ y p − y np ] As there are three outcomes, there are three ATEs one for “Loss”, one for “No Change”, and one for “Gain” 9 / 41

Average treatment effects In the case of one outcome, the average treatment effect (ATE) is E [ y p − y np ] As there are three outcomes, there are three ATEs one for “Loss”, one for “No Change”, and one for “Gain” ATE Loss = E [( wchange p == “ Loss ”) − ( wchange np == “ Loss ”)] ATE Nochange = E [( wchange p == “ No change ”) − ( wchange np == “ No change ”)] ATE Gain = E [( wchange p == “ Gain ”) − ( wchange np == “ Gain ”)] 9 / 41

Average treatment effects I will provide some details about the average treatment effect for “Loss” The details for the outcomes of “No change” and “Gain” are analogous 10 / 41

the average treatment effect (ATE) of the program on the Loss outcome ATE Loss ATE Loss = E [( wchange p == “ Loss ”) − ( wchange np == “ Loss ”)] The first line says that ATE Loss is the mean diffence in the outcomes when everyone participates instead of no one participates 11 / 41

the average treatment effect (ATE) of the program on the Loss outcome ATE Loss ATE Loss = E [( wchange p == “ Loss ”) − ( wchange np == “ Loss ”)] = E [ wchange p == “ Loss ”] − E [ wchange np == “ Loss ”] The second line says that the mean of the differences is the difference in the means 12 / 41

the average treatment effect (ATE) of the program on the Loss outcome ATE Loss ATE Loss = E [( wchange p == “ Loss ”) − ( wchange np == “ Loss ”)] = E [ wchange p == “ Loss ”] − E [ wchange np == “ Loss ”] = Pr [ wchange p == “ Loss ”] − Pr [ wchange np == “ Loss ”] The third line says that because the mean of binary outcome is the probability that the event is true, the ATE Loss is the difference in the probability an individual is in the state of “Loss” when everyone participates instead of no one participates 13 / 41

I am going to use the ERM comand eoprobit to estimate the parameters of Pr [ wchange p == “ Loss ” | x ] and Pr [ wchange np == “ Loss ” | x ] and 14 / 41

I am going to use the ERM comand eoprobit to estimate the parameters of Pr [ wchange p == “ Loss ” | x ] and Pr [ wchange np == “ Loss ” | x ] and Then I use margins or estat teffects to estimate E [ Pr [ wchange p == “ Loss ” | x ]] − E [ Pr [ wchange np == “ Loss ” | x ]] = Pr [ wchange p == “ Loss ”] − Pr [ wchange np == “ Loss ”] = ATE Loss 14 / 41

I am going to use the ERM comand eoprobit to estimate the parameters of Pr [ wchange p == “ Loss ” | x ] and Pr [ wchange np == “ Loss ” | x ] and Then I use margins or estat teffects to estimate E [ Pr [ wchange p == “ Loss ” | x ]] − E [ Pr [ wchange np == “ Loss ” | x ]] = Pr [ wchange p == “ Loss ”] − Pr [ wchange np == “ Loss ”] = ATE Loss The ATE Loss is the mean difference in the probability an individual is in the state of “Loss” when everyone participates instead of no one participates 14 / 41

Models for the ordinal outcome For exogenous treatment, we do a one-step equivalent to fitting two separate ordinal probit models One fit to partipants Another fit to non partipants 15 / 41

Model for partipants  “ Loss ” if x β 0 + ǫ 0 ≤ cut 1 0   wchange = “ No change ” if cut 1 0 < x β 0 + ǫ 0 ≤ cut 2 0  “ Gain ” if cut 2 0 < x β 0 + ǫ 0  x β 0 = β 1 , 0 age + β 2 , 0 over + β 3 , 0 phealth for the observations at which prog =0, and ǫ 0 , is standard normal 16 / 41

Model for nonparticipants  “ Loss ” if x β 1 + ǫ 1 ≤ cut 1 1   wchange = “ No change ” if cut 1 1 < x β 1 + ǫ 1 ≤ cut 2 1  “ Gain ” if cut 2 1 < x β 1 + ǫ 1  x β 1 = β 1 , 1 age + β 2 , 1 over + β 3 , 1 phealth for the observations at which prog =1 ǫ 1 is standard normal 17 / 41

 “ Loss ” if x β 0 + ǫ 0 ≤ cut 1 0   wchange = “ No change ” if cut 1 0 < x β 0 + ǫ 0 ≤ cut 2 0  “ Gain ” if cut 2 0 < x β 0 + ǫ 0  x β 0 = β 1 , 0 age + β 2 , 0 over + β 3 , 0 phealth for the observations at which prog =0, and  “ Loss ” if x β 1 + ǫ 1 ≤ cut 1 1   wchange = “ No change ” if cut 1 1 < x β 1 + ǫ 1 ≤ cut 2 1  “ Gain ” if cut 2 1 < x β 1 + ǫ 1  x β 1 = β 1 , 1 age + β 2 , 1 over + β 3 , 1 phealth for the observations at which prog =1 ǫ 0 , and ǫ 1 are normal corr ( ǫ 0 , ǫ 1 ) is not identified or estimated 18 / 41