Gov 51: Randomized Experiments Matthew Blackwell Harvard - PowerPoint PPT Presentation

Gov 51: Randomized Experiments Matthew Blackwell Harvard University 1 / 10

Changing minds on gay marriage • Question: can we efgectively persuade people to change their minds? • Hugely important question for political campaigns, companies, etc. • Psychological studies show it isn’t easy. • Contact Hypothesis : outgroup hostility diminished when people from difgerent groups interact with one another. • Today we’ll explore this question the context of support for gay marriage and contact with a member of the LGBT community. • 𝑍 • 𝑈 2 / 10 𝑗 = support for gay marriage (1) or not (0) 𝑗 = contact with member of LGBT community (1) or not (0)

Causal efgects & counterfactuals • 𝑍 outcomes is observable. • Fundamental problem of causal inference : only one of the two potential 𝑗 (0) 𝑗 (1) − 𝑍 • Causal efgect for citizen 𝑗 : 𝑍 with a member of the LGBT community? 𝑗 (0) : would 𝑗 have supported gay marriage if they didn’t have contact member of the LGBT community? • What does “ 𝑈 𝑗 (1) : would 𝑗 have supported gay marriage if they had contact with a • 𝑍 • Two potential outcomes : member of the LGBT community? • Would citizen 𝑗 have supported gay marriage if they had contact with a 3 / 10 𝑗 ” mean? ⇝ counterfactuals , “what if” 𝑗 causes 𝑍

Sigma notation 𝑜 𝑜 • Σ 𝑜 𝑜 𝑍 𝑗=1 ∑ • We will often refer to the sample size (number of units) as 𝑜 . • Notation is a bit clunky, so we often use the Sigma notation : 𝑜 𝑍 • We often want sums: how many in our sample support gay marriage? 𝑜 ) 2 , … , 𝑍 1 , 𝑍 • We often have 𝑜 measurements of some variable: (𝑍 4 / 10 1 + 𝑍 2 + 𝑍 3 + ⋯ + 𝑍 𝑗 = 𝑍 1 + 𝑍 2 + 𝑍 3 + ⋯ + 𝑍 𝑗=1 means sum each value from 𝑍 1 to 𝑍

Averages • The sample average or sample mean is simply the sum of all values divided by the number of values. • Sigma notation allows us to write this in a compact way: 𝑜 𝑜 ∑ 𝑗=1 𝑍 𝑗 • Suppose we surveyed 6 people and 3 supported gay marriage: 5 / 10 𝑍 = 1 𝑍 = 1 6 (1 + 1 + 1 + 0 + 0 + 0) = 0.5

Quantity of interest • We want to estimate the average causal efgects over all units: • When will the difgerence-in-means is a good estimate of the SATE? • 𝑍 control : observed average outcome for control group • What we can estimate instead: • Why can’t we just calculate this quantity directly? 𝑗 (0)} 𝑗 (1) − 𝑍 {𝑍 𝑗=1 ∑ 𝑜 𝑜 6 / 10 Sample Average Treatment Efgect (SATE) = 1 Difgerence in means = 𝑍 treated − 𝑍 control • 𝑍 treated : observed average outcome for treated group

Randomized control trials (RCT) • Control group ≈ what would have happened to treatment group if they 𝑗 (0) 𝑜 1 • Randomized control trial : each unit’s treatment assignment is had taken control. • Similar on both observable and unobservable characteristics. • Treatment and control group are identical on average • Randomization ensures balance between treatment and control group. • Flip a coin; draw red and blue chips from a hat; etc determined by chance. 7 / 10 • 𝑍 control ≈ 𝑗=1 𝑍 𝑜 ∑ • 𝑍 treated − 𝑍 control ≈ SATE

Some potential problems with RCTs • Placebo efgects : • Respondents will be afgected by any intervention, even if they shouldn’t have any efgect. • Hawthorne efgects : • Respondents act difgerently just knowing that they are under study. 8 / 10

Balance checking • Can we determine if randomization “worked”? • If it did, we shouldn’t see large difgerences between treatment and control group on pretreatment variable . • Pretreatment variable are those that are unafgected by treatment. • We can check in the actual data for some pretreatment variable 𝑌 9 / 10 • 𝑌 treated : average value of variable for treated group. • 𝑌 control : average value of variable for control group. • Under randomization, 𝑌 treated − 𝑌 control ≈ 0

Multiple treatments • Instead of 1 treatment, we might have multiple treatment arms : • Control condition • Treatment A • Treatment B • Treatment C, etc • In this case, we will look at multiple comparisons: • If treatment arms are randomly assigned, these difgerences will be good estimators for each causal contrast. 10 / 10 • 𝑍 treated, A − 𝑍 control • 𝑍 treated, B − 𝑍 control • 𝑍 treated, A − 𝑍 treated, B