Definition Treatment Effects Statistical Inference Statistical Foundations I Department of Government London School of Economics and Political Science
Definition Treatment Effects Statistical Inference 1 What is an experiment? 2 Treatment Effects 3 Statistical Inference
Definition Treatment Effects Statistical Inference 1 What is an experiment? 2 Treatment Effects 3 Statistical Inference
Definition Treatment Effects Statistical Inference Principles of causality 1 Correlation/Relationship 2 Nonconfounding 3 Direction (“temporal precedence”) 4 Mechanism 5 Appropriate level of analysis
Definition Treatment Effects Statistical Inference Principles of causality 1 Correlation/Relationship 2 Nonconfounding 3 Direction (“temporal precedence”) 4 Mechanism 5 Appropriate level of analysis
Definition Treatment Effects Statistical Inference Experiments are different
Definition Treatment Effects Statistical Inference Experiments are different 1 Draw causal inferences through design
Definition Treatment Effects Statistical Inference Experiments are different 1 Draw causal inferences through design 2 Randomization breaks selection bias and fixes temporal precedence
Definition Treatment Effects Statistical Inference Experiments are different 1 Draw causal inferences through design 2 Randomization breaks selection bias and fixes temporal precedence 3 We don’t need to “control” for anything
Definition Treatment Effects Statistical Inference Experiments are different 1 Draw causal inferences through design 2 Randomization breaks selection bias and fixes temporal precedence 3 We don’t need to “control” for anything 4 We see “causal effects” in the comparison of experimental groups
Definition Treatment Effects Statistical Inference Definitions I The observation of units after, and possibly before, a randomly assigned intervention in a controlled setting, which tests one or more precise causal expectations
Definition Treatment Effects Statistical Inference Definitions I A randomized experiment, or randomized control trial (RCT), is: The observation of units after, and possibly before, a randomly assigned intervention in a controlled setting, which tests one or more precise causal expectations
Definition Treatment Effects Statistical Inference Definitions I A randomized experiment, or randomized control trial (RCT), is: The observation of units after, and possibly before, a randomly assigned intervention in a controlled setting, which tests one or more precise causal expectations If we manipulate the thing we want to know the effect of ( X ), and control (i.e., hold constant) everything we do not want to know the effect of ( Z ), the only thing that can affect the outcome ( Y ) is X .
Definition Treatment Effects Statistical Inference Definitions II
Definition Treatment Effects Statistical Inference Definitions II Unit : A physical object at a particular point in time
Definition Treatment Effects Statistical Inference Definitions II Treatment : An intervention, whose effect(s) we wish to assess relative to some other (non-)intervention
Definition Treatment Effects Statistical Inference Definitions II Outcome : The variable we are trying to explain
Definition Treatment Effects Statistical Inference Definitions II Potential outcomes : The outcome value for each unit that we would observe if that unit received each treatment Multiple potential outcomes for each unit, but we only observe one of them
Definition Treatment Effects Statistical Inference Definitions II Causal effect : The comparisons between the unit-level potential outcomes under each intervention This is what we want to know!
Definition Treatment Effects Statistical Inference Example
Definition Treatment Effects Statistical Inference Example Unit : Schools in Kenya
Definition Treatment Effects Statistical Inference Example Outcome : Student learning
Definition Treatment Effects Statistical Inference Example Treatment : An additional teacher per class, reducing effective class size
Definition Treatment Effects Statistical Inference Example Potential outcomes : 1 Knowledge in a “large” class 2 Knowledge in a “small class
Definition Treatment Effects Statistical Inference Example Causal effect : Difference in knowledge between the two conditions
Definition Treatment Effects Statistical Inference Units Units can be almost anything Common units in experimental designs: Individual people Sites (schools, classes, surgeries) Areas (districts, states) Units are period-specific Randomization can occur over time
Definition Treatment Effects Statistical Inference Outcomes Experiments can have many outcome concepts/measures Quite common to think about just one at a time Outcomes can be anything that: Is observable/measurable Can be measured at the level of randomization or lower
Definition Treatment Effects Statistical Inference Treatments Synonyms: manipulation, intervention, factor, condition, cell Treatments are operationalizations of independent variables in a causal theory A set of treatments generates observable variation in X
Definition Treatment Effects Statistical Inference Developing Treatments From theory, we derive testable hypotheses Hypotheses are expectations about differences in outcomes across levels of a putatively causal variable In an experiment, an hypothesis must be testable by an ATE The experimental manipulations induce variation in the causal variable that enable tests of the hypotheses
Definition Treatment Effects Statistical Inference Example: Framing and Attention 1 Theory: Presentation of information affects politicians’ attention Hypothesis: Information framed as a conflict draws more attention from political elites than information not framed as a conflict. Manipulation: Control group: Presentation of headline information Treatment group: Same information presented as conflict Outcome: How likely are legislators to read full article 1 Walgrave, Sevenans, Van Camp, Loewen (2017) – “What Draws Politicians’ Attention? An Experimental Study of Issue Framing and its Effect on Individual Political Elites”
Definition Treatment Effects Statistical Inference Ex.: Presence/Absence Theory: Legislators vote in line with constituents’ preferences Hypothesis: Exposure to a poll of constituent views shifts legislative votes. Manipulation: Control group receives no polling information. Treatment group receives a letter containing polling information. Outcome: How legislators vote on relevant piece of legislation
Definition Treatment Effects Statistical Inference Ex.: Levels/doses Theory: Legislators vote in line with constituents’ preferences Hypothesis: Exposure to a poll of constituent views shifts legislative votes. Manipulation: Control group receives no polling information. Treatment group 1 receives a letter containing polling information. Treatment group 2 receives two letters containing polling information. etc. Outcome: How legislators vote on relevant piece of legislation
Definition Treatment Effects Statistical Inference Ex.: Qualitative variation Theory: Legislators vote in line with constituents’ preferences Hypothesis: Exposure to a poll of constituent views shifts legislative votes. Manipulation: Control group receives no polling information. Treatment group 1 receives a letter containing polling information suggesting public support. Treatment group 2 receives a letter containing polling information suggesting public opposition. Outcome: How legislators vote on relevant piece of legislation
Definition Treatment Effects Statistical Inference Treatments Test Hypotheses!
Definition Treatment Effects Statistical Inference Treatments Test Hypotheses! Derive experimental design from hypotheses
Definition Treatment Effects Statistical Inference Treatments Test Hypotheses! Derive experimental design from hypotheses Experimental “factors” are expressions of hypotheses as randomized groups
Definition Treatment Effects Statistical Inference Treatments Test Hypotheses! Derive experimental design from hypotheses Experimental “factors” are expressions of hypotheses as randomized groups What intervention each group receives depends on hypotheses presence/absence levels/doses qualitative variations
Definition Treatment Effects Statistical Inference Questions?
Definition Treatment Effects Statistical Inference Complexities Experiments can have additional “moving parts” Control groups and placebo groups Pre-treatment outcome measurement Within-subjects design features Repeated measures of outcomes Cluster randomization Sampling from a population . . . None of these are necessary for causal inference
Definition Treatment Effects Statistical Inference 1 What is an experiment? 2 Treatment Effects 3 Statistical Inference
Definition Treatment Effects Statistical Inference The Fundamental Problem of Causal Inference! Units have multiple potential outcomes We can only observe one of them! Thus we never know the individual-level causal effect of a treatment for a given unit
Recommend
More recommend
