experiments and causal inference
play

Experiments and Causal Inference Erik Gahner Larsen Advanced - PowerPoint PPT Presentation

Feb 05, 2023 •335 likes •1.02k views

Experiments and Causal Inference Erik Gahner Larsen Advanced applied statistics, 2015 1 / 67 Articles published in APSR 10 15 0 5 Experiments in political science research 190609 191014 191519 192024 Political Science

  1. Experiments and Causal Inference Erik Gahner Larsen Advanced applied statistics, 2015 1 / 67

  2. Articles published in APSR 10 15 0 5 Experiments in political science research 1906−09 1910−14 1915−19 1920−24 Political Science (Druckman et al. 2011, 5) 1925−29 Cambridge Handbook of Experimental 1930−34 1935−39 1940−44 1945−49 1950−54 1955−59 1960−64 1964−69 1970−74 1975−79 1980−84 1985−89 1990−94 1995−99 2000−04 2005−09 2 / 67

  3. Quotes #1 “Like it or not, social scientists rely on the logic of experimentation even when analyzing nonexperimental data.” (Green and Gerber 2003, 110) “In some sense every empirical researcher is reporting the results of an experiment. Every researcher who behaves as if an exogenous variable varies independently of an error term effectively views their data as coming from an experiment.” (Harrison and List 2004, 1009) 3 / 67

  4. Quotes #2 “[T]here is no reason to suppose that case study research follows a divergent logic of inquiry relative to experimental research.” (Gerring and McDermott 2007, 689) “If you can’t devise an experiment that answers your question in a world where anything goes, then the odds of generating useful results with a modest budget and nonexperimental survey data seem pretty slim. The description of an ideal experiment also helps you formulate causal questions precisely.” (Angrist and Pischke 2009, 5) 4 / 67

  5. Agenda ▸ Causal effects ▸ Experiments and assumptions ▸ Issues in experimental research ▸ Types of experiments 5 / 67

  6. Example: The effect of facebook use on life satisfaction ▸ How would we test this in an observational setting? ▸ What is the problem? 6 / 67

  7. What is it all about? ▸ We need strong designs in order to make causal inferences ▸ Remember: Science is all about causality ▸ The issue it not the data we have . . . but the data we do not have. ▸ “Easy” to measure the factual world ▸ What about the counterfactual world? ▸ We need theoretical and statistical tools to make valid counterfactuals 7 / 67

  8. What is it all about? ▸ From estimation strategies ( statistics ) to identification strategies ( design ): ▸ “Without an experiment, a natural experiment, a discontinuity, or some other strong design, no amount of econometric or statistical modeling can make the move from correlation to causation persuasive.” (Sekhon 2009, 503) 8 / 67

  9. Neyman-Rubin causal model ▸ We are interested in potential outcomes to define causal effects ▸ For individual i , we have a potential outcome: Y i ▸ Treatment: W i ▸ Potential outcome given treatment treatment status: Y i ( W i ) ▸ Two potential outcomes: Y i ( 1 ) , Y i ( 0 ) ▸ Unit causal effect: The difference between a unit’s potential outcome under treatment and the unit’s potential outcome under control. τ i = Y i ( 1 ) − Y i ( 0 ) 9 / 67

  10. Example: facebook and life satisfaction User i Y i ( 0 ) (no facebook) 1 60 2 20 3 80 4 30 5 40 6 75 7 40 8 20 9 60 10 75 Average 50 10 / 67

  11. Example: facebook and life satisfaction User i Y i ( 0 ) (no facebook) Y i ( 1 ) (facebook) 1 60 70 2 20 50 3 80 80 4 30 45 5 40 50 6 75 60 7 40 45 8 20 30 9 60 85 10 75 85 Average 50 60 11 / 67

  12. Example: facebook and life satisfaction User i Y i ( 0 ) (no facebook) Y i ( 1 ) (facebook) τ i 1 60 70 10 2 20 50 30 3 80 80 0 4 30 45 15 5 40 50 10 6 75 60 -15 7 40 45 5 8 20 30 10 9 60 85 25 10 75 85 10 Average 50 60 10 12 / 67

  13. FPCI 13 / 67

  14. The Fundamental Problem of Causal Inference (FPCI) ▸ “It is impossible to observe the value of Y i ( 1 ) and Y i ( 0 ) on the same unit and, therefore, it is impossible to observe the effect of W i on i .” (Holland 1986, 947) ▸ We observe one outcome: the realised outcome R i = W i Y i ( 1 ) + ( 1 − W i ) Y i ( 0 ) 14 / 67

  15. Example: facebook and life satisfaction User i Y i ( 0 ) Y i ( 1 ) W i 1 60 ? 0 2 ? 50 1 3 80 ? 0 4 ? 45 1 5 40 ? 0 6 ? 60 1 7 ? 45 1 8 20 ? 0 9 ? 85 1 10 75 ? 0 15 / 67

  16. Example: facebook and life satisfaction User i Y i ( 0 ) Y i ( 1 ) R i (observed outcome) W i 1 60 ? 0 60 2 ? 50 1 50 3 80 ? 0 80 4 ? 45 1 45 5 40 ? 0 40 6 ? 60 1 60 7 ? 45 1 45 8 20 ? 0 20 9 ? 85 1 85 10 75 ? 0 75 16 / 67

  17. So, you cannot prove causality with statistics? 17 / 67

  18. Well, you can only prove causality with statistics. ▸ Rosenbaum (2010, 35) ▸ The FPCI is a missing data problem. What is the solution? 18 / 67

  19. Random assignment Figur 1: Random selection 19 / 67

  20. Random assignment ▸ Create two groups of observations that are, in expectation, identical prior to application of the treatment (Green and Gerber 2012, 31) ▸ Create a counterfactual group. ▸ Guarantees that the treatment is prior to the outcome, avoiding posttreatment and simultaneity biases. ▸ P ( W ) = 0.5 ( coin flip ) 20 / 67

  21. Assumption I: Ignorability of Treatment Assignment ▸ Pretreatment covariates, X ▸ Unconfoundedness ( Y ( 1 ) , Y ( 0 ) , X ) ⊥ W ▸ What about ( Y ( 1 ) , Y ( 0 )) ⊥ W ∣ X ? ▸ We will address this issue next week 21 / 67

  22. Average treatment effect ▸ What most scholars are interested in ▸ Average treatment effect: τ ATE = E [ Y ( 1 ) − Y ( 0 )] = E [ Y ( 1 )] − E [ Y ( 0 )] 22 / 67

  23. Assumption II: Stable Unit Treatment Value Assumption (SUTVA) ▸ A collection of implied assumptions about the effect of treatments on individuals ▸ “The potential outcomes for any unit do not vary with the treatments assigned to other units, and, for each unit, there are no different forms or versions of each treatment level, which lead to different potential outcomes.” (Imbens and Rubin 2015, 10) 23 / 67

  24. Assumption II: Stable Unit Treatment Value Assumption (SUTVA) 1. Noninterference: Potential outcomes for unit i depend only on the treatment assignment of unit i (no interference or spillover effect): ( Y ( 1 ) , Y ( 0 )) ⊥ W j , ∀ i ≠ j 2. Exclusion restriction: Only one version of each treatment possible for each unit 24 / 67

  25. Assumption II: Stable Unit Treatment Value Assumption (SUTVA) Two implications (from Heckman 2005, 11): ▸ Rules out social interactions and general equilibrium effects. ▸ Rules out any effect of the assignment mechanism on potential outcomes. 25 / 67

  26. Assumption III: Compliance ▸ W i is assignment to treatment ▸ Subjects can - in many cases - decide not to comply ▸ D i : treatment status (1 if treated, 0 if not) 26 / 67

  27. Assumption III: Compliance, always-takers ▸ Always-takers will always be treated ▸ W i = 1, D i = 1 ▸ W i = 0, D i = 1 ▸ Facebook example: Will use facebook independent of treatment assignment 27 / 67

  28. Assumption III: Compliance, never-takers ▸ Never-takers will never be treated ▸ W i = 1, D i = 0 ▸ W i = 0, D i = 0 ▸ Facebook example: Will not use facebook independent of treatment assignment 28 / 67

  29. Assumption III: Compliance, cooperators ▸ Cooperators will. . . cooperate ▸ W i = 1, D i = 1 ▸ W i = 0, D i = 0 ▸ Facebook example: Will only use facebook if assigned to treatment 29 / 67

  30. Assumption III: Compliance, defiers ▸ Defiers will. . . do the opposite ▸ W i = 1, D i = 0 ▸ W i = 0, D i = 1 ▸ Facebook example: Will use facebook if not assigned to treatment and not use facebook if assigned to treatment 30 / 67

  31. So, which cases inform causal inference? ▸ The cases whose treatment status can be changed (hint: cooperators) 31 / 67

  32. Assumption III: Compliance How do we know that W i = 1, D i = 1 is a cooperator and not an always-taker? How do we know that W i = 0, D i = 0 is a cooperator and not a never-taker? 32 / 67

  33. Assumption III: Compliance ▸ We only have realised outcomes (we need a counterfactual) ▸ Hard to say whether we are dealing with compliance or noncompliance ▸ Remember: Try to measure compliance! 33 / 67

  34. Intention-to-treat ▸ Our effects are often intention-to-treat (ITT) estimates. ▸ Mean difference on Y between subjects assigned to treatment and subjects not assigned to treatment. 34 / 67

  35. Example: Noncompliance with Encouragement W i to Exercise D i ▸ From Table 5.5 in Rosenbaum (2002, 182). ▸ Y = forced expiratory volume (higher numbers signifying better lung function) ▸ Will subject exercice with encouragement? ( d i ( 1 ) ) ▸ Will subject exercice without encouragement? ( d i ( 0 ) ) 35 / 67

  36. Example: Noncompliance with Encouragement W i to Exercise D i User i d i ( 1 ) d i ( 0 ) 1 1 1 2 1 1 3 1 0 4 1 0 5 1 0 6 1 0 7 1 0 8 1 0 9 0 0 10 0 0 36 / 67

  37. What are the potential outcomes? 37 / 67

  38. Example: Noncompliance with Encouragement W i to Exercise D i User i d i ( 1 ) d i ( 0 ) Y i ( 1 ) Y i ( 0 ) 1 1 1 71 71 2 1 1 68 68 3 1 0 64 59 4 1 0 62 57 5 1 0 59 54 6 1 0 58 53 7 1 0 56 51 8 1 0 56 51 9 0 0 42 42 10 0 0 39 39 38 / 67

  39. Let’s assign some treatments and see the realised outcomes. 39 / 67

Recommend

Causal Inference An introduction based on S. Wagers course on Causal Inference (OIT 661) Imke

Causal Inference An introduction based on S. Wagers course on Causal Inference (OIT 661) Imke

Causal Inference An introduction based on S. Wagers course on Causal Inference (OIT 661) Imke Mayer November 23, 2018 Group Meeting, CMAP Outline 1. Treatment effect estimation in randomized experiments 2. Beyond a single randomized

1.48k views • 47 slides
Modes of Statistical Inference for Causal Efgects Plus an overview of the testing based approach

Modes of Statistical Inference for Causal Efgects Plus an overview of the testing based approach

Modes of Statistical Inference for Causal Efgects Plus an overview of the testing based approach to causal inference for experiments on networks CSBS Causal Inference Workshop @ Illinois Jake Bowers Political Science & Statistics

1k views • 88 slides
Causal inference Gary Goertz Kroc Institute for International Peace Studies University of Notre

Causal inference Gary Goertz Kroc Institute for International Peace Studies University of Notre

Causal inference Gary Goertz Kroc Institute for International Peace Studies University of Notre Dame ggoertz@nd.edu Spring 2018 Experiments are the gold standard for causal inference Strategies of causal inference are satisfactory (with

318 views • 11 slides
Causal inference Part I.b: randomized experiments, matching and regression (this lecture starts

Causal inference Part I.b: randomized experiments, matching and regression (this lecture starts

Causal inference Part I.b: randomized experiments, matching and regression (this lecture starts with other slides on randomized experiments) Frank Venmans Example of a randomized experiment: Job Training Partnership Act (JTPA) Largest

690 views • 30 slides
High-dimensional causal inference, graphical modeling and structural equation models Peter B

High-dimensional causal inference, graphical modeling and structural equation models Peter B

High-dimensional causal inference, graphical modeling and structural equation models Peter B uhlmann Seminar f ur Statistik, ETH Z urich cannot do confirmatory causal inference without randomized intervention experiments... but we can

1.03k views • 88 slides
Causal Inference and Response Surface Modeling Inference and

Causal Inference and Response Surface Modeling Inference and

Causal Inference and Response Surface Modeling Inference and Representa6on DS-GA-1005 Fall 2015 Guest lecturer: Uri Shalit What is Causal Inference?

503 views • 48 slides
Introduction to Causal Inference Lan Liu University of Minnesota at Twin Cities liux3771@umn.edu

Introduction to Causal Inference Lan Liu University of Minnesota at Twin Cities liux3771@umn.edu

Introduction to Causal Inference Lan Liu University of Minnesota at Twin Cities liux3771@umn.edu 1 Table of contents Causal ... or not? How Topics in Causal Inference Tools we use... Causal Inference in Industry 2 The Danger of Ice Cream

961 views • 52 slides
{ a) policy evaluation (treatment effects ) Old b) attribution but for gems

{ a) policy evaluation (treatment effects ) Old b) attribution but for gems

JSM-2016 presented: 8.1.16 OUTLINE CAUSAL INFERENCE IN STATISTICS: 1. The causal revolution from statistics to policy intervention to counterfactuals 2. The fundamental laws of causal inference A Gentle Introduction 3. From

369 views • 12 slides
A Brief Introduction to Causal Inference Brady Neal causalcourse.com What is causal inference?

A Brief Introduction to Causal Inference Brady Neal causalcourse.com What is causal inference?

A Brief Introduction to Causal Inference Brady Neal causalcourse.com What is causal inference? Inferring the effects of any treatment/policy/intervention/etc. 2 / 28 Brady Neal What is causal inference? Inferring the effects of any

2.8k views • 102 slides
Causal Inference Theory and Applications Dr. Matthias Uflacker, Johannes Huegle, Christopher

Causal Inference Theory and Applications Dr. Matthias Uflacker, Johannes Huegle, Christopher

Causal Inference Theory and Applications Dr. Matthias Uflacker, Johannes Huegle, Christopher Schmidt April 24, 2018 Agenda April 24, 2018 Jupyter Notebook Causal Inference in Application Recap Causal Inference in a Nutshell

520 views • 33 slides
Political Science 209 - Fall 2018 Causal Inference Florian Hollenbach 7th September 2018 Causal

Political Science 209 - Fall 2018 Causal Inference Florian Hollenbach 7th September 2018 Causal

Political Science 209 - Fall 2018 Causal Inference Florian Hollenbach 7th September 2018 Causal Inference What do you think is causal inference? Florian Hollenbach 1 Causal Inference causal: relationship between things where one causes

759 views • 47 slides
Causal Inference By: Miguel A. Hern an and James M. Robins Part I: Causal inference without

Causal Inference By: Miguel A. Hern an and James M. Robins Part I: Causal inference without

Causal Inference By: Miguel A. Hern an and James M. Robins Part I: Causal inference without models Chapter 1: A definition of causal effect 27 th November, 2013 27 th November, 2013 Chapter 1 (Hern an & Robins) Definition of causal

491 views • 13 slides
Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs

Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs

Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs Causation Causal inference Why/when causality matters Hurdles to causal inference & strategies to overcome them Correlation Vs Causation

626 views • 45 slides
Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs

Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs

Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs Causation Causal inference Why/when causality matters Hurdles to causal inference & strategies to overcome them Correlation Vs Causation

457 views • 26 slides
Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs

Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs

Geographic Data Science - Lecture IX Causal Inference Dani Arribas-Bel Today Correlation Vs Causation Causal inference Why/when causality matters Hurdles to causal inference & strategies to overcome them Correlation Vs Causation

741 views • 45 slides
Randomized Experiments The goal of randomized experiments is to identify The causal

Randomized Experiments The goal of randomized experiments is to identify The causal

Randomized Experiments The goal of randomized experiments is to identify The causal effect! Advantage of causal effect described by statisticians: The advantage of causal predictors compared with non-causal predictors is that

1.07k views • 86 slides
Causal reasoning and inference with causal Bayes nets Alexander Gebharter Duesseldorf Center for

Causal reasoning and inference with causal Bayes nets Alexander Gebharter Duesseldorf Center for

Causal reasoning and inference with causal Bayes nets Alexander Gebharter Duesseldorf Center for Logic and Philosophy of Science Heinrich Heine University Duesseldorf 28.04.2016 Alexander Gebharter (DCLPS) Reasoning & inference with CBNs

1.09k views • 106 slides
Contents 1 Causal Inference and Predictive Comparison 2 1.1 How Predictive Comparison Can

Contents 1 Causal Inference and Predictive Comparison 2 1.1 How Predictive Comparison Can

Causal Inference in Observational Studies Contents 1 Causal Inference and Predictive Comparison 2 1.1 How Predictive Comparison Can Mislead . . . . . . . . . . . . . 2 1.2 Adding Predictors as a Solution . . . . . . . . . . . . . . . . . .

274 views • 14 slides
CAUSAL INFERENCE AS COMPUTATIONAL LEARNING Judea Pearl University of California Los Angeles

CAUSAL INFERENCE AS COMPUTATIONAL LEARNING Judea Pearl University of California Los Angeles

CAUSAL INFERENCE AS COMPUTATIONAL LEARNING Judea Pearl University of California Los Angeles (www.cs.ucla.edu/~judea) OUTLINE Inference: Statistical vs. Causal distinctions and mental barriers Formal semantics for

773 views • 64 slides
Causal Inference and Experimentation Macartan Humphreys mh2245@columbia.edu November 15, 2011

Causal Inference and Experimentation Macartan Humphreys mh2245@columbia.edu November 15, 2011

Causal Inference Design Road Map Analysis Estimands and Estimators Troubleshooting Prospects and Limitations Causal Inference and Experimentation Macartan Humphreys mh2245@columbia.edu November 15, 2011 Macartan Humphreys

627 views • 51 slides
Gov 2002 - Causal Inference III: Regression Discontinuity Designs Matthew Blackwell Arthur

Gov 2002 - Causal Inference III: Regression Discontinuity Designs Matthew Blackwell Arthur

Gov 2002 - Causal Inference III: Regression Discontinuity Designs Matthew Blackwell Arthur Spirling October 16th, 2014 Introduction Causal for us so far: selection of observables, instrumental variables for when this doesnt hold

2.07k views • 179 slides
Probality-free causal inference via the Algorithmic Markov Condition Dominik Janzing Max Planck

Probality-free causal inference via the Algorithmic Markov Condition Dominik Janzing Max Planck

Probality-free causal inference via the Algorithmic Markov Condition Dominik Janzing Max Planck Institute for Intelligent Systems T ubingen, Germany 23. June 2015 Can we infer causal relations from passive observations? Recent study

721 views • 52 slides
A Partial Solution To the Fundamental Problem of Causal Inference Some of our most important

A Partial Solution To the Fundamental Problem of Causal Inference Some of our most important

A Partial Solution To the Fundamental Problem of Causal Inference Some of our most important questions are causal questions . 100,000 50,000 Gross National Income Per Capita

850 views • 61 slides
On estimation of functional causal models: Post - nonlinear causal model as an

On estimation of functional causal models: Post - nonlinear causal model as an

On estimation of functional causal models: Post - nonlinear causal model as an example Kun Zhang, Zhikun W ang, Bernhard Schlkopf Dept. Empirical Inference Max Planck Institute for Intelligent Systems Tbingen, Germany 1

320 views • 20 slides

More recommend