Causal Discovery Richard Scheines Peter Spirtes, Clark Glymour, - PowerPoint PPT Presentation

Causal Discovery Richard Scheines Peter Spirtes, Clark Glymour, and many others Dept. of Philosophy & CALD Carnegie Mellon Graphical Models --11/30/05 1

Outline 1. Motivation 2. Representation 3. Connecting Causation to Probability (Independence) 4. Searching for Causal Models 5. Improving on Regression for Causal Inference Graphical Models --11/30/05 2

1. Motivation Non-experimental Evidence Aggressivenes Day Care A lot John A lot Mary None A little Typical Predictive Questions • Can we predict aggressiveness from the amount of violent TV watched • Can we predict crime rates from abortion rates 20 years ago Causal Questions: • Does watching violent TV cause Aggression? • I.e., if we change TV watching, will the level of Aggression change? Graphical Models --11/30/05 3

Bayes Netw orks Disease [Heart Disease, Reflux Disease, other] Qualitative Part: Shortness of Breath Directed Graph Chest Pain [Yes, No] [Yes, No] P(Disease = Heart Disease) = .2 P(Disease = Reflux Disease) = .5 P(Disease = other) = .3 P(Chest Pain = yes | D = Heart D.) = .7 Quantitative Part: P(Shortness of B = yes | D= Hear D. ) = .8 Conditional P(Chest Pain = yes | D = Reflux) = .9 P(Shortness of B = yes | D= Reflux ) = .2 Probability Tables P(Chest Pain = yes | D = other) = .1 P(Shortness of B = yes | D= other ) = .2 Graphical Models --11/30/05 4

Bayes Netw orks: Updating Given: Data on Symptoms Disease [Heart Disease, Reflux Disease, other] Chest Pain = yes Shortness of Breath Chest Pain [Yes, No] [Yes, No] Updating P(D = Heart Disease) = .2 P(D = Reflux Disease) = .5 P(D = other) = .3 P(Chest Pain = yes | D = Heart D.) = .7 P(Shortness of B = yes | D= Hear D. ) = .8 Wanted: P(Chest Pain = yes | D = Reflux) = .9 P(Shortness of B = yes | D= Reflux ) = .2 P(Disease | Chest Pain = yes ) P(Chest Pain = yes | D = other) = .1 P(Shortness of B = yes | D= other ) = .2 Graphical Models --11/30/05 5

Causal Inference Given: Data on Symptoms Chest Pain = yes Updating P(Disease | Chest Pain = yes ) Causal Inference P(Disease | Chest Pain set= yes ) Graphical Models --11/30/05 6

Causal Inference When and how can we use non-experimental data to tell us about the effect of an intervention? Manipulated Probability P(Y | X set= x, Z= z ) from Unmanipulated Probability P(Y | X = x, Z= z ) Graphical Models --11/30/05 7

2. Representation 1. Association & causal structure - qualitatively 2. Interventions 3. Statistical Causal Models 1. Bayes Networks 2. Structural Equation Models Graphical Models --11/30/05 8

Causation & Association X and Y are associated (X _||_ Y) iff ∃ x 1 ≠ x 2 P(Y | X = x 1 ) ≠ P(Y | X = x 2 ) Association is symmetric: X _||_ Y ⇔ Y _||_ X X is a cause of Y iff ∃ x 1 ≠ x 2 P(Y | X set= x 1 ) ≠ P(Y | X set= x 2 ) Causation is asymmetric: X Y ⇔ X Y Graphical Models --11/30/05 9

10 X is a direct cause of Y relative to S , iff ≠ P(Y | X set= x 2 , Z set= z ) ∃ z ,x 1 ≠ x 2 P(Y | X set= x 1 , Z set= z ) Direct Causation Y Graphical Models --11/30/05 X where Z = S - { X,Y}

11 Rash Each edge X → Y represents a direct causal claim: Causal Graphs Rash X is a direct cause of Y relative to V Infection Graphical Models --11/30/05 Exposure Causal Graph G = { V,E } Exposure Chicken Pox

Causal Graphs Do Not need to be Omitted Causes 1 Omitted Causes 2 Cause Complete Exposure Symptoms Infection Do need to be Omitted Common Cause Complete Common Causes Exposure Symptoms Infection Graphical Models --11/30/05 12

Modeling Ideal Interventions Ideal Interventions (on a variable X): (on a variable X): • Completely determine the value or distribution of a variable X • Directly Target only X (no “fat hand”) E.g., Variables: Confidence, Athletic Performance Intervention 1: hypnosis for confidence Intervention 2: anti-anxiety drug (also muscle relaxer) Graphical Models --11/30/05 13

14 Modeling Ideal Interventions Temperature Room Interventions on the Effect Pre-experimental System Graphical Models --11/30/05 Sweaters On Post

15 Modeling Ideal Interventions Interventions on the Cause Temperature Room Pre-experimental System Graphical Models --11/30/05 Sweaters On Post

Interventions & Causal Graphs • Model an ideal intervention by adding an “intervention” variable outside the original system • Erase all arrows pointing into the variable intervened upon Intervene to change Inf Pre-intervention graph Post-intervention graph ? Exp Inf Rash Exp Inf Rash I Graphical Models --11/30/05 16

Conditioning vs. Intervening P(Y | X = x 1 ) vs. P(Y | X set= x 1 ) Teeth Slides Graphical Models --11/30/05 17

Causal Bayes Netw orks The Joint Distribution Factors S m oking [0,1] According to the Causal Graph, i.e., for all X in V Y ellow F ingers L u ng C an cer P( V ) = Π P(X|Immediate [0,1] [0,1] Causes of(X)) P(S = 0) = .7 P(S,YF, L) = P(S) P(YF | S) P(LC | S) P(S = 1) = .3 P(YF = 0 | S = 0) = .99 P(LC = 0 | S = 0) = .95 P(YF = 1 | S = 0) = .01 P(LC = 1 | S = 0) = .05 P(YF = 0 | S = 1) = .20 P(LC = 0 | S = 1) = .80 P(YF = 1 | S = 1) = .80 P(LC = 1 | S = 1) = .20 Graphical Models --11/30/05 18

19 Structural Equation Models Longevity 2. Statistical Constraints 1. Structural Equations Education Graphical Models --11/30/05 Income Causal Graph Statistical Model

Structural Equation Models Education Causal Graph Income Longevity z Structural Equations: One Equation for each variable V in the graph: V = f(parents(V), error V ) for SEM (linear regression) f is a linear function z Statistical Constraints: Joint Distribution over the Error terms Graphical Models --11/30/05 20

Structural Equation Models Causal Graph Education Equations: Education = ε ed Income Income = β 1 Education + ε income Longevity Longevity = β 2 Education + ε Longevity SEM Graph Statistical Constraints: Education (path diagram) ( ε ed , ε Income , ε Income ) ~ N(0, Σ 2 ) β 2 β 1 − Σ 2 diagonal Income Longevity - no variance is zero ε Longevity ε Income Graphical Models --11/30/05 21

22 Causation to Probability 3. Connecting Graphical Models --11/30/05

The Markov Condition Causal Statistical Structure Predictions Causal Markov Axiom Independence Causal Graphs X _||_ Z | Y X Y Z i.e., P(X | Y) = P(X | Y, Z) Graphical Models --11/30/05 23

Causal Markov Axiom If G is a causal graph, and P a probability distribution over the variables in G, then in P: every variable V is independent of its non-effects, conditional on its immediate causes. Graphical Models --11/30/05 24

Causal Markov Condition Two Intuitions: 1) Immediate causes make effects independent of remote causes (Markov). 2) Common causes make their effects independent (Salmon). Graphical Models --11/30/05 25

Causal Markov Condition 1) Immediate causes make effects independent of remote causes (Markov). E = Exposure to Chicken Pox I = Infected S = Symptoms Markov Cond. E || S | I E I S Graphical Models --11/30/05 26

27 2) Effects are independent conditional on their common YF || LC | S Causal Markov Condition Markov Cond. Graphical Models --11/30/05 Lung Cancer (LC) Sm oking (S) Y ellow Fingers causes. (Y F)

Causal Structure ⇒ Statistical Data A cyclic Causal G raph X X X 2 3 1 Causal M arkov A xiom (D -separation) Independence X 3 |X X 2 1 Graphical Models --11/30/05 28

Causal Markov Axiom In SEMs, d-separation follows from assuming independence among error terms that have no connection in the path diagram - i.e., assuming that the model is common cause complete. Graphical Models --11/30/05 29

Causal Markov and D-Separation • In acyclic graphs: equivalent • Cyclic Linear SEMs with uncorrelated errors: • D-separation correct • Markov condition incorrect • Cyclic Discrete Variable Bayes Nets: • If equilibrium --> d-separation correct • Markov incorrect Graphical Models --11/30/05 30

D-separation: Conditioning vs. Intervening T P(X 3 | X 2 ) ≠ P(X 3 | X 2 , X 1 ) X 1 X 3 X 2 X 3 _||_ X 1 | X 2 T P(X 3 | X 2 set= ) = P(X 3 | X 2 set=, X 1 ) X 1 X 3 X 2 X 3 _||_ X 1 | X 2 set= I Graphical Models --11/30/05 31

32 From Statistical Data to Probability to Causation 4. Search Graphical Models --11/30/05

Causal Discovery Statistical Data ⇒ Causal Structure Data Equivalence Class of Causal Graphs X X X 2 3 1 Statistical Causal Markov Axiom X X X 2 1 3 Inference (D-separation) X X X 2 3 1 Discovery Algorithm Independence X 3 | X X 1 2 Background Knowledge - X 2 before X 3 - no unmeasured common causes Graphical Models --11/30/05 33

Representations of D-separation Equivalence Classes We want the representations to: • Characterize the Independence Relations Entailed by the Equivalence Class • Represent causal features that are shared by every member of the equivalence class Graphical Models --11/30/05 34