Causality Bernhard Sch olkopf and Jonas Peters MPI for Intelligent - PowerPoint PPT Presentation

Causality Bernhard Sch¨ olkopf and Jonas Peters MPI for Intelligent Systems, T¨ ubingen MLSS, T¨ ubingen 21st July 2015

Charig et al.: “Comparison of treatment of renal calculi by open surgery, (...) ”, British Medical Journal, 1986 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Charig et al.: “Comparison of treatment of renal calculi by open surgery, (...) ”, British Medical Journal, 1986 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

J. Mooij et al.: Distinguishing cause from effect using observational data: methods and benchmarks , submitted B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Assume P ( X 1 , . . . , X 4 ) has been induced by X 1 = f 1 ( X 3 , N 1 ) G 0 X 1 X 2 = N 2 X 3 = f 3 ( X 2 , N 3 ) X 2 X 3 X 4 = f 4 ( X 2 , X 3 , N 4 ) X 4 • N i jointly independent • G 0 has no cycles Functional causal model. Can the DAG be recovered from P ( X 1 , . . . , X 4 )? B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Assume P ( X 1 , . . . , X 4 ) has been induced by X 1 = f 1 ( X 3 , N 1 ) G 0 X 1 X 2 = N 2 X 3 = f 3 ( X 2 , N 3 ) X 2 X 3 X 4 = f 4 ( X 2 , X 3 , N 4 ) X 4 • N i jointly independent • G 0 has no cycles Functional causal model. Can the DAG be recovered from P ( X 1 , . . . , X 4 )? No. JP, J. Mooij, D. Janzing and B. Sch¨ olkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 S. Shimizu, P. Hoyer, A. Hyv¨ arinen, A. Kerminen: A linear non-Gaussian acyclic model for causal discovery . JMLR, 2006 P. B¨ uhlmann, JP, J. Ernest: CAM: Causal add. models, high-dim. order search and penalized regr., Annals of Statistics 2014 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Assume P ( X 1 , . . . , X 4 ) has been induced by X 1 = f 1 ( X 3 ) + N 1 G 0 X 1 X 2 = N 2 X 3 = f 3 ( X 2 ) + N 3 X 2 X 3 X 4 = f 4 ( X 2 , X 3 ) + N 4 X 4 • N i ∼ N (0 , σ 2 i ) jointly independent • G 0 has no cycles Additive noise model with Gaussian noise. Can the DAG be recovered from P ( X 1 , . . . , X 4 )? Yes iff f i nonlinear. JP, J. Mooij, D. Janzing and B. Sch¨ olkopf: Causal Discovery with Continuous Additive Noise Models , JMLR 2014 P. B¨ uhlmann, JP, J. Ernest: CAM: Causal add. models, high-dim. order search and penalized regr. , Annals of Statistics 2014 S. Shimizu, P. Hoyer, A. Hyv¨ arinen, A. Kerminen: A linear non-Gaussian acyclic model for causal discovery . JMLR, 2006 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Consider a distribution generated by Y = f ( X ) + N Y X Y with N Y , X ind ∼ N B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Consider a distribution generated by Y = f ( X ) + N Y X Y with N Y , X ind ∼ N Then, if f is nonlinear, there is no X = g ( Y ) + M X X Y with M X , Y ind ∼ N JP, J. Mooij, D. Janzing and B. Sch¨ olkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Consider a distribution corresponding to Y = X 3 + N Y X Y with N Y , X ind ∼ N with X ∼ N (1 , 0 . 5 2 ) N Y ∼ N (0 , 0 . 4 2 ) B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

15 10 Y 5 0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 X B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

15 10 Y 5 0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 X B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

15 10 Y 5 0 −0.5 0.0 0.5 1.0 1.5 2.0 2.5 X B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

15 10 Y 5 0 −0.8 −0.6 −0.4 −0.2 0.0 0.2 0.4 gam(X ~ s(Y))$residuals B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Surprise (under some assumptions): 2 variables ⇒ p variables JP, J. Mooij, D. Janzing and B. Sch¨ olkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Surprise (under some assumptions): 2 variables ⇒ p variables JP, J. Mooij, D. Janzing and B. Sch¨ olkopf: Causal Discovery with Continuous Additive Noise Models, JMLR 2014 Let P ( X 1 , . . . , X p ) be induced by a ... conditions identif. structural equation model: X i = f i ( X PA i , N i ) - ✗ additive noise model: X i = f i ( X PA i ) + N i nonlin. fct. ✓ causal additive model: X i = � k ∈ PA i f ik ( X k ) + N i nonlin. fct. ✓ linear Gaussian model: X i = � k ∈ PA i β ik X k + N i linear fct. ✗ . (results hold for Gaussian noise) B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

GAUL GAUSS “the LINEAR” B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

100 IGCI LiNGaM 90 Significant Additive Noise PNL 80 70 Accuracy (%) 60 Not significant 50 40 30 20 10 0 0 20 40 60 80 100 Decision rate (%) see also D. Lopez-Paz, K. Muandet, B. Sch¨ olkopf, I. Tolstikhin: Towards a Learning Theory of Cause-Effect Inference , ICML 2015 E. Sgouritsa, D. Janzing, P. Hennig, B. Sch¨ olkopf: Inf. of Cause and Effect with Unsupervised Inverse Regr., AISTATS 2015 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Real data : genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Real data : genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) true hits: ≈ 0 . 1% of pairs B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Real data : genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) true hits: ≈ 0 . 1% of pairs “Invariant prediction” method: E = { obs , int } B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

Real data : genetic perturbation experiments for yeast (Kemmeren et al., 2014) p = 6170 genes n obs = 160 wild-types n int = 1479 gene deletions (targets known) true hits: ≈ 0 . 1% of pairs “Invariant prediction” method: E = { obs , int } JP, P. B¨ uhlmann, N. Meinshausen: Causal inference using inv. pred.: identification and conf. intervals , arXiv, 1501.01332 D. Rothenhaeusler, C. Heinze et al.: backShift: Learning causal cyclic graphs from unknown shift interv. , arXiv 1506.02494 M. Rojas-Carulla et al.: A Causal Perspective on Domain Adaptation , arXiv 1507.05333 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

observational training data interventional training data interventional test data point 1 0.5 (interv. on genes other than 5954 and 4710) (intervention on gene 5954) ACTIVITY GENE 4710 ACTIVITY GENE 4710 0 0.0 −1 −2 −0.5 −3 −1.0 −4 −5 −1.0 −0.5 0.0 0.5 −1.0 −0.5 0.0 0.5 −5 −4 −3 −2 −1 0 1 ACTIVITY GENE 5954 ACTIVITY GENE 5954 ACTIVITY GENE 5954 most significant pair B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

2 observational training data interventional training data interventional test data point (interv. on genes other than 3729 and 3730) (intervention on gene 3729) 1.0 1 ACTIVITY GENE 3730 ACTIVITY GENE 3730 0 0.5 −1 0.0 −2 −3 −0.5 −4 −0.5 0.0 0.5 1.0 −0.5 0.0 0.5 1.0 −4 −3 −2 −1 0 1 2 ACTIVITY GENE 3729 ACTIVITY GENE 3729 ACTIVITY GENE 3729 2nd most significant pair B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

1.5 observational training data interventional training data interventional test data point 2 (interv. on genes other than 3672 and 1475) (intervention on gene 3672) ACTIVITY GENE 1475 1.0 ACTIVITY GENE 1475 1 0 0.5 −1 0.0 −2 −3 −0.5 −0.5 0.0 0.5 1.0 1.5 −0.5 0.0 0.5 1.0 1.5 −3 −2 −1 0 1 2 ACTIVITY GENE 3672 ACTIVITY GENE 3672 ACTIVITY GENE 3672 3rd most significant pair B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

8 # STRONG INTERVENTION EFFECTS 6 PERFECT INVARIANT HIDDEN−INVARIANT 4 PC RFCI REGRESSION (CV−Lasso) GES and GIES RANDOM (99% prediction− 2 interval) 0 0 5 10 15 20 25 # INTERVENTION PREDICTIONS B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

http://xkcdsw.com/3039 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015

B. Watterson: It’s a magical world , Andrews McMeel Publishing, 1996 B. Sch¨ olkopf & J. Peters (MPI) Causality 21st July 2015