Pot -Luck Bring Your Ow n Problem Isabelle Guyon, Clopinet - PowerPoint PPT Presentation

Causality Challenge Pot -Luck Bring Your Ow n Problem Isabelle Guyon, Clopinet Constantin Aliferis and Alexander Statnikov, Vanderbilt Univ. André Elisseeff and Jean-Philippe Pellet, IBM Zürich Gregory F. Cooper, Pittsburg University Peter Spirtes, Carnegie Mellon 1

Causality Workbench • Goal: Benchmark causal discovery algorithms . • Method: – Challenges. – Repository of datasets, tasks, models, software, etc. – Interactive workbench. – Weekly teleconference seminar. • So far… – Causality Challenge #1: Causation and Prediction (WCCI 2008). – Causality Challenge #2: Pot-luck (NIPS 2008). • Winter, 2008: Start developing an interactive workbench. • June, 2009: KDD workshop on causality in time series? 2

Why a new challenge? • Causality challenge #1 – Favor “depth” • Single well defined task • Rigorous performance assessment • Causality challenge #2 – Favor “breadth” • Many different tasks • Encourage creativity 3

http://clopinet.com/causality 4 5

Pot-Luck challenge Task Participants Type (views) CYTO 2 (394) real self eval LOCANET 10 (558) real artif PROMO 3 (570) artif self eval SIGNET 2 (415) artif TIED 1 (330) artif CauseEffectPairs 5 (218) real Stemmatology 0 (109) self eval real 5

Other donated datasets Task Views Type WebLogs 90 self eval real MIDS 65 artif NOISE 43 artif real SECOM 59 real SEFTI 35 real http://clopinet.com/causality 6

Winners PRIZES Kun Zhang and Aapo Hyvärinen Best benchmark result: Guido Nolte Best contributed task: MENTIONS Significant advance on REGED: Ernest Mwebaze and John Quinn You Zhou, Changzhang Wang, Jianxin Yin, Zhi Geng SIDO: Mehreen Saeed SIGNET: Cheng Zheng and Zhi Geng SIGNET: Advanced Analytics, Intel, LTD TIED: 7

Causal discovery from real m anipulations The CYTO problem 1 2 3 LFA- 1 CD3 CD28 PI 3K RAS L Cytohesin JAB- 1 Zap70 VAV 10 A 4 SLP- 76 6 Lck T PKC LC γ P Akt PI P3 Act ivat ors 1. α - CD3 5 7 2. α - CD28 PI P2 PKA Raf 3. I CAM- 2 8 4. PMA 5. β 2cAMP Mek1/ 2 MAPKKK MAPKKK I nhibitors 9 6. G06976 Erk1/ 2 MEK4/ 7 MEK3/ 6 7. AKT inh 8. Psitect 9. U0126 JNK p38 8 10. LY294002 Karen Sachs et al

What if w e cannot experim ent? The LOCANET problem • Experiments may be infeasible, costly or unethical. • Using only observations we may want to predict the effect of new policies. • Policies may consist in manipulating several variables. • Task: Find the local causal structure around a given target variable (depth 3 network) in four datasets (REGED, CINA, SIDO, and MARTI). 9

Multiple alternative solutions The TIED problem Two disjoint subsets of X 1 X 2 X 3 X 11 Y variables V 1 and V 2 are 0 0 0 0 0 Target Information 1 1 1 1 1 Equivalent w.r.t. target Y 2 2 2 2 2 TIE Y ( V 1 , V 2 ), iff: 3 3 3 3 3 – V 1 ⊥ Y TIE Y (X 1 , X 2 ) – V 2 ⊥ Y TIE Y (X 1 , X 3 ) TIE Y (X 1 , X 11 ) – V 1 ⊥ Y | V 2 TIE Y (X 2 , X 3 ) TIE Y (X 2 , X 11 ) – V 2 ⊥ Y | V 1 TIE Y (X 3 , X 11 ) 10 Alexander Statnikov & Constantin Aliferis

A dynam ical system : SIGNET Abscisic Acid Signaling Netw ork Donated by Jenkins and Soni 11 Resimulated from Li, Assmann, 47 Albert, PLOS, 2006

Causality in tim e series: PROMO A sim ulated m arketing task • 100 products • 1000 promotions • 3 years of daily data • Goal: quantify the effect of promotions 1000 on sales. other The difficulties include: - non iid samples - seasonal effects 100 - promotions are binary, sales Jean-Philippe Pellet & André Elisseeff are continuous 12

Causal direction am ong only tw o variables? The CauseEffectPairs problem • Many causal discovery methods rely on tests of conditional independence between 3 or more variables. • Task: Find the causal direction among pairs of variables (real data, e.g. temperature and altitude). Dominik Janzing 13

What’s next? • Proceedings of NIPS workshop (JMLR, early 2009): – Submit revised 10-page paper by December 19, 2008. • Depth vs. breadth → focus months: – Teleconference presentations on one particular challenge. – Deadline for submission; result analysis and debate. • Causality challenge #3: – Focus on time series. – Target KDD, June 2009. • Interactive workbench: – Under development; target next NIPS. http://clopinet.com/causality 14