causal impact for app store analysis
play

Causal Impact for App Store Analysis - PowerPoint PPT Presentation

Feb 07, 2023 •335 likes •658 views

Causal Impact for App Store Analysis http://google.github.io/CausalImpact/CausalImpact.html CREST Open Workshop 23/11/15 William Martin What does it do? Measures the impact of an event (intervention) on a metric over time Impact significant

  1. Causal Impact for App Store Analysis http://google.github.io/CausalImpact/CausalImpact.html CREST Open Workshop 23/11/15 William Martin

  2. What does it do? Measures the impact of an event (intervention) on a metric over time Impact significant or not? Confidence interval? Google uses it for measuring the success of ad campaigns CREST Open Workshop 23/11/15 William Martin

  3. What about correlation analysis? Correlation analysis Causal impact analysis Looks at snapshot of data Looks at time series of data Tells us relationship between Tells us how significant an event vectors (+ve or -ve correlation, was or no correlation) CREST Open Workshop 23/11/15 William Martin

  4. How does it do it? Trains a predictor (prior time period) Makes set of predictions (posterior time period) Compares the observed vector with the predicted vector CREST Open Workshop 23/11/15 William Martin

  5. Input Vectors Compare projection with observed N umber Target App y of ratings App xn ... Controls App x2 N umber App x1 of ratings Week 1 2 …. n Release CREST Open Workshop 23/11/15 William Martin event

  6. Predictor Model Components CREST Open Workshop 23/11/15 William Martin

  7. Predictor Model Components local trend value Local trend noise sampled from expected Normal distribution increase CREST Open Workshop 23/11/15 William Martin

  8. Predictor Model Components local trend value Local trend noise sampled from expected Normal distribution increase Seasonal variance Adds seasonal component Set length and no. seasons CREST Open Workshop 23/11/15 William Martin

  9. Predictor Model Components local trend value Local trend noise sampled from expected Normal distribution increase Seasonal variance Adds seasonal component Set length and no. seasons Control variance Spike and slab prior zero coefficients small (equal) coefficients CREST Open Workshop 23/11/15 William Martin

  10. What does it do? Maathuis, Marloes H., and Preetam Nandy. "A review of some recent advances in causal inference." arXiv preprint arXiv:1506.07669 (2015). CREST Open Workshop 23/11/15 William Martin

  11. Causal Assumptions External events that are not accounted for by variances do not apply Meaning external events must do one of the following: Happen globally Happen in the prior time period CREST Open Workshop 23/11/15 William Martin

  12. Causal Assumptions The control data vectors are unaffected by the event (release) Non-releasing apps = control set The relationship between the target and control data vectors is unchanged in the series Control set must not contain app or derivatives CREST Open Workshop 23/11/15 William Martin

  13. Input Metrics N umber Obtain: p-value for each of ratings N umber metric, for each release of ratings / week rank of D ownloads R ating Week 1 2 …. n Release event CREST Open Workshop 23/11/15 William Martin

  14. Results - Scribblenauts Remix Posterior tail-area probability p: 0.00111 The blue region indicates prediction with 95% confidence interval CREST Open Workshop 23/11/15 William Martin

  15. Apps often have rapid / agile release cycles McIlroy et al. found that 14% of 10,713 apps updated within 2 weeks CREST Open Workshop 23/11/15 William Martin

  16. Apps often have rapid / agile release cycles McIlroy et al. found that 14% of 10,713 apps updated within 2 weeks Do releases correlate with good performance? Do releases affect performance? CREST Open Workshop 23/11/15 William Martin

  17. Dataset July 2014 - July 2015 Recorded apps that are consistently (every week) in the most popular free or paid lists: Google Play apps: 307 releases: 1,570 Windows Phone apps: 726 releases: 1,617 CREST Open Workshop 23/11/15 William Martin

  18. Developer controlled Metrics factors: P - price RT - release text Performance metrics: R - rating D - download rank N - number of ratings NW - number of ratings in last week CREST Open Workshop 23/11/15 William Martin

  19. Do app metrics change over time? CREST Open Workshop 23/11/15 William Martin

  20. Do app metrics change over time? D, N and NW have a high standard deviation over 12 months D, N and NW are likely to change R has very small standard deviation So rating is very stable, unlikely to change CREST Open Workshop 23/11/15 William Martin

  21. Do release statistics have a correlation with app performance? CREST Open Workshop 23/11/15 William Martin

  22. Do release statistics have a correlation with app performance? No strong correlations are observed number of releases release interval CREST Open Workshop 23/11/15 William Martin

  23. Do releases impact app performance? CREST Open Workshop 23/11/15 William Martin

  24. Do releases impact app performance? 40% of releases impact performance in Google apps 55% of releases impact performance in Windows apps CREST Open Workshop 23/11/15 William Martin

  25. What characterises impactful releases? CREST Open Workshop 23/11/15 William Martin

  26. What characterises impactful releases? RT - release text content size change in size P - price Day - day of release CREST Open Workshop 23/11/15 William Martin

  27. What characterises impactful releases? (new, feature) better RT - release text than (bug, fix) content size Releases that mention (new, feature) are more change in size likely to be impactful, and to positively affect Rating P - price compared with releases that mention (bug, fix) Day - day of release CREST Open Workshop 23/11/15 William Martin

  28. What characterises impactful releases? (new, feature) better RT - release text than (bug, fix) content more descriptive release text size change in size Releases with longer release text are more likely P - price to positively impact Rating Day - day of release Google Windows CREST Open Workshop 23/11/15 William Martin

  29. What characterises impactful releases? (new, feature) better RT - release text than (bug, fix) content more descriptive release text size change in size higher prices P - price Day - day of release Releases with higher prices are more likely to positively impact Rating CREST Open Workshop 23/11/15 William Martin

  30. What characterises impactful releases? (new, feature) better RT - release text than (bug, fix) content more descriptive release text size change in size higher prices P - price Day - day of release Saturday to Tuesday Releases from Saturday to Tuesday are more likely to be impactful Google Windows CREST Open Workshop 23/11/15 William Martin

  31. Conclusions Causal Impact Analysis can point to significant changes We look at groups of significant releases to minimise risk of external factors Useful developer guidelines found that apply to multiple platforms CREST Open Workshop 23/11/15 William Martin

  32. http://google.github.io/CausalImpact/CausalImpact.html CREST Open Workshop 23/11/15 William Martin

Recommend

Causal Realism and Causal Analysis in the Social Sciences Presentation by Daniel Little

Causal Realism and Causal Analysis in the Social Sciences Presentation by Daniel Little

Causal Realism and Causal Analysis in the Social Sciences Presentation by Daniel Little University of Michigan-Dearborn Acknowledgement These ideas derive from Chapters 9 and 10 of my Microfoundations, Method and Causation (Transaction

666 views • 30 slides
On Causal Analysis for Heterogeneous Networks Katerina Marazopoulou, David Arbour, David Jensen

On Causal Analysis for Heterogeneous Networks Katerina Marazopoulou, David Arbour, David Jensen

University of Massachusetts Amherst College of Information and Computer Sciences On Causal Analysis for Heterogeneous Networks Katerina Marazopoulou, David Arbour, David Jensen KDD Workshop on Causal Discovery August 2017 Causal inference in

690 views • 34 slides
Basic Concepts of Causal Mediation Analysis and Some Extensions Vanessa Didelez School of

Basic Concepts of Causal Mediation Analysis and Some Extensions Vanessa Didelez School of

Basic Concepts of Causal Mediation Analysis and Some Extensions Vanessa Didelez School of Mathematics University of Bristol Joint work with: Philip Dawid, Sara Geneletti, Svend Kreiner Symposium: Causal Mediation Analysis Ghent, January 2013

729 views • 49 slides
SRNS Causal Analysis Program Tim Worrell SRNS Contractor Assurance Manager EFCOG Working Group -

SRNS Causal Analysis Program Tim Worrell SRNS Contractor Assurance Manager EFCOG Working Group -

SRNS Causal Analysis Program Tim Worrell SRNS Contractor Assurance Manager EFCOG Working Group - April 24, 2018 Causal Analysis - Overview 226.1B Requirements The contractor assurance system must include a structured issues

594 views • 14 slides
Foundations of Causal Discovery Frederick Eberhardt KDD Causality Workshop 2016 Causal Discovery

Foundations of Causal Discovery Frederick Eberhardt KDD Causality Workshop 2016 Causal Discovery

Foundations of Causal Discovery Frederick Eberhardt KDD Causality Workshop 2016 Causal Discovery data sample x y z w samples 2 Causal Discovery assumptions, e.g. causal Markov causal faithfulness functional form etc.

1.66k views • 144 slides
Hotzone GIS analysis for targeting surveys for Phytophthora ramorum, causal agent of Sudden Oak

Hotzone GIS analysis for targeting surveys for Phytophthora ramorum, causal agent of Sudden Oak

Hotzone GIS analysis for targeting surveys for Phytophthora ramorum, causal agent of Sudden Oak Death Roger Magarey, Glenn Fowler, APHIS-CPHST; Manuel Colunga (MSU) Hotzone GIS Analysis What is GIS Hotzone analysis Examples Crude

559 views • 25 slides
COUNTERFACTUAL REASONING Jonathan Laurent , Jean Yang (Carnegie Mellon University), Walter Fontana

COUNTERFACTUAL REASONING Jonathan Laurent , Jean Yang (Carnegie Mellon University), Walter Fontana

CAUSAL ANALYSIS OF RULE-BASED MODELS THROUGH COUNTERFACTUAL REASONING Jonathan Laurent , Jean Yang (Carnegie Mellon University), Walter Fontana (Harvard Medical School) CAUSAL ANALYSIS Some techniques have been developed to analyze the causal

646 views • 33 slides
Events and Causal (Conditional+) Factors Analysis Mary Coffey ECFA and ECFA+ ECFA Buys

Events and Causal (Conditional+) Factors Analysis Mary Coffey ECFA and ECFA+ ECFA Buys

Events and Causal (Conditional+) Factors Analysis Mary Coffey ECFA and ECFA+ ECFA Buys and Clark 1995 Assists in verifying the sequence of events leading to the incident and the possible causal factors for each event Provides

233 views • 20 slides
I I Can ant Belie elieve e It Its No Not C Causal ! ! Scalable Causal Consistency

I I Can ant Belie elieve e It Its No Not C Causal ! ! Scalable Causal Consistency

I I Can ant Belie elieve e It Its No Not C Causal ! ! Scalable Causal Consistency cy wit with No o Slo lowdown wn Cas ascad ades Syed Akbar Mehdi 1 , Cody Littley 1 , Natacha Crooks 1 , Lorenzo Alvisi 1,4 , Nathan Bronson 2 ,

677 views • 55 slides
For Objec*ve Causal Inference, Design Trumps Analysis Donald B.

For Objec*ve Causal Inference, Design Trumps Analysis Donald B.

For Objec*ve Causal Inference, Design Trumps Analysis Donald B. Rubin Department of Sta*s*cs Harvard University 16 March 2012 1 Prologue to Objec*ve

1.16k views • 82 slides
Behavioral Study of Bot Obedience using Causal Relationship Analysis Pekka Pietikinen, Lari

Behavioral Study of Bot Obedience using Causal Relationship Analysis Pekka Pietikinen, Lari

Behavioral Study of Bot Obedience using Causal Relationship Analysis Pekka Pietikinen, Lari Huttunen Oulu University Secure Programming Group Botnets have become an increasing menace Tens of strategically placed hosts to hundreds of

164 views • 15 slides
Causal Programming Causal Programming Joshua Brul Joshua Brul

Causal Programming Causal Programming Joshua Brul Joshua Brul

5/2/2019 talk slides Causal Programming Causal Programming Joshua Brul Joshua Brul file:///home/josh/causal-programming-guest-lecture/slides.html?print-pdf#/ 1/67 5/2/2019 talk slides Smoking/cancer structural causal model

716 views • 67 slides
Identification of Causal Genetic Drivers of Human Disease through Systems-Level Analysis of

Identification of Causal Genetic Drivers of Human Disease through Systems-Level Analysis of

Biologists are from Venus, Mathematicians are from Mars, They cosegregate on Earth, And conditionally associate to create a DIGGIT. Identification of Causal Genetic Drivers of Human Disease through Systems-Level Analysis of Regulatory Networks

709 views • 37 slides
Causal Consistency for Distributed Data Stores and Applications as They are Kazuyuki Shudo ,

Causal Consistency for Distributed Data Stores and Applications as They are Kazuyuki Shudo ,

COMPSAC 2016 June 2016 Causal Consistency for Distributed Data Stores and Applications as They are Kazuyuki Shudo , Takashi Yaguchi Tokyo Tech Background: Distributed data store Database management system (DBMS) that consists of multiple

350 views • 12 slides
A causal analysis of mothers education on birth inequalities Silvia Bacci 1 , Francesco

A causal analysis of mothers education on birth inequalities Silvia Bacci 1 , Francesco

A causal analysis of mothers education on birth inequalities Silvia Bacci 1 , Francesco Bartolucci , Luca Pieroni Dipartimento di Economia, Finanza e Statistica - Universit di Perugia Universit La Sapienza, Roma, 20-22 June

511 views • 25 slides
On estimation of functional causal models: Post - nonlinear causal model as an

On estimation of functional causal models: Post - nonlinear causal model as an

On estimation of functional causal models: Post - nonlinear causal model as an example Kun Zhang, Zhikun W ang, Bernhard Schlkopf Dept. Empirical Inference Max Planck Institute for Intelligent Systems Tbingen, Germany 1

320 views • 20 slides
Causal and Non-Causal Feature Selection for Ridge Regression Gavin Cawley School of Computing

Causal and Non-Causal Feature Selection for Ridge Regression Gavin Cawley School of Computing

Causal and Non-Causal Feature Selection for Ridge Regression Gavin Cawley School of Computing Sciences University of East Anglia Norwich, United Kingdom gcc@cmp.uea.ac.uk Wednesday 3 rd June 2008 Introduction Causal feature selection

246 views • 13 slides
Sapporo Sapporo Namba Namba Shinjuku Shinjuku Store Store Store Store West Store West

Sapporo Sapporo Namba Namba Shinjuku Shinjuku Store Store Store Store West Store West

Sapporo Sapporo Namba Namba Shinjuku Shinjuku Store Store Store Store West Store West Store Securities Code: 3048 B A C Yurakucho Yurakucho Fujisawa Fujisawa Kashiwa Kashiwa Store Store Store Store Store Store D F

587 views • 32 slides
Introduction to Causal Inference Lan Liu University of Minnesota at Twin Cities liux3771@umn.edu

Introduction to Causal Inference Lan Liu University of Minnesota at Twin Cities liux3771@umn.edu

Introduction to Causal Inference Lan Liu University of Minnesota at Twin Cities liux3771@umn.edu 1 Table of contents Causal ... or not? How Topics in Causal Inference Tools we use... Causal Inference in Industry 2 The Danger of Ice Cream

961 views • 52 slides
Causal analysis within the framework of structural autoregressive models Alessio Moneta Scuola

Causal analysis within the framework of structural autoregressive models Alessio Moneta Scuola

Causal analysis within the framework of structural autoregressive models Alessio Moneta Scuola Superiore SantAnna a.moneta@santannapisa.it Universitat Rovira i Virgili 13 February 2018 Outline 1. Why VAR models? 2. From VAR to SVAR

1.11k views • 89 slides
Sensitivity Analysis of Linear Structural Causal Models Carlos Cinelli UCLA Joint work with

Sensitivity Analysis of Linear Structural Causal Models Carlos Cinelli UCLA Joint work with

Sensitivity Analysis of Linear Structural Causal Models Carlos Cinelli UCLA Joint work with Daniel Kumor, Bryant Chen, Judea Pearl and Elias Bareinboim ICML, Long Beach, June 2019 Motivating example: smoking and cancer 1 Lets start with

427 views • 32 slides
A Causal loop design and analysis. Research Project University of Victoria By Antonio J. Marante

A Causal loop design and analysis. Research Project University of Victoria By Antonio J. Marante

Supervised Injection Facility: A Causal loop design and analysis. Research Project University of Victoria By Antonio J. Marante Supervisory Committee Dr. Abdul Roudsari, Supervisor Dr. Scott Macdonald, Co-Supervisor Dr. Alex Kuo, Chair

528 views • 25 slides
Probabilistic Causal Analysis of Social Influence F. Bonchi 1 F. Gullo 2 B. Mishra 3 D. Ramazzotti

Probabilistic Causal Analysis of Social Influence F. Bonchi 1 F. Gullo 2 B. Mishra 3 D. Ramazzotti

Introduction Background Problem 1 Problem 2 Experiments Probabilistic Causal Analysis of Social Influence F. Bonchi 1 F. Gullo 2 B. Mishra 3 D. Ramazzotti 4 1 ISI Foundation, Italy and Eurecat, Spain, francesco.bonchi@isi.it 2 UniCredit,

484 views • 32 slides
Wren: Nonblocking Reads in a Partitioned Transactional Causally Consistent Data Store Kristina

Wren: Nonblocking Reads in a Partitioned Transactional Causally Consistent Data Store Kristina

Wren: Nonblocking Reads in a Partitioned Transactional Causally Consistent Data Store Kristina Spirovska Advisor: Willy Zwaenepoel Diego Didona Research area: Causal Consistency in Distributed Data Stores PhD Stage: Finisher EuroDW18,

477 views • 12 slides

More recommend