Wrap up & Experimentation CS147L Lecture 8 Mike Krieger - PowerPoint PPT Presentation

Jan 02, 2024 •181 likes •1.38k views

Wrap up & Experimentation CS147L Lecture 8 Mike Krieger Wednesday, November 25, 2009 Intro Wednesday, November 25, 2009 Welcome back! Wednesday, November 25, 2009 By the end of today... - Questions from implementations - A few

Even better: Protovis - Stanford Graphics lab project - http://vis.stanford.edu/protovis/ Wednesday, November 25, 2009
Significant change? - Chi-Squared test Wednesday, November 25, 2009
Chi-Squared - Idea: measure whether a particular distribution of measures deviates significantly from expected Wednesday, November 25, 2009
Null hypothesis - Button color has no impact on click- through rate Wednesday, November 25, 2009
Testing Where: O(i) is the observed frequency, E(i) is the expected frequency degrees of freedom = (number of categories) - 1 Wednesday, November 25, 2009
Sample data Blue Red Green Total 267 267 266 800 Significant? Wednesday, November 25, 2009
Sample data Blue Red Green Total 270 250 260 800 Significant? Wednesday, November 25, 2009
Sum the differences Blue: (270-267)^2 / 267 = 0.03 Red: (250-267)^2 / 267 = 1.08 Green: (280-267)^2 / 267 = 0.63 0.03 + 1.08 + 0.63 = 1.74 = χ 2 Wednesday, November 25, 2009
Look it up in table - (or use R/SPSS/something fancier) - http://www2.lv.psu.edu/jxm57/irp/ chisquar.html Wednesday, November 25, 2009
Significant? p ~ 0.4 (we want 0.05) not significant Wednesday, November 25, 2009
Sample data Blue Red Green Total 240 230 330 800 Significant? Wednesday, November 25, 2009
Sum the differences Blue: (240-267)^2 / 267 = 2.73 Red: (230-267)^2 / 267 = 5.12 Green: (330-267)^2 / 267 = 14.86 2.73 + 5.12 + 14.86 = 22 = χ 2 Wednesday, November 25, 2009
Significant? p < 0.001 highly significant Wednesday, November 25, 2009
Wrap-up - Can be a bit of work - But can lead to amazing insights - Plus, data analysis & visualization is really fun Wednesday, November 25, 2009
Google Analytics Wednesday, November 25, 2009
Why? - It's free! - For a small company or project, much better than rolling out some of these analysis tools for yourself Wednesday, November 25, 2009
Signing up - http://analytics.google.com Wednesday, November 25, 2009
Integrating JS Insert before </body> Wednesday, November 25, 2009
Tip: Tracking JS events _trackevent function, takes (category, event, optional_key, optional_value) Wednesday, November 25, 2009
Using Web Optimizer - Attached to AdSense - Provides A/B testing tools with integrated confidence interval and significant different calculations Wednesday, November 25, 2009
Demo Wednesday, November 25, 2009
Final notes Wednesday, November 25, 2009

Recommend

Validation and Experimentation Board of Innovation The Growth Revolution Intro Validation

Validation and Experimentation Board of Innovation The Growth Revolution Intro Validation of new Ventures Experimentation Process Assumptions Hypotheses Experiment Types Metrics Wrap-up Introductions HI THERE! 3 ABOUT US

1.08k views • 86 slides

building a culture of experimentation at Spotify @bendressler - experimentation lead user

building a culture of experimentation at Spotify @bendressler - experimentation lead user retention product satisfaction # of track plays predicting things is hard hi expectations why and how we experiment - in a consumer oriented live

1.03k views • 75 slides

Dynamic Delegation of Experimentation Yingni Guo Northwestern University ngni Guo (NU)

Dynamic Delegation of Experimentation Yingni Guo Northwestern University ngni Guo (NU) Delegation of Experimentation 1 / 55 Introduction Delegating Experimentation I study how to manage innovation in a hierarchical organization. A principal

1.68k views • 119 slides

Internet-scale Experimentation The challenges of large-scale networked system experimentation and

Internet-scale Experimentation The challenges of large-scale networked system experimentation and measurements MIT Tech Review on The Loon Project The state of affairs An ever growing Internet ~3 billion people 15 billion devices

1.21k views • 107 slides

Course wrap up J uly 26, 2005 CS 486/ 686 Universit y of Wat erloo Out line Course wrap

Course wrap up J uly 26, 2005 CS 486/ 686 Universit y of Wat erloo Out line Course wrap up Final exam inf o Ot her AI courses AI j obs AI research 2 CS486/686 Lecture Slides (c) 2005 P. Poupart Agent s and

482 views • 12 slides

Course wrap up November 27, 2008 CS 486/686 University of Waterloo Outline Course wrap up

Course wrap up November 27, 2008 CS 486/686 University of Waterloo Outline Course wrap up Final exam info Other AI courses AI jobs AI research 2 CS486/686 Lecture Slides (c) 2008 P. Poupart Topics Covered

448 views • 18 slides

Experimentation in Virtual Environments Will Steptoe 29 th January 2010 Whats in this

Experimentation in Virtual Environments Will Steptoe 29 th January 2010 Whats in this lecture? 1. Introduction to experimentation in VEs 2. Experimental Design 3. Case study What is an Experiment? A set of actions and observations,

692 views • 46 slides

The Cake is Baked...now what? Let cool almost to room temperature Wrap in plastic wrap

The Cake is Baked...now what? Let cool almost to room temperature Wrap in plastic wrap twice Wrap in foil once and FREEZE **Buttercream, frosted cakes, frosted cupcakes, cookies, cookies with royal icing, cake pops all freeze great,

206 views • 4 slides

The Role and Relevance of Experimentation in Informatics Viola Schiaffonati Artificial

The Role and Relevance of Experimentation in Informatics Viola Schiaffonati Artificial Intelligence and Robotics Laboratory Politecnico di Milano Experimentation: role and relevance Starting point: philosophy of science perspective

881 views • 20 slides

YEAR END WRAP- YEAR END WRAP -UP UP A Review of Legislative, Labour and Employment Law

YEAR END WRAP- YEAR END WRAP -UP UP A Review of Legislative, Labour and Employment Law Developments in 2008 Colleen Dunlop Adrian Schofield November 28, 2008 1 www.emondharnden.com Session Overview Case Law Update Keays

694 views • 31 slides

Session 16: Wrap up 1 1. Strategic commitment (from Session 15). 2. Entry as a congestion

P1 SepOct 2012 Timothy Van Zandt Prices & Markets Page 1 Session 16 Wrap up Session 16: Wrap up 1 1. Strategic commitment (from Session 15). 2. Entry as a congestion game. 3. Wrap up. Part of An entry deterrance

425 views • 10 slides

Seminar Series C E E R Cyber-Physical Experimentation Environment for RADICS Advancing the

Seminar Series C E E R Cyber-Physical Experimentation Environment for RADICS Advancing the state of art, CEER is a game changer. A generational leap in capabilities for Cyber- Physical Experimentation in the Electric Power Grid Increased

792 views • 49 slides

Outline Course wrap up Final exam info Course wrap up Other AI courses AI jobs

Outline Course wrap up Final exam info Course wrap up Other AI courses AI jobs July 25, 2006 AI research CS 486/686 University of Waterloo 2 CS486/686 Lecture Slides (c) 2006 P. Poupart Rational Agents Agents and

592 views • 3 slides

Animal Testing Angie Bannister What is Animal Testing? animal testing, animal experimentation or

Animal Testing Angie Bannister What is Animal Testing? animal testing, animal experimentation or animal research, (refers to the experimentation carried out on animals) Cosmetics, pharmaceutical drugs, cleaning chemicals etc. tested

496 views • 13 slides

Session 16: Wrap up 1. Strategic commitment (from Session 15). 2. Entry as a congestion

Session 16: Wrap up 1. Strategic commitment (from Session 15). 2. Entry as a congestion game. 3. Wrap up. Session 16 Wrap up Slide 1 P1 SepOct 2012 Timothy Van Zandt Prices & Markets Part of An entry deterrance game

426 views • 20 slides

FOOD SERVICE UPDATE Seed Change Grant Wrap up Wrap up of the Seed Change grant which sent us

FOOD SERVICE UPDATE Seed Change Grant Wrap up Wrap up of the Seed Change grant which sent us to the Farm to School Conference in Madison, Wisconsin Added a second high tunnel, solar powered temperature control, 6 more hydroponic beds, a

228 views • 9 slides

JACKPOT: Online Experimentation of Cloud Microservices BY M. TOSLALI 1 , S. PARTHASARATHY 2 , F.

Jackpot: Online Experimentation of Cloud Microservices 7/15/2020 1 JACKPOT: Online Experimentation of Cloud Microservices BY M. TOSLALI 1 , S. PARTHASARATHY 2 , F. OLIVEIRA 2 , AND A. K. COSKUN 1 1 BOSTON UNIVERSITY; 2 IBM T.J. WATSON Talk

494 views • 18 slides

TOK presentation Exemplar 1 300 RLS: Stanford Prison Experiment KQ: Is experimentation a

TOK presentation Exemplar 1 300 RLS: Stanford Prison Experiment KQ: Is experimentation a good method of investigation in the human sciences? 553 RLS: Stanford Prison Experiment Milgram experiment experiment KQ: Is experimentation a

400 views • 5 slides

Fleet Experimentation Program Overall Classification: UNCLASSIFIED Navy Warfare Development

UNCLASSIFIED Fleet Experimentation Program Overall Classification: UNCLASSIFIED Navy Warfare Development Command UNCLASSIFIED ForwardFor the Fleet Experimentation Goals UNCLASSIFIED Why We Experiment: Improve how we fight

442 views • 13 slides

Benchmarking) A Practical Guide to Experimentation (and publics ou privs. recherche franais

HAL Id: hal-01959453 scientifjques de niveau recherche, publis ou non, panion: Proceedings of the Genetic and Evolutionary Computation Conference Companion, Jul 2018, Nikolaus Hansen. A Practical Guide to Experimentation (and Benchmarking).

970 views • 78 slides

1.4 WESTAR-WRAP-TDWG Contract with NAU ITEP Presented Tuesday, February 12, 2019 By Bill Auberle

Overview of WRAP-ITEP Task 1.4 WESTAR-WRAP-TDWG Contract with NAU ITEP Presented Tuesday, February 12, 2019 By Bill Auberle & Charlie Schlinger of EN3 Task 1.4 Tribal O&G Emissions Inventory and Outreach 1) Contact Tribes in the WRAP

479 views • 21 slides

Course wrap up CS 486/686 University of Waterloo Lecture 24: July 24, 2017 Outline Course

Course wrap up CS 486/686 University of Waterloo Lecture 24: July 24, 2017 Outline Course wrap up Final exam info (see course website) Other AI courses, options and programs AI research AI jobs 2 CS486/686 Lecture

339 views • 12 slides

Lecture 5: Wireless Physical Lecture 5: Wireless Physical Layer: Wrap-up Layer: Wrap-up Mythili

Lecture 5: Wireless Physical Lecture 5: Wireless Physical Layer: Wrap-up Layer: Wrap-up Mythili Vutukuru CS 653 Spring 2014 Jan 20, Monday Recap Radio waves as carriers of data bits in mobile physical layers Radio waves as

589 views • 10 slides

Experimentation for Speed, Safety & Learning in CD @davekarow The future is already here

Experimentation for Speed, Safety & Learning in CD @davekarow The future is already here it's just not very evenly distributed. William Gibson Coming up: What a Long Strange Trip Its Been Definitions Stories From Role

289 views • 25 slides