Statistical inference via data science: A "tidy" approach - PowerPoint PPT Presentation

Jul 03, 2023 •161 likes •352 views

Statistical inference via data science: A "tidy" approach Albert Y. Kim Joint Math Meetings Denver CO, USA January 18, 2020 Slides available at twitter.com/rudeboybert Statistical inference via data science 2 What

Statistical inference via data science:   A "tidy" approach Albert Y. Kim Joint Math Meetings   Denver CO, USA   January 18, 2020 Slides available at twitter.com/rudeboybert
Statistical inference via   data science… 2
What is the tidyverse? From: tidyverse.org 3
Why use the tidyverse? 1. It encourages students to “play the whole game” 2. It’s transferable 3. It bridges the gap between tools for learning statistics & tools for doing statistics 4
1. It encourages students to “play the whole game” • Emphasize exploratory data analysis (EDA) • “To (data) wrangle or not to wrangle? That is the question” • IMO to do no data wrangling betrays true nature of the work From: YouTube, r4ds (2017), Perkins (2009) 5
2.a) It transfers: Data visualization From: Wilkinson (2005), ggplot2 package, TechCrunch 6
2.b) It transfers: Data wrangling Normal forms & database normalization From: Codd (1970) 7
3. It bridges the gap between tools for learning statistics & tools for doing statistics tidyverse design principle #4:   Design for humans From: McNamara (2015), Robinson blogpost, tidy tools manifesto 8
Using the tidyverse in intro stats assuming no prior algebra nor coding 1. Statistical modeling 2. Statistical inference 9
EDA to Motivate Statistical Modeling Question: Are there demographic differences in teaching evaluations? From: Chance Magazine 10
EDA to Motivate Model Selection 11
EDA to Motivate Statistical Inference A “you don’t need no PhD in Statistics” moment:   Question: Is there a difference in response? Versus just saying: “The p-value is 0!” 12
“There is only one test” From: Downey blogpost 13
infer package for “tidy” statistical inference From: Bray, Ismay, Chasnovski, Baumer, and Cetinkaya-Rundel 14
What is mean year of minting of all ! pennies? Using bootstrap resampling with replacement: library (tidyverse) library (infer) pennies_sample %>% specify (response = year) %>% generate (reps = 1000) %>% calculate (stat = "mean") 15
How to make room for the tidyverse In my opinion: • Drop (combinatorics-based) probability theory χ 2 • De-emphasize tests & ANOVA as much as feasible given upstream consequences • Lean on “There is only one test” framework • Drop asymptotic theory in favor of simulation based inference: bootstrap & permutation tests 16
Guiding Paper “Mere Renovation is Too Little Too Late: We Need to Rethink Our Undergraduate Curriculum from the Ground Up” by Cobb (2015) • Make fundamental concepts accessible • Minimize prerequisites to research • Substitute “mathematics” with “computation” as the engine of statistics 17
For more info check out: • Available free online at moderndive.com • Print copies now on sale at Taylor & Francis booth &   CRC Press website: Use discount code ASA18 • Slides available at twitter.com/rudeboybert 18
EDA to Motivate Model Selection 2017 Massachusetts Public High School Data 19

Recommend

Tidy data CLEAN IN G DATA IN P YTH ON Daniel Chen Instructor Tidy data Tidy Data paper

Tidy data CLEAN IN G DATA IN P YTH ON Daniel Chen Instructor Tidy data Tidy Data paper by Hadley Wickham, PhD Formalize the way we describe the shape of data Gives us a goal when formatting our data Standard way to organize data

462 views • 27 slides

ERACER: A Database Approach for Statistical Inference and Data Cleaning Chris Mayfield Jennifer

ERACER: A Database Approach for Statistical Inference and Data Cleaning Chris Mayfield Jennifer Neville Sunil Prabhakar Department of Computer Science, Purdue University West Lafayette, Indiana, USA SIGMOD 2010, Indianapolis Problem Real

711 views • 21 slides

Modes of Statistical Inference for Causal Efgects Plus an overview of the testing based approach

Modes of Statistical Inference for Causal Efgects Plus an overview of the testing based approach to causal inference for experiments on networks CSBS Causal Inference Workshop @ Illinois Jake Bowers Political Science & Statistics

1k views • 88 slides

Tidy data Tidy datasets are all alike but every messy dataset is messy in its own way

Tidy data Tidy datasets are all alike but every messy dataset is messy in its own way Hadley Wickham Tidy data Three rules: 1. Each variable forms a column 2. Each observation forms a row 3. Each type of observational unit forms a

851 views • 33 slides

Tidy data & tidy tools Hadley Wickham Assistant Professor / Dobelman Family Junior Chair

Tidy data & tidy tools Hadley Wickham Assistant Professor / Dobelman Family Junior Chair Department of Statistics / Rice University October 2011 Monday, October 31, 11 1. What is tidy data? 2. Data tidying (3/5) 3. Tidy tools 4. Case

1.25k views • 55 slides

Statistical Inference Review Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science

Statistical Inference Review Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ January 20, 2020 Network Science Analytics Statistical

712 views • 48 slides

Classical Sabermetrics vs. Formal Statistical Inference: Towards a Unified Approach to

Classical Sabermetrics vs. Formal Statistical Inference: Towards a Unified Approach to Quantitative Baseball Research Patrick Kilgo, Brian Schmotzer, Hillary Superak, Paul Weiss, Jeff Switchenko, Lisa Elon, Jason Lee, and Lance Waller

632 views • 48 slides

UQ, STAT2201, 2017, Lecture 6 Unit 6 Statistical Inference Ideas. 1 Statistical Inference is

UQ, STAT2201, 2017, Lecture 6 Unit 6 Statistical Inference Ideas. 1 Statistical Inference is the process of forming judgements about the parameters of a population typically on the basis of random sampling . 2 The random variables X 1 , X 2

670 views • 32 slides

Theory of statistical inference: a lazy approach to obtaining asymptotic results in parametric

Theory of statistical inference: a lazy approach to obtaining asymptotic results in parametric models Hien D. Nguyen 1 , 2 1 DECRA Research Fellow, Australian Research Council. 2 Lecturer, Department of Mathematics and Statistics, La Trobe

834 views • 67 slides

Foundations for Inference I Dajiang Liu @PHS525 Feb-09-2016 Statistical Inference

Foundations for Inference I Dajiang Liu @PHS525 Feb-09-2016 Statistical Inference Statistical inference is usually performed using a randomly selected sample from the population The estimates obtained from the sample may not actually

168 views • 14 slides

1 Introduction to Statistics and Data Analysis 2 1.1 Overview: Statistical Inference,

Probability, Statistics, and Statistical Methods 1 1 Introduction to Statistics and Data Analysis 2 1.1 Overview: Statistical Inference, Samples, Populations, and the Role of Probability 3 4 1.2 Sampling Procedures; Collection of

932 views • 54 slides

Lecture 22/Chapter 19 Finding Data in Life: 1. Part 4. Statistical Inference Ch. 19

Course Divided into Four Parts (Review) Lecture 22/Chapter 19 Finding Data in Life: 1. Part 4. Statistical Inference Ch. 19 scrutinizing origin of data Diversity of Sample Proportions Finding Life in Data: summarizing data 2. yourself or

279 views • 5 slides

Unit 1: Introduction to data To do: 3. Introduction to statistical inference Performance

Unit 1: Introduction to data To do: 3. Introduction to statistical inference Performance Assessment 1: by 11:59pm on Friday. Problem Set 3: by next Thursday before class. Sta 101 - Spring 2015 Lab 2: by next Wednesday. Readiness

296 views • 7 slides

Statistical Inference Definition : A model is a family { P ; } of possible

Statistical Inference Definition : A model is a family { P ; } of possible distributions for some random variable X . WARNING: Data set is X , so X will generally be a big vector or matrix or even more compli- cated object.)

305 views • 3 slides

Statistical Natural Language Processing Statistical models: learning, inference, estimation,

Statistical Natural Language Processing Statistical models: learning, inference, estimation, prediction ar ltekin University of Tbingen Seminar fr Sprachwissenschaft Summer Semester 2017 Statistical models: learning, inference,

497 views • 25 slides

Spatial Statistical Inference in Functional Modeling fMRI data Magnetic Resonance Imaging (fMRI)

Overview Introduction Spatial Statistical Inference in Functional Modeling fMRI data Magnetic Resonance Imaging (fMRI) data Predicted response GLM Model Examining the performace of a trend surface model Overview Inference on fMRI data

456 views • 28 slides

Data Science in the Wild Lecture 6: Running Experiments Eran Toch Data Science in the Wild,

Data Science in the Wild Lecture 6: Running Experiments Eran Toch Data Science in the Wild, Spring 2019 1 Agenda 1. About experiments 2. Statistical inference 3. Forming hypotheses 4. Designing an experiment Data Science in the Wild,

733 views • 52 slides

http://bitly.com/windle2 clickers (link emailed) Calculator (just something that can do square

Teaching team Data Analysis and Statistical Inference Introduction Professor: Dr. Jesse Windle - jbw44@stat.duke.edu Sta 101 - Spring 2015 TAs: David Clancy Duke University, Department of Statistical Science Xinyi (Chris) Li

220 views • 10 slides

Unit 1: Introduction to data 4. Introduction to statistical inference Problem set (PS) 1 is

Announcements Unit 1: Introduction to data 4. Introduction to statistical inference Problem set (PS) 1 is due Tomorrow, 12.30 pm STA 104 - Summer 2017 Performance assessment (PA) 1 is due tomorrow, 12.30 pm Duke University, Department of

355 views • 6 slides

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical

Belief Propagation Algorithm Interest Group presentation by Eli Chertkov Inference Statistical inference is the determination of an underlying probability distribution from observed data. Probabilistic Graphical Models 1 1 =

522 views • 17 slides

Statistical Inference Fall 2018 Tyson S. Barrett, PhD The Whole Idea All can be done in R Why

EDUC 7610 Chapter 4 Statistical Inference Fall 2018 Tyson S. Barrett, PhD The Whole Idea All can be done in R Why Statistical Inference? So far, weve used regression just to describe our sample But our goal is to understand the

653 views • 40 slides

STK-IN4300 Statistical Learning Methods in Data Science Statistical Boosting Boosting as a

STK-IN4300 - Statistical Learning Methods in Data Science Outline of the lecture AdaBoost Introduction algorithm STK-IN4300 Statistical Learning Methods in Data Science Statistical Boosting Boosting as a forward stagewise additive modelling

211 views • 8 slides

Empirical Comparison of Approximate Inference Algorithms for Networked Data Prithviraj Sen Lise

Empirical Comparison of Approximate Inference Algorithms for Networked Data Prithviraj Sen Lise Getoor Department of Computer Science University of Maryland, College Park. Workshop on Open Problems in Statistical Relational Learning, 2006

326 views • 16 slides

Statistical Inference for Incomplete Ranking Data: The Case of Rank-Dependent Coarsening Mohsen

Statistical Inference for Incomplete Ranking Data: The Case of Rank-Dependent Coarsening Mohsen Ahmadi Fahandar 1 ullermeier 1 es Couso 2 Eyke H In 1 Intelligent Systems Group, Paderborn University, Germany 2 Department of Statistics,

485 views • 27 slides