UCSD Robustness Summer School David Donoho 20190812 David Donoho - PowerPoint PPT Presentation

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? UCSD Robustness Summer School David Donoho 20190812 David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Outline What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Questions for Today ◮ What is Statistics? ◮ What is Statistical Research? ◮ How did ‘Robustness in Statistics’ Emerge? ◮ What has ‘Robustness in Statistics’ Delivered? ◮ How is any of this related to what CS people call robustness? David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Two possible definitions: ◮ Academic discipline Mathematical of inference from data sampled from generative models ◮ Professional discipline Organizing term for professionally trained data analysts David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Helpful framing for CS people When you are raising money, its AI . when you are hir- ing, it’s machine learning . When you are actually doing it, it’s statistics . – Various . David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Some history of Academic Statistics ◮ Cardano, Pascal ◮ John Arbuthnott & Nicholas Bernoulli (1710’s) ◮ John Graunt (1662) ◮ Abraham DeMoivre (1718) ◮ Bayes (1763), Laplace (ca. 1783,1813) ◮ Gauss (1809) + Legendre (1805) ◮ Chebyshev (ca. 1860) ◮ Galton (1880) ◮ Pearson (1890) ◮ Student (1905) ◮ Fisher (1920’s) ◮ Neyman-Pearson (1932) ◮ Kolmogorov (1932) ◮ Wald (1940’s) ◮ Tukey (1960’s) See Books of Stephen Stigler, eg. Statistics on the Table . David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Killer App: Scientific inference ◮ 1 Million Papers/year (many: clinical medicine) ◮ Large fraction use inference ◮ P -values ◮ Type I, Type II errors, P Values ◮ Confidence statements ◮ (Generalized) Linear Model hypothesis tests Common Situations ◮ Small data ◮ Formal tests ◮ Prescriptive conclusions David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Achievements of Scientific Inference ◮ Green Revolution ◮ Cancer research ◮ Surgical procedures ◮ Quality manufacture David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Deliverables ◮ Generative Models ◮ Theoretical Frameworks ◮ Data Analysis Tools David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Deliverables, I: Generative Models ◮ Normal Distribution ◮ bivariate Normal Distribution ◮ Chi-squared, F ◮ Binomial Poisson ◮ Poisson Processes ◮ Brownian Motion David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Deliverables, II: Theoretical Frameworks/Analyses Theoretical Frameworks ◮ Optimality ◮ Decision Theory ◮ Bayes Decision Theory ◮ Minimax Decision Theory ◮ Design of Experiments ◮ Asymptotic Analysis ◮ Universality (e.g. Central Limit Theorem) ◮ Ill-Posed problems and Regularization David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Deliverables, III: Data Analysis Tools Data Analysis Tools ◮ Fisher, analysis of variance ◮ Fisher, Yates, Design of Experiments ◮ Survival Analysis, censoring ◮ Fixed and Random effects analysis ◮ Observational Studies, Causal inference ◮ Time Series methods ◮ Multivariate methods ◮ Robust methods ◮ Regularization methods ◮ . . . ◮ Meta Analysis David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Successes outside ‘Statistics’ strictu sensu ◮ Statistical Signal Processing in Radar/RF ◮ Matched Filtering & Detection ◮ Blind Equalization and Deconvolution ◮ On-line algorithms ◮ Nobel Prizes in economics/physics/chemistry ◮ Harry Markowitz – portfolio allocation ◮ Robert Engle – Volatility clustering ◮ . . . ◮ W.E. Moerner – fluourescence microscopy ◮ Higgs/CERN – Higgs mass ◮ Kip Thorne – gravitational waves David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Psychology of Statistical Research ◮ Noise is huge, signal is weak. ◮ Generative model provides driving insight ◮ Tools for use by thousands of researchers correctly ◮ Possibility to be completely misled (replication crisis) David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Goals in Statistics Research Papers develop methodology that ◮ Applies very widely ◮ Many people could actually use correctly ◮ Understood using generative model; ◮ derive from optimality; ◮ establish operating characteristics David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Stats Impact R provides a kind of universal platform for statistics ◮ Stats papers typically lead to R packages ◮ Packages supply code, data used in paper ◮ Allow replication of results ◮ Allows others to benchmark new methods on same data ◮ Allow others to use methodology easily on fresh data. David Donoho UCSD Robustness Summer School

What is Statistics? What is Statistical Research? Paradigm Shifts in Statistics How did Robustness in Statistics Emerge? What has ’Robustness in Statistics’ delivered? What has ’Robustness in Statistics’ to do with CS? Paradigm Shifts ◮ 1900 – Math Paper Paradigm ◮ 1960 – Data Analysis Process ◮ 1960 – Data Reuse Framework ◮ 1985 – Common Task Framework ◮ 2005 – Data Science Pipelines David Donoho UCSD Robustness Summer School