Teaching Confounder-Based Statistical Literacy 19 June, 2019 1 2 2019 Univ. New Mexico 2019 Univ. New Mexico 0D 0D Confounding: Teaching Confounder-Based Statistical Literacy Common Misuse Confounding is used to show that “association is Milo Schield, US not causation”. We then spend an entire semester on Fellow: American Statistical Assoc. US Rep: International Statistical Literacy Project randomness (never mentioning confounding again). This is “Bait and Switch”. June 19, 2019 “Bait and switch” is unethical! Dept. Math & Statistics “Bait and switch” is professional negligence! University of New Mexico This is arguably why most students see less value in www.StatLit.org/pdf/2019-Schield-UNM-Slides.pdf ‘statistics’ after taking the intro research-methods course – than they did before taking the course. 3 4 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico My First Day First Day #1: Coincidence (Chance) #2: Confounding Studies show: “People that read home and fashion Do some people have special powers? magazines are much more likely to get pregnant Let’s find out. Who gets longest run? than people that read car and sport magazines.” Q1. Could the winner have special powers? Q1 What’s an alternate explanation? Gender Q2. What’s another explanation? Q2 How can we see this in the data? Stratify! Luck, coincidence, chance or “skill”? Suppose the best hospital had the highest death rate. Q3. How can we find out right now? Q3. Is this strong evidence it’s a bad hospital? Patient health Q4. What’s an alternate explanation? Do it again (Repeat) Stratify! Q5. How can we see this in the data? 5 6 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Statistical Literacy: Our Students Two Approaches . Mathematics: Rationalism Define statistical literacy. Show what follows. Business: Empirical*/Teleological Who is the customer? What do they need? Today: Empirical first; Rationalist second. * See Schield (2008). Von Mises’ Frequentist Approach to Statistics www.statlit.org/pdf/2008SchieldBurnhamASA.pdf 3 citations, 2 recommendations, 500+ reads on ResearchGate. 2019-Schield-UNM-slides.pdf 1
Teaching Confounder-Based Statistical Literacy 19 June, 2019 7 2015 8 2019 Univ. New Mexico 2019 Univ. New Mexico 0D 0D Math-Stat Teachers Harvard Biz Review Cases 42K vs. Non-Math Students Word Prevalence: Abstract Math majors have highest Math SATs . 9 10 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Most students take statistics. Statistical Literacy: Our Audience/Customers NOT . Reading numbers in the news. Not just pedagogy in traditional stats: • Including major projects in statistics • Using resampling to create confidence intervals • Use of resampling to run hypothesis tests • Analyzing results of clinical trials • Analyzing results of random surveys/polls 11 12 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Statistical Literacy: Statistical Literacy: An Overview Four Kinds of Arguments Statistical literacy studies statistics Statistical literacy studies statistics as evidence in arguments. as evidence in arguments. Most statistical arguments involve observational GENERALIZATION statistics. These are easily confounded . From Some to All EXPLANATION PREDICTION Confounding is what connects statistical literacy to From Present to Past. From Past to Future. OBSERVABLES From Effect to Cause From Act to Effect the humanities, the liberal arts, the social sciences, the professions and the soft physical sciences From Group to Subject (geology, astronomy, epidemiology, etc.) SPECIFICATION 2019-Schield-UNM-slides.pdf 2
Teaching Confounder-Based Statistical Literacy 19 June, 2019 13 14 2019 Univ. New Mexico 2019 Univ. New Mexico 0D 0D Defining a “statistic” Defining a “statistic” in Traditional Statistics in Statistical Literacy In order to unpack this definition, we must be clear 1. Statistics are different from numbers. on how we define “statistic”. 2. Statistics are between number and words. 3. Statistics are numbers in context – In traditional inference-based courses, where the context matters. a statistic is a property of a sample. 4. Statistics are counts and measures of real things • Typically random samples. Consequence: Statistics can be influenced. • Typically small random samples. StatLit studies ALL the influences on a statistic 15 16 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Statistical Literacy People that shave their face Studies ALL Influences are taller… . . 17 18 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Study Design Can Study Design Can Ward Off Confounders Ward Off Confounders Experiments vs. Observational Study . Strength of Argument: Support given by the reasons (premises) -- assuming they are true Roof: point of dispute Walls: support of point if reasons are true Floor: Experiment Observational Study truth of reasons 2019-Schield-UNM-slides.pdf 3
Teaching Confounder-Based Statistical Literacy 19 June, 2019 19 20 2019 Univ. New Mexico 2019 Univ. New Mexico 0D 0D Ward off Confounders: Prevalence Quasi-Experiments Ngrams . . 21 22 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Prevalence Ngrams Prevalence Ngrams . . 23 24 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Prevalence Ngrams Three Big Contributions to Human Knowledge . 1. Association is not causation 2. The Central Limit theorem; the formula for Standard Error: statistical average error. Howard Wainer calls this “the most dangerous equation” (next to E=mc 2 ) 3. Fisher’s uses of random assignment to control for pre-existing confounders 2019-Schield-UNM-slides.pdf 4
Teaching Confounder-Based Statistical Literacy 19 June, 2019 25 26 2019 Univ. New Mexico 2019 Univ. New Mexico 0D 0D Biggest Omissions Statistical Literacy: relative to Human Knowledge Four Biggest Problems What are statistical educators biggest sins in 1. Lack of focus on confounding teaching introductory statistics? 2. Students 1. Ignoring multivariate data, observational studies • may become cynics about every statistic. and confounding. • will have less respect for our discipline. 2. Failing to show that controlling for a confounder 3. Teachers: can change statistical significance. Math/stat teachers: not trained to teach literacy. 3. Ignoring the Cornfield conditions. Math/stat teachers: don’t want to teach literacy. 4. Ignoring how definitions can influence Stat. Sig. 4. Textbook and teacher training materials 27 28 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Statistical Literacy Confounding Almost Absent and Confounding in GAISE 2005 K-12 report : The first line: “The ultimate goal: Statistical literacy: the discipline that studies: statistical literacy”. Confounding is mentioned * all the influences on a statistic. twice: once to define and once to note it may In observational studies, confounding is arguably create patterns that are not a “reliable basis for the most common – most important – influence. statistical inference”. College report : Confounding is mentioned only The statistical literacy “debate” is ultimately between the ‘pro’ and the ‘anti’ confounders. once. It is not defined; it appears in a sample Schield is – and has always been – pro-confounder. problem in a list of words that may apply in See Schield (1998) for “confounding factors”. analyzing data from an observational study. 29 30 0D 2019 Univ. New Mexico 0D 2019 Univ. New Mexico Confounding mentioned Silence on in GAISE 2016 Update Smoking and Lung Cancer Plus: Confounding shown 18 times (big increase): Extremely important observational study. Smoking is a most-likely cause of cancer. Twice up front: MINUS: Goal 9: Ethics: “with large data sets, … under-standing confounding … becomes even more relevant.” p 11 • Not in most statistics textbooks. • Recommendation: Multivariable thinking. Examples • Not mentioned in GAISE 2005 College. “show how confounding plays an important role…” p.15 PLUS: Nine times in appendix B: 34, 38 (3), 40 (2), 41(3) • Discussed in detail in GAISE 2004 K-12. Seven times later: Footnote 105; 113, 120, 122 (4). But confounding was never used in the discussion Minus: Not in any one-line recommendations/goals 2019-Schield-UNM-slides.pdf 5
Recommend
More recommend
