Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence STAT 113 Hypothesis Testing I Colin Reimer Dawson Oberlin College October 5, 2017 1 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Outline Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 2 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Self-Check (Pairs): Find a 90% CI for a Correlation • What percentiles of the bootstrap distribution are needed to get a 90% CI? Work these out before moving on. • Go to http://lock5stat.com/statkey/ and select CI for slope/correlation, under “Bootstrap CI”. • Select “Restaurant Tips (Pct Tip as a Function of Bill)” from the first dropdown menu. (Vars are % Tip and $s of Bill) • Generate 10,000 bootstrap samples, and click the “Two-tail” check box. • Click on the box that says 0.950, and change the value to 0.90 (this is the confidence level). • What is the resulting confidence interval for the population correlation? Interpret it in real world terms. Is it wider or narrower than the 95% one? Explain why that makes sense in the context of estimation. 3 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Outline Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 4 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Hypothesis Testing Sometimes we are not interested so much in a specific quantitative estimate as we are in evaluating a qualitative claim: 1. Do more people disapprove than approve of Donald Trump’s job performance? 2. Do people in the population tip more (as a %) for more expensive restaurant meals? 3. Does a new treatment work better than the old one on average in the population ? 5 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence The Lady Tasting Tea In a party in the 1920s in Cambridge, a lady (Dr. Muriel Bristol, a psychologist) claimed she could tell a cup of tea had been prepared by adding milk before or after the tea was poured. The statistician Ronald Fisher, who was also in attendance, proposed to put it to a blind taste test w/ 10 cups of tea prepared in random order. • Is her claim plausible if she gets 5 of 10 correct? 10 of 10? 9 of 10? • How much success is enough to believe her? Why? 6 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Outline Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 7 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Falsification Karl Popper: scientific theories can’t be fully verified (there is always another possible explanation), only falsified . 8 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Falsification With Randomness • When sampling, we will occasionally get strange results just by chance. So we can’t falsify absolutely. • But we can say a hypothesis is implausible if the data would be very unlikely if the hypothesis were true. 9 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence The Null Hypothesis • R.A. Fisher: Formulate the negation of your research hypothesis, and establish conditions under which it can be rejected . • Fisher called this “antihypothesis” the null hypothesis , and developed null hypothesis significance testing (NHST). 10 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence The Alternative Hypothesis • Jerzy Neyman and Egon Pearson added the idea of the alternative hypothesis to Fisher’s null hypothesis formulation. • Idea: don’t reject H 0 in a vacuum — reject in favor of another hypothesis , the alternative hypothesis (or H 1 ). • This is usually the one you set out to investigate: the drug is better; the correlation is positive. 11 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Outline Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 12 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence The Null and Alternative Hypothesis What are the null and alternative hypotheses (abbreviated H 0 and H 1 ) corresponding to each of the following research claims? • Dr. Bristol can tell the difference between cups of tea more often than random guessing. • There is a positive linear association between pH and mercury in Florida lakes. • Lab mice eat more on average when the room is light. 13 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Statistics vs. Parameters • Summary values (like mean, median, standard deviation) can be computed for populations or for samples. • In a population, such a summary value is called a parameter • In a sample, these values are called statistics , and are used to estimate the corresponding parameter Value Population Parameter Sample Statistic ¯ Mean µ X Proportion p p ˆ Correlation ρ r ˆ Slope of a Line β 1 β 1 X 1 − ¯ ¯ Difference in Means µ 1 − µ 2 X 2 . . . . . . . . . 14 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Quantifying H 0 and H 1 • Pairs/Threes (5 min.): All of these hypotheses are statements about the population (or about “long run behavior”). Can we “quantify” them by translating them into true/false statements about population parameters ? Identify the relevant population parameter for each of the following claims and state the null and alternative hypotheses (abbreviated H 0 and H 1 ), as statements about that parameter. • Dr. Bristol can tell the difference between cups of tea more often than random guessing. H 0 : p correct = 0 . 5 , H 1 : p correct > 0 . 5 , where p correct is her “long run” success rate • There is a positive linear association between pH and mercury in Florida lakes. H 0 : ρ = 0 , H 1 : ρ > 0 , where ρ is the correlation coefficient between pH and Hg in all Florida lakes • Lab mice eat more on average when the room is light. H 0 : µ light − µ dark = 0 , H 1 : µ light − µ dark > 0 , where µ are “long run”/population means for an appropriate measure of amount of food consumed 15 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Outline Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence 16 / 17
Hypothesis Testing Null Hypothesis Testing Formulating Statistical Hypotheses Preview: Quantifying Evidence Quantifying the Evidence • Idea: if our sample would be very unlikely assuming the null hypothesis ( H 0 ), but not so unlikely assuming H 1 , then we will reject H 0 and accept H 1 . • We say the result is statistically significant . • If the sample statistic is not that surprising, our test is inconclusive (the result is not statistically significant ) • Key questions: • How do we quantify how unlikely the sample is? • What are we even measuring the likelihood of? 17 / 17
Recommend
More recommend
