Unit 3: Foundations for inference Lecture 3: Decision errors, significance levels, sample size, and power Statistics 101 Thomas Leininger May 31, 2013
Visualization of the day The Flesch/Flesch-Kincaid readability tests are designed to indicate comprehension difficulty when reading a passage of contemporary academic English. http://www.guardian.co.uk/world/interactive/2013/feb/12/state-of-the-union-reading-level Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 2 / 12
Video of the day 2013 is the International Year of Statistics https://www.youtube.com/watch?feature=player embedded&v=nTBZuQR7dRc Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 3 / 12
Two-sided hypothesis testing with p-values Two-sided hypothesis testing with p-values 1 Significance level vs. confidence level 2 Statistical vs. Practical Significance 3 Statistics 101 U3 - L3: Decision errors, significance levels, sample size, and power Thomas Leininger
Two-sided hypothesis testing with p-values Two-sided hypothesis testing with p-values From yesterday: A poll by the National Sleep Foundation found that college students average about 7 hours of sleep per night. A sample of 169 Duke students yielded an average of 6.88 hours, with a standard deviation of 0.94 hours. Assuming that this is a random sample representative of all Duke students (bit of a leap of faith?) , a hypothesis test was conducted to evaluate if Duke students on average sleep less than 7 hours per night. The p-value for this hypothesis test is 0.0485. Which of the following is correct? If the research question was “Do the data provide convincing evidence that the average amount of sleep Duke students get per night is different than the national average?”, the alternative hypothesis would be different. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 4 / 12
Two-sided hypothesis testing with p-values Two-sided hypothesis testing with p-values First scenario (Duke students lower than US average) H 0 : µ = 7 H A : µ < 7 Second scenario (Duke students different than US average) H 0 : µ = 7 H A : µ � 7 Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 5 / 12
Two-sided hypothesis testing with p-values Two-sided hypothesis testing with p-values First scenario (Duke students lower than US average) H 0 : µ = 7 H A : µ < 7 Second scenario (Duke students different than US average) H 0 : µ = 7 H A : µ � 7 Hence the p-value would change as well: p-value = 0 . 0485 × 2 = 0 . 097 6.88 7.00 7.12 Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 5 / 12
Two-sided hypothesis testing with p-values Recap: Hypothesis testing framework Set the hypotheses. 1 Check assumptions and conditions. 2 Calculate a test statistic and a p-value. 3 Make a decision, and interpret it in context of the research 4 question. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 6 / 12
Two-sided hypothesis testing with p-values Recap: Hypothesis testing for a population mean Set the hypotheses 1 H 0 : µ = null value H A : µ < or > or � null value Check assumptions and conditions 2 Independence: random sample/assignment, 10% condition when sampling without replacement Normality: nearly normal population or n ≥ 30 , no extreme skew Calculate a test statistic and a p-value (draw a picture!) 3 Z = ¯ x − µ s SE , where SE = √ n Make a decision, and interpret it in context of the research 4 question If p-value < α , reject H 0 , data provide evidence for H A If p-value > α , do not reject H 0 , data do not provide evidence for H A Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 7 / 12
Significance level vs. confidence level Two-sided hypothesis testing with p-values 1 Significance level vs. confidence level 2 Statistical vs. Practical Significance 3 Statistics 101 U3 - L3: Decision errors, significance levels, sample size, and power Thomas Leininger
Significance level vs. confidence level Significance level vs. confidence level Two sided 0.95 0.025 0.025 -1.96 0 1.96 Two sided HT with α = 0 . 05 is equivalent to 95% confidence interval. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 8 / 12
Significance level vs. confidence level Significance level vs. confidence level Two sided One sided 0.95 0.9 0.025 0.025 0.05 0.05 -1.96 0 1.96 0 1.65 Two sided HT with α = 0 . 05 One sided HT with α = 0 . 05 is equivalent to is equivalent to 95% confidence interval. 90% confidence interval. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 8 / 12
Significance level vs. confidence level Agreement of CI and HT Confidence intervals and hypothesis tests agree, as long as the two methods use equivalent levels of significance / confidence. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 9 / 12
Significance level vs. confidence level Agreement of CI and HT Confidence intervals and hypothesis tests agree, as long as the two methods use equivalent levels of significance / confidence. A two sided hypothesis with threshold of α is equivalent to a confidence interval with CL = 1 − α . A one sided hypothesis with threshold of α is equivalent to a confidence interval with CL = 1 − (2 × α ) . Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 9 / 12
Significance level vs. confidence level Agreement of CI and HT Confidence intervals and hypothesis tests agree, as long as the two methods use equivalent levels of significance / confidence. A two sided hypothesis with threshold of α is equivalent to a confidence interval with CL = 1 − α . A one sided hypothesis with threshold of α is equivalent to a confidence interval with CL = 1 − (2 × α ) . If H 0 is rejected, a confidence interval that agrees with the result of the hypothesis test should not include the null value. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 9 / 12
Significance level vs. confidence level Agreement of CI and HT Confidence intervals and hypothesis tests agree, as long as the two methods use equivalent levels of significance / confidence. A two sided hypothesis with threshold of α is equivalent to a confidence interval with CL = 1 − α . A one sided hypothesis with threshold of α is equivalent to a confidence interval with CL = 1 − (2 × α ) . If H 0 is rejected, a confidence interval that agrees with the result of the hypothesis test should not include the null value. If H 0 is failed to be rejected, a confidence interval that agrees with the result of the hypothesis test should include the null value. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and power May 31, 2013 9 / 12
Significance level vs. confidence level Question A 95% confidence interval for the average waiting time at an emer- gency room is (128 minutes, 147 minutes). Which of the following is false? (a) A hypothesis test of H A : µ � 120 min at α = 0 . 05 is equivalent to this CI. (b) A hypothesis test of H A : µ > 120 min at α = 0 . 025 is equivalent to this CI. (c) This interval does not support the claim that the average wait time is 120 minutes. (d) The claim that the average wait time is 120 minutes would not be rejected using a 90% confidence interval. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and powerMay 31, 2013 10 / 12
Significance level vs. confidence level Question A 95% confidence interval for the average waiting time at an emer- gency room is (128 minutes, 147 minutes). Which of the following is false? (a) A hypothesis test of H A : µ � 120 min at α = 0 . 05 is equivalent to this CI. (b) A hypothesis test of H A : µ > 120 min at α = 0 . 025 is equivalent to this CI. (c) This interval does not support the claim that the average wait time is 120 minutes. (d) The claim that the average wait time is 120 minutes would not be rejected using a 90% confidence interval. Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and powerMay 31, 2013 10 / 12
Statistical vs. Practical Significance Two-sided hypothesis testing with p-values 1 Significance level vs. confidence level 2 Statistical vs. Practical Significance 3 Statistics 101 U3 - L3: Decision errors, significance levels, sample size, and power Thomas Leininger
Statistical vs. Practical Significance Sample Size Question All else held equal, will p-value be lower if n = 100 or n = 10 , 000 ? (a) n = 100 (b) n = 10 , 000 Statistics 101 (Thomas Leininger) U3 - L3: Decision errors, significance levels, sample size, and powerMay 31, 2013 11 / 12
Recommend
More recommend
