Inference Statistical inference Definition: Definition: The act - PDF document

10/20/2016 Outline Statistics in medicine • Inference Lecture 2: Hypothesis testing and inference • Hypothesis testing • Type I error • Type II error Fatma Shebl, MD, MS, MPH, PhD • Confidence interval Assistant Professor Chronic Disease Epidemiology Department Yale School of Public Health Fatma.shebl@yale.edu S L I D E 0 S L I D E 1 Inference Statistical inference • Definition: • Definition: – The act or process of reaching a – The process of drawing conclusions conclusion about something from about a population from quantitative or known facts or evidence qualitative information obtained from a sample of observations using methods of statistics to describe the data and to http://www.merriam-webster.com/dictionary/inference test suitable hypothesis. Or simply, – Drawing conclusions about population from a sample. S L I D E 2 S L I D E 3 Hypothesis testing Hypothesis testing • Hypothesis: • Hypothesis testing: – Definition: Is a prediction about what the – An approach to statistical inference resulting examination of appropriately collected data will in a decision to reject or not to reject the null show hypothesis – Usually investigator starts with a research question such as “Are individuals infected with hepatitis C virus (HCV) are at excess risk of hepatocellular carcinoma (HCC)?” therefore the researcher states a hypothesis such as, “HCV infection is associated with excess risk of HCC” S L I D E 4 S L I D E 5 1

10/20/2016 Hypothesis testing 1- Postulate hypotheses • Steps of hypothesis testing • Two hypotheses are postulated • 1- Postulate hypotheses – Null • 2- Design a study, and collect data – Alternative • 3- Perform tests of statistical significance • 4- Assess the evidence • 5- Draw a conclusion S L I D E 6 S L I D E 7 1- Postulate hypotheses 1- Postulate hypotheses • Null: • Alternative – Written as H 0 – Written as H A – The hypothesis that there is no real difference – The hypothesis that a real difference exists between groups (means, proportion, …etc.) between groups (means, proportion ,…etc .) being compared being compared • eg., there is no real difference in HCC risk • eg., there is real difference in HCC risk between HCV+ and HCV- subjects. between HCV+ and HCV- subjects. – If the data is consistent with the H 0 – If the data is consistent with the H A hypothesis  fail to reject H 0 hypothesis  accept H A (reject H 0 ) – If the data is not consistent with the H 0 hypothesis  reject H 0 S L I D E 8 S L I D E 9 2- Design a study and collect data Type I error • Definition: • Choose the best study design – Rejecting the null hypothesis when it is true – Accepting the alternative hypothesis when it is false • Determine the acceptable risk of error (type I • aka. and Type II errors) – Alpha error • Calculate the required sample size based on the – False-positive error above • Generally we would like very small chance of rejecting the null when it is true (to minimize harm) • Collect the data needed to test the study’s Truth Statistical test results research question H 0 False H 0 True (there is a difference) (there is no difference) Decision Reject Correct Type I error Statistically of H 0 Power False-positive significant statistical α=P(reject H 0 | H 0 true) test Fail to Type II error Correct Statistically reject False-negative insignificant H 0 β =P(accept H 0 | H 0 false) S L I D E 10 S L I D E 11 2

10/20/2016 Type II error Power • Definition: • Definition: – Accepting the null hypothesis when it is false • Is the probability of detecting a difference if it actually exists – Rejecting the alternative hypothesis when it is true • Is the complement of type II error • Aka. • Is the true positive rate (sensitivity) – Beta error – False-negative error • Power= P(reject H 0 | H 0 false) = 1- β • Generally we would like small chance of rejecting the alternative when it is true Truth Statistical test Truth Statistical test results results – 20% is acceptable H 0 False H 0 True H 0 False H 0 True (there is a difference) (there is no difference) (there is a difference) (there is no difference) Decision Reject Correct Type I error Statistically Decision Reject Correct Type I error Statistically of H 0 Power False-positive significant of H 0 Power False-positive significant statistical α=P(reject H 0 | H 0 true) statistical α=P(reject H 0 | H 0 true) test test Fail to Type II error Correct Statistically Fail to Type II error Correct Statistically reject False-negative insignificant reject False-negative insignificant H 0 β =P(accept H 0 | H 0 false) H 0 β =P(accept H 0 | H 0 false) S L I D E 12 S L I D E 13 Power Power • Depends on the true magnitude of the association • Definition: – Strong association is easier to detect than a weak association • Is the probability of detecting a difference if it • Depends on the variance of the measure of effect actually exists – The lesser the variance in the measure of effect, the less difficult to detect it • Is the complement of type II error • Increase with • Is the true positive rate (sensitivity) – Increasing effect size (RR) • Power= P(reject H 0 | H 0 false) = 1- β – Cohort study/RCT: increasing frequency of the outcome in source population (up to about 0.5) • 80% is acceptable – Case-control study: increasing frequency of exposure in source – Interpretation: There is 80% chance that the population (up to about 0.5) statistical test will detect the proposed – Decreasing variance in the measure of effect difference, given that difference exists – Increasing sample size – Increasing significance level S L I D E 14 S L I D E 15 3- Perform test of statistical 4- Assess the evidence significance • Calculate the estimates • Compare the p value with the preselected alpha level • Calculate a test statistic – A measure of the difference between the • Inspect whether the confidence interval actual sample estimate and the population include the fixed value parameter proposed by the H 0 • Obtain the p value for the data • Calculate the confidence interval of the estimate S L I D E 16 S L I D E 17 3

10/20/2016 4- Assess the evidence 4- Assess the evidence • Alpha level: • Alpha level: – The highest acceptable risk of committing type I – Interpretation of .05 alpha level error • The investigator is willing to take not • 5% is acceptable more than 5% risk of erroneously – Arbitrary level rejecting the null hypothesis when it – <5%  reject the null is in fact true – >5%  fail to reject the null • Decided upon prior to data collection S L I D E 18 S L I D E 19 4- Assess the evidence 4- Assess the evidence P value • The probability of observing statistic as extreme or more extreme than the observed statistic given that the null hypothesis is true Shaded Shaded area=.025 of Shaded area=.025 of total area area=.05 of total area under the total area • The p value is obtained from the test of significance under the curve under the curve 1.645 SE curve 1.96 SE Two-tailed test One-tailed test One-tailed test (one-sided) Two-tailed test (two-sided) When the hypothesis is “difference in When the hypothesis is “difference in one direction” both directions” A>B or B>A A>B and A<B Figure modified from a graph generated by http://www.imathas.com/stattools/norm.html S L I D E 20 S L I D E 21 4- Assess the evidence 4- Assess the evidence P value Meaning of statistical significance • Magnitude of p value depends on • Small p value – Sample size (n): as n increases, p decreases – The results are unlikely to occur by chance – It is not equivalent to clinical or biological relevance – Standard deviation (SD): as SD increases, p – It is not equivalent to true association increases • It could be true association • It could be artifactitious due to confounding • Large p value – Null is true – The power of the study is low to detect a difference S L I D E 22 S L I D E 23 4