HYPOTHESIS TESTING PART II LEARNING GOALS get more intimate with p - PowerPoint PPT Presentation

INTRODUCTION TO DATA ANALYSIS HYPOTHESIS TESTING PART II

LEARNING GOALS ▸ get more intimate with p -values ▸ distribution under true H 0 ▸ relation to confidence intervals ▸ develop a basic sense of how clever math (e.g., Central Limit Theorem) helps approximate sampling distributions we don’t aim for perfect understanding of this math in this course! ▸ ▸ become able to interpret & apply some statistical tests ▸ Pearson’s -tests χ 2 ▸ z -test ▸ one-sample t -test

p -value revisit

RECAP BAYESIAN PARAMETER ESTIMATION FREQUENTIST HYPOTHESIS TESTING ▸ model captures prior beliefs ▸ model captures a hypothetically M M about data-generating process assumed data-generating process ▸ prior over latent parameters ▸ fix parameter value of interest ▸ likelihood of data ▸ likelihood of data ▸ Bayesian posterior inference using ▸ single out some aspect of the data as most important (test statistic) observed data D obs ▸ look at distribution of test statistic ▸ compare posterior beliefs to some given the assumed model parameter value of interest (sampling distribution) ▸ check likelihood of test statistic applied to the observed data D obs

P-VALUE p ( D obs ) = P ( T | H 0 ⪰ H 0, a t ( D obs ))

RELATION OF P-VALUES AND CONFIDENCE INTERVALS ▸ assumptions: ▸ p-value and CI are constructed / approximated in the same way ▸ two-sided test with and alternative H 0 : θ = θ 0 H a : θ ≠ θ 0 ▸ correspondence result: p ( D ) < α i ff θ 0 ∉ CI ( D )

approximating sampling distributions

LAW OF LARGE NUMBERS

CENTRAL LIMIT THEOREM CLT gives us information about the distribution of estimated means, e.g., as when we estimate repeatedly in different (hypothetical experiments).

Pearson’s χ 2 -tests

χ 2 PEARSON -TESTS ▸ tests for categorical data (with more than two categories) ▸ two flavors: ▸ test of goodness of fit ▸ test of independence ▸ sampling distribution is a -distribution χ 2

χ 2 -DISTRIBUTION ▸ standard normal random variables: X 1 , … X n ▸ derived RV: Y = X 2 1 + … + X 2 n ▸ it follows (by construction) that: y ∼ χ 2 - distribution( n )

PEARSON’S -TEST [GOODNESS OF FIT] χ 2 Is it conceivable that each category (= pair of music+subject choice) has been selected with the same flat probability of 0.25?

⃗ ⃗ ⃗ ⃗ FREQUENTIST MODEL FOR PEARSON’S -TEST [GOODNESS OF FIT] χ 2 p n ∼ Multinomial( p , N ) N k ( n i − np i ) 2 χ 2 = ∑ np i i =1 n FACT: The sampling distribution of is χ 2 approximately: χ 2 χ 2 ∼ χ 2 - distribution( k − 1)

⃗ ⃗ PEARSON’S -TEST [GOODNESS OF FIT] χ 2 p N n χ 2 k ( n i − np i ) 2 χ 2 = ∑ np i i =1 χ 2 ∼ χ 2 - distribution( k − 1)

⃗ ⃗ PEARSON’S -TEST [GOODNESS OF FIT] χ 2 p N n χ 2 k ( n i − np i ) 2 χ 2 = ∑ np i i =1 χ 2 ∼ χ 2 - distribution( k − 1)

⃗ ⃗ PEARSON’S -TEST [GOODNESS OF FIT] χ 2 p N n χ 2 k ( n i − np i ) 2 χ 2 = ∑ np i i =1 χ 2 ∼ χ 2 - distribution( k − 1)

PEARSON’S -TEST [GOODNESS OF FIT] χ 2 How to interpret / report the result: What about the lecturer’s conjecture that (colorfully speaking) logic + metal = 🥱 ?