Estimation Theory STAT 432 | UIUC | Fall 2019 | Dalpiaz Questions? - PowerPoint PPT Presentation

Estimation Theory STAT 432 | UIUC | Fall 2019 | Dalpiaz

Questions? Comments? Concerns?

Administration • CBTF Syllabus Exam • PrairieLearn Quiz • Quiz Policy

Last Week This Week

Do You Remember STAT 400? PROBS DISTRIBUTION S = t = f( DATA ) STATISTIC I

DISTRIBUTIONS FAMILY OF - X ∼ bin ( n , p ) . . . parameters A p ( x | n , p ) = ( x ) p x (1 − p ) n − x , n ← x = 0,1,…, n , n ∈ ℕ , 0 < p < 1 ✓ PARAMETER SAMPLE SPACE SPACE pmf PROBABILITY A OUTPUTS -

DISTRIBUTIONS X ∼ N ( μ , σ 2 ) ⋅ exp [ ] , 2 − 1 x − μ 2 ( ) 1 - f ( x | μ , σ 2 ) = − ∞ < x < ∞ , − ∞ < μ < ∞ , σ > 0 σ 2 π σ # PARAM SAM }§ae€ SPACE pdf DENSITIES OUTPUTS . NII PROBS '

X ∼ N ( μ = 5, σ 2 = 4 ) O P [ X = 4] = ? = =pt¥¥ P [ X < 4] = ? - 2) phorm ( 4 = 5 , so - mean = ,

X ∼ N ( μ = 5, σ 2 = 4 ) P [2 < X < 4] = ? P [ X > 3] = ? - Z ) ) pnorn(3,mean=5,sd=Z)i¥ - ( 42,4 ) diff ( , mean = 5 , so - I prior = - *¥¥E¥

X ∼ N ( μ = 5, σ 2 = 4 ) " " " " " " " " " " P [ X > c ] = 0.75, c = ? "

PROBABILITY R IN - . . ) d * ( x , pdf/pmf = . - . . ) cdf P[Xe×)= p * ( q = , P[ x. c) =p . ) : q * ( p , = c . . . ) OBS GENERATES r * ( n , RANDOM → - . * DISTRIBUTIONS = NAMED

Expectations - N ( ux.fi ) - N/a r : ) X. Y Y X mo , w/o end STILL TRUE E [ X - Y ) ← - my u× = ' +16 r } ✓ Are [2+3×-44] 9 of WD NEED c- =

Estimation

What is Statistics ?

What is Statistics ? • Technically: The study of statistics . • Practically: The science of collecting, organizing, analyzing, interpreting, and presenting data .

What is a statistic ? f- ( DAIA )

What is a statistic ? • Technically: A function of (sample) data.

Terminology • Population: The entire group of interest. • Parameter: A (usually unknown) numeric value associated with the population. • Sample: A subset of individuals taken from the population. • Statistic: A numeric value computed using the sample data.

The Big Picture - E " " - RANDO - u , HOPEFUL 'T Sampo , no / popo O O SAMPLE 02 , - tho ) x . . t = f ( x . ,Xr . . X ) ⑤ . . . .

Random Sample • A random sample is a sample where each individual in the population is equally likely to be included. Why do we care? ASSUMPTION HD MAKE AN TO WE LIKE

What is an estimator ? • A statistic that attempts to provide a good guess for an unknown population parameter.

! cylon A Fw Parameters Estimators ( - II. * EG ) xn ) IG , ,x . . - - VARG ] ( SD " i€( - t ) Xi g2= censuses - BETA ( N , B ) MCE , MOM X , , Xr , , Xn " to - . . . P[ X - 4 ] MCE ECDF ,

Statistics are Random Variables • An Estimator / A statistic: A function that tells us what calculation we will perform after taking a sample from the population. • An Estimate / The value of a statistic: The numeric result of performing the calculation on a particular sample of data. POTENTIAL VALUE µ # VARIABLE RANDOM . OF X vs x RANDOM VARIABLE

⇒ Why so much focus on the mean? TO WANT MINIMIZE - 2aXtaJ=E[ - 2aE[x ] - a ) ) = Efx x ) a ' ' + ↳ E ¥Eaction WEE . - LEG ) c- Za O ÷ Efx - at ] - - = la=ETx]/ - SQUARED PREDICT minimize to THE MEAN ERROR LOSS .

UNBIASED ←¥¥l# = ECO ] O BIASED #§µE[ on ]

̂ Definitions bias [ ̂ θ ] ≜ 피 [ ̂ θ ] − θ θ ] ≜ 피 [ ( ] 2 var [ ̂ θ − 피 [ ̂ θ ] ) θ ] ≜ 피 [ ( ̂ θ ] + ( Bias [ ̂ 2 MSE [ ̂ ] = var [ ̂ θ ] ) 2 θ − θ )

̂ MSEEX ] MSE us X 1 , X 2 , X 3 ∼ N ( μ , σ 2 ) I - = Eft ] Bias - a EEE ]=u O n = X = 1 er er - = ∑ ¯ X i = 043 = VAR n MSE i =1 - = SIT = It Itf μ = 1 4 X 1 + 1 5 X 2 + 1 in E 6 X 3 - In - Eon Bias - = - a- u ) + II. or =¥÷m/nsEG ' - ⇐ a) + ¥ . Van = - TO 0 SMALL CLOSE WHEN U

Probability Models

Professor Professorson received the following number of emails on each of the previous 40 days: 5 6 2 5 3 3 4 1 4 4 3 4 6 2 3 6 7 1 3 3 - - - - - - 5 1 8 6 1 3 2 5 3 5 4 4 2 4 0 5 0 2 5 3 - - - • What is the probability that Professor Professorson receives three emails on a particular day? 9/40 • What is the probability that Professor Professorson receives nine emails on a particular day? 0/40

Idea: Use a Poisson distribution to model the number of emails received per day. But which Poisson distribution?

/ jpfx.SI#E o.am , y pix =D

The previous “analysis” is flawed. • The previous 40 days is not a random sample. • Why do we need a random sample? • We were just guessing and checking. • How do we know if Poisson was a reasonable distribution? • How do we pick a good lambda given some data ? • How do we know if our method for picking is good?

Maximum Likelihood

* a ' "

- f ( x IO ) " D Assume Xu X. , Xi , IF ①ne=trgm%¥g ⇒ IIf) . - . -

Fitting a Probability Distribution

5=5 data = c(4, 10, 9, 3, 2, 4, 3, 7, 3, 5) = 4) p [ × LIKE THINGS WANT ESTIMATE TO CDF Empirical F- Cx )=F[ x. D= #*n ur f[ x - 43=1/5 = # P' [ X x ) - - Discrete - → P' ( x - o ) o - -

CATEGORICAL ? NUMERIC ? DATA LOOK IS → ' ' Fi DATA THE iii. AT . i i i - i DISCRETE CONTINUOUS DISTRIBUTION ? ASSUME PROBABILITY RANGE ? - ? O , 1,2 , . . . MLE from → PARAMETERS ESTIMATE - - Q plots Q CHECK RESULTS → - > 3) P[X LIKE QUANTITIES ESTIMATE USE -3 RESULTS -

Pols ( X ) X . , Xr X. - . . . . 5=5 I I O ⑤ For IF nice is Fon ko ) h (a) =P [ X= 2) THEN hCG ) nee is = icx=g=5÷

Questions? Comments? Concerns?