Point Estimates: Parameters I (7.1.1) Point Estimates: Parameters I - PowerPoint PPT Presentation

Outline: Chapter 7 Parameter Estimation Using the Sample Mean E U h S l M 2 � Point Estimates (7.1) P i E i (7 1) � Properties of Point Estimates (7.2) 204312 PROBABILITY AND 204312 PROBABILITY AND � Confidence Intervals (8.1) RANDOM PROCESSES FOR COMPUTER ENGINEERS COMPUTER ENGINEERS Lecture 10: Chapter 7 p 1st Semester, 2007 Monchai Sopitkamon, Ph.D. Point Estimates: Parameters I (7.1.1) Point Estimates: Parameters I (7.1.1) Point Estimates: Parameters II Point Estimates: Parameters II � Parameters – denote a quantity that is a property � Ex. (Machine Breakdowns): of an unknown prob distribution. Let p 0 be the prob that a machine breakdown is due to � For example, a mean, a variance, or a particular operator misuse. quantile of the prob distribution q p p 0 = P (misuse) is a parameter since it depends on the � Parameters are unknown and the goal of statistical prob distribution that governs the cause of machine inference is to estimate them inference is to estimate them b breakdowns. kd In practice, p 0 is an unknown quantity, but can be estimated from counting the number of machine ti t d f ti th b f hi breakdown causes.

Point Estimates: Statistics I Point Estimates: Statistics I Point Estimates: Estimation I Point Estimates: Estimation I � A point estimate of an unknown parameter θ is a θ i A i t ti t f k t � Statistics denote a quantity that is a property of a ˆ statistic that is a “best guess” of the value of θ . θ sample. � More than one good point estimates of a � For example, a sample mean, a sample variance, parameter is possible. or a particular sample quantile. p p q x x � Point estimates are as good as the data set from � Statistics are random variables ( Xs ) whose w c which they are calculated. ey a e ca cu a ed observed values (e.g., , s ) can be calculated observed values (e g s 2 ) can be calculated x x from a set of data observations ( x 1 , …, x n ) � So the question is how representative the sample is of the population relating to the parameter being of the population relating to the parameter being � Statistics are used to estimate unknown St ti ti d t ti t k estimated. parameters. Point Estimates: Estimation II Point Estimates: Estimation II Point Estimates: Estimation III Point Estimates: Estimation III The relationship between a point estimate Estimation of the population mean and an unknown parameter θ by the sample mean

Point Estimates: Estimation V Point Estimates: Estimation IV Point Estimates: Estimation IV Estimating the probability that a machine breakdown is due to operator misuse machine breakdown is due to operator misuse Estimating the population mean and variance of the students’ midterm scores Outline Outline Properties of Point Estimates (7.2) Properties of Point Estimates (7.2) � Point Estimates (7.1) � Point Estimates (7.1) � Two basic criteria for determining good point � Properties of Point Estimates (7.2) estimates of a particular parameter: unbiased � Confidence Intervals (8.1) estimates and minimum variance estimates. � Hypothesis Testing (8.2) � These criteria help to decide which statistics to use p as point estimates.

Properties of Point Estimates: Unbiased Properties of Point Estimates: Unbiased Estimates I Estimates II θ θ ˆ � A point estimate for a parameter θ is θ i A i t ti t f t unbiased if θ ˆ E ( ) = θ θ � The property of unbiasedness requires a θ ˆ point estimate to have a prob distribution w/ mean equal to θ � If a point estimate is not unbiased, then its bias can be defined as θ ˆ bias = E ( ) – θ The smaller the absolute value of the bias, the The smaller the absolute value of the bias the An unbiased point estimate better. Properties of Point Estimates: Unbiased Properties of Point Estimates: Unbiased Estimates III Estimates IV � A sequence of Bernoulli trials with a constant A f B lli t i l ith t t unknown success prob p . � Parameter p is estimated by conducting a sequence of n trails and X counts the number of success observed. Point estimate of p is X X p = ˆ Unbiased point estimate ? n Since X ∼ B ( n , p ) � E ( X ) = np B ( ) � E ( X ) = Si X Therefore, ⎛ ⎞ 1 ( ) ( ) 1 X = = = = = = = = ⎜ ⎜ ⎟ ⎟ ˆ ( ( ) ) E E p p E E E E X X np np p p ⎝ ⎠ n n n A biased point estimate

Properties of Point Estimates: Unbiased Properties of Point Estimates: Unbiased Estimates V Estimates VI � Ex. Machine Breakdowns E M hi B kd � Point Estimate of a Population Mean Suppose that X 1 , …, X n is a sample of If X 1 , …, X n is a sample of observations from a prob observations from a prob distribution w/ a distribution w/ a mean µ ( or E(X )) , then the sample mean µ and a variance σ 2 . The sample mean mean μ = ˆ is an unbiased point estimate of the n X ∑ ∑ X population mean µ ? popu a o ea µ + + + + � i i X X X X μ = = = = 1 1 ˆ i n X Since E ( X i ) = µ , 1 ≤ i ≤ n so that n n ⎛ ⎞ ( ) 1 n 1 n 1 ( ) ∑ ∑ ( ) is an unbiased point estimate of the population mean µ . μ = = ⎜ ⎟ = = μ = μ ˆ E E X E X E X n i i ⎝ ⎝ ⎠ ⎠ n n n = = 1 1 1 1 i i i i Properties of Point Estimates: Unbiased Properties of Point Estimates: Unbiased Estimates VII Estimates VIII 19 20 � The sample mean has expected value and � Ex.Quiz 7.1 (Y&G): Let X be an exponential RV w/ X variance E( X ) = 1. Let be the sample mean of n ind. X ( ) samples of . How many samples n are needed to Var X = = X ( ) ( ) ( ) E X E X Var X n guarantee that the variance of the sample mean X Proof: Proof is ≤ 0.01 ? 1 1 [ ] [ ] = + + = + + = � � ( ) ( ) ( ) ( ) ( ) ( ) E X E X E X E X E X E X 1 n n n From Var(X) = E(X) when X is an exp. RV., therefore From Var(X) E(X) when X is an exp. RV., therefore From Var( aY ) = a 2 Var( Y ), Var( ) = Var( X 1 +…+ X n )/ n 2 2 / 2 X Var(X) = 1. Since X i are iid, Var( X 1 +…+ X n ) = Var( X 1 )+ … + Var( X n ) ( ) ( ) 1 1 From From Var x Var x = = = = n Var( X ) ( ( ) ) 0.01 Var X n n Thus, ( ) ( ) nVar X Var x Var X = = ( ( ) ) As n approaches ∞ , variance As n approaches ∞ variance 2 2 n n Hence, we need n = 100 samples. of the sample mean approaches 0

Properties of Point Estimates: Unbiased Properties of Point Estimates: Minimum Variance Estimates VII of Estimates II f E II � Point Estimate of a Population Variance If X 1 , …, X is a sample of observations from a prob If X 1 , …, X n is a sample of observations from a prob distribution w/ a variance σ 2 . The sample variance ( ( ) ) ∑ ∑ ∑ ( ( ) ) ∑ n n − 2 2 2 2 2 2 n n − X X nX X X X = i σ = = ′ = = i 1 = i 2 2 ˆ 1 ( ) i S V X − − n 1 1 n n is an unbiased point estimate of the population variance σ 2 n ′ σ σ = = = = = = 2 � � ( ( ˆ ) ) ( ( ( ( )) )) ( ( ( ( )) )) ( ( ) ) E E E V E V X X E V E V X X Var X Var X − n n 1 n � The smaller the variance Var( ) of the point estimate, θ ˆ The unbiased point estimate 2 is better than the The unbiased point estimate 2 is better than the the better the point estimate th b tt th i t ti t unbiased point estimate 1 because it has a smaller variance Properties of Point Estimates: Minimum Variance Properties of Point Estimates: Minimum Variance of Estimates IV f E i IV of Estimates III f E III � In that case, the point estimate that provides smaller value of mean square error (MSE) is preferable. ˆ θ ˆ � By definition, MSE( ) = E (( − θ ) 2 ) , which θ y , ( ) (( ) ) , reduces to MSE( ) = Var( ) + bias 2 = Var( θ )/ n + bias 2 θ θ ˆ θ θ ˆ Var( θ )/ n + bias MSE( ) Var( ) + bias � From the previous figure, suppose that and and θ θ ˆ ≈ ≈ θ θ θ θ θ θ ˆ ≈ ≈ θ θ θ θ 2 2 ( ( 1 1 . 19 19 , 0 0 . 04 04 ) ) ( ( 1 1 . 2 2 , 0 0 . 02 02 ) ) N N N N 1 2 Comparing point preferable θ ˆ = θ + θ = θ 2 2 2 estimates with ( ) 0 . 04 ( 0 . 1 ) 0 . 05 MSE 1 different biases and different biases and ˆ θ = θ + θ = θ 2 2 2 ( ) 0 . 02 ( 0 . 2 ) 0 . 06 MSE different variances 2