The Distribution of the Sample Mean Suppose that X 1 , X 2 , . . . , - PowerPoint PPT Presentation

ST 380 Probability and Statistics for the Physical Sciences The Distribution of the Sample Mean Suppose that X 1 , X 2 , . . . , X n are a simple random sample from some distribution with expected value µ and standard deviation σ . The most important parameter of most distributions is the expected value µ , and it is often estimated by the sample mean ¯ X . So the sampling distribution of the sample mean ¯ X plays a central role in estimating µ . Some aspects of that sampling distribution are known exactly, and for some others we have useful approximations for large n . 1 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences Mean and Standard Deviation For any n ≥ 1, the sampling distribution of ¯ X has the properties: E (¯ X ) = µ ¯ X = µ ; X = σ 2 V (¯ X ) = σ 2 n ; ¯ and hence the standard deviation of ¯ X is X = σ σ ¯ √ n 2 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences Normal Populations If X 1 , X 2 , . . . , X n are a sample from a normal distribution, then for any n ≥ 1, ¯ X is also normally distributed. We already know its expected value is µ and its standard deviation is σ/ √ n , so µ, σ 2 � � ¯ . X ∼ N n In particular, for any z , � ¯ X − µ � = Φ( z ) . P σ/ √ n ≤ z 3 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences Other Populations If X 1 , X 2 , . . . , X n are a simple random sample from any distribution with expected value µ and standard deviation σ , then for large n , ¯ X is approximately normally distributed. Again, we know its expected value is µ and its standard deviation is σ/ √ n , so µ, σ 2 � � ¯ . X ≈ N n The Central Limit Theorem states that the approximation holds in the limit: � ¯ � X − µ → Φ( z ) as n → ∞ . P σ/ √ n ≤ z 4 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences Binomial Distribution We can use the Central Limit Theorem to approximate the binomial distribution. Suppose that X 1 , X 2 , . . . , X n are the success indicators in a binomial experiment. That is, each is a Bernoulli variable with P ( X i = 1) = p for some 0 < p < 1. Then E ( X i ) = p and V ( X i ) = p (1 − p ) . 5 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences The Central Limit Theorem implies that for large n � � ¯ X − p ≈ Φ( z ) . P ≤ z � p (1 − p ) / n So, if X = X 1 + X 2 + · · · + X n = n ¯ X , then � � X − np ≈ Φ( z ) . P ≤ z � np (1 − p ) The approximation is improved by replacing X − np by X − np + 1 / 2 (a continuity correction). 6 / 13 Joint Probability Distributions Distribution of the Sample Mean

ST 380 Probability and Statistics for the Physical Sciences Distribution of a Linear Combination The sample mean ¯ X and the sample total n ¯ X are both examples of a linear combination of X 1 , X 2 , . . . , X n . A general linear combination is of the form Y = a 1 X 1 + a 2 X 2 + · · · + a n X n for some constants a 1 , a 2 , . . . , a n . For example, ¯ X is the special case a i = 1 / n , i = 1 , 2 , . . . , n . 7 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences Mean and Variance Suppose that E ( X i ) = µ i and V ( X i ) = σ 2 i , i = 1 , 2 , . . . , n . Then E ( Y ) = a 1 E ( X 1 ) + a 2 E ( X 2 ) + · · · + a n E ( X n ) = a 1 µ 1 + a 2 µ 2 + · · · + a n µ n and, if X 1 , X 2 , . . . , X n are uncorrelated, V ( Y ) = a 2 1 V ( X 1 ) + a 2 2 V ( X 2 ) + · · · + a 2 n V ( X n ) = a 2 1 σ 2 1 + a 2 2 σ 2 2 + · · · + a 2 n σ 2 n . 8 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences If X 1 , X 2 , . . . , X n are correlated, the variance becomes n n � � V ( Y ) = a i a j Cov( X i , X j ) i =1 j =1 n n − 1 n � � � a 2 = i V ( X i ) + 2 a i a j Cov( X i , X j ) i =1 i =1 j = i +1 n n − 1 n � � � a 2 i σ 2 = i + 2 a i a j σ i σ j ρ i , j . i =1 i =1 j = i +1 In the last expression, we use the definition ρ i , j = Corr( X i , X j ) = Cov( X i , X j ) . σ i σ j 9 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences The proofs of these results are straightforward if tedious, and depend on nothing more than the fact that if g ( X 1 , X 2 , . . . , X n ) and h ( X 1 , X 2 , . . . , X n ) are any two functions of X 1 , X 2 , . . . , X n , then E [ g ( X 1 , X 2 , . . . , X n ) + h ( X 1 , X 2 , . . . , X n )] = E [ g ( X 1 , X 2 , . . . , X n )] + E [ h ( X 1 , X 2 , . . . , X n )] . The earlier statements about ¯ X , that X ) = σ 2 E (¯ X ) = µ and V (¯ n , are just the special case for uncorrelated X 1 , X 2 , . . . , X n and a i = 1 / n , µ i = µ, σ i = σ, i = 1 , 2 , . . . , n . 10 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences Difference Between Two Variables We often need to compare two measurements. For example, X 1 = blood pressure before taking a medication X 2 = blood pressure 1 hour after taking medication Y = X 2 − X 1 = change in blood pressure . This is the special case n = 2 , a 1 = − 1 , a 2 = 1. 11 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences So E ( Y ) = µ 2 − µ 1 and V ( Y ) = σ 2 1 + σ 2 2 − 2 ρ 1 , 2 σ 1 σ 2 and, if ρ 1 , 2 = 0, V ( Y ) = σ 2 1 + σ 2 2 . Note that when X 1 and X 2 are uncorrelated, the variances add , because a 2 1 = a 2 2 = 1. 12 / 13 Joint Probability Distributions Distribution of a Linear Combination

ST 380 Probability and Statistics for the Physical Sciences Normal Variables If X 1 , X 2 , . . . , X n are independent and normally distributed, then any linear combination Y = a 1 X 1 + a 2 X 2 + · · · + a n X n is also normally distributed. This general result includes as a special case the fact that ¯ X is normally distributed when X 1 , X 2 , . . . , X n are independent and normally distributed. A more general Central Limit Theorem states that Y is approximately normally distributed when n is large, provided no a i X i contributes substantially to the sum. 13 / 13 Joint Probability Distributions Distribution of a Linear Combination