Outline Probability Calculations Using the Normal Distribution - PDF document

12/21/2006 219323 Probability y and Statistics for Software and Knowledge Engineers Lecture 6: The Normal Distribution Monchai Sopitkamon, Ph.D. Outline � Probability Calculations Using the Normal Distribution (5.1) � Linear Combinations of Normal Random Variables (5.2) � Approximating Distributions with the Normal Distribution (5.3) � Distributions Related to the Normal � Distributions Related to the Normal Distribution (5.4) 1

12/21/2006 Probability Calculations Using the Normal Distribution (5.1) � The most important distribution of all The most important distribution of all � Used as the basis for many statistical inference methods � Is a natural prob distribution for modeling error distributions and other natural phenomena other natural phenomena Probability Calculations Using the Normal Distribution: Definition I � Normal or Gaussian distribution has Normal or Gaussian distribution has a pdf for - ∞≤ x ≤∞ 1 2 2 = − − μ σ 2 ( x ) / f ( x ) e σ π 2 with E ( X ) = µ and Var( X ) = σ 2 � Notation X ∼ N ( µ , σ 2 ) � RV X has a normal distribution w/ mean µ and normal distribution w/ mean µ and variance σ 2 2

12/21/2006 Probability Calculations Using the Normal Distribution: Definition II Figure 5.1 The effect of changing the mean of a normal distribution Probability Calculations Using the Normal Distribution: Standard Normal Distribution I � Normal distribution with mean µ = o and variance σ 2 = 1 and variance σ = 1 1 φ = − � Pdf 2 for - ∞ ≤ x ≤ ∞ x / 2 ( x ) e π 2 x � Cdf ∫ ∫ Φ = φ φ ( ( x ) ) ( ( y y ) ) dy y − ∞ 3

12/21/2006 Probability Calculations Using the Normal Distribution: Standard Normal Distribution II Φ (x) is the cdf of a standard normal The standard distribution distribution normal distribution Probability Calculations Using the Normal Distribution: Standard Normal Distribution III If Z has standard normal distribution 1 - Φ ( x ) = P ( Z ≥ x ) = P ( Z ≤ - x ) = Φ (- x ) ( ) ( ) ( ) ( ) P ( Z ≤ 0 31) = Φ (0 31) P ( Z ≤ 0.31) Φ (0.31) Φ ( x ) + Φ (- x ) = 1 = 0.6217 (from Table I) 4

12/21/2006 Probability Calculations Using the Normal Distribution: Standard Normal Distribution IV � Table I can be used to find percentiles of the standard normal percentiles of the standard normal distribution Φ ( x ) = 0.8 x ∈ (0.84, 0.85) Table I (backward) 80 th percentile point � For α < 0.5, (1 – α ) x100 th percentile F 0 5 (1 ) 100 th il of the distribution is denoted by z α , so that Φ ( z α ) = 1 – α Critical point Probability Calculations Using the Normal Distribution: Standard Normal Distribution V If Z ∼ N (0, 1), then P (| Z | ≤ z α /2 ) = P (- z α /2 ≤ Z ≤ z α /2 ) = Φ ( z Φ ( z α /2 ) Φ ( z α /2 ) (1 /2 ) - Φ (- z /2 ) = (1 – α /2) - α /2 α /2) α /2 Φ ( z α ) = 1 – α ( ) 1 = 1 – α The critical points z α of the The critical points z α / 2 of the standard normal distribution standard normal distribution 5

12/21/2006 Probability Calculations for Normal Distribution I � If X ∼ N ( µ , σ 2 ), then If X N ( µ , σ ), then − μ X = ≈ Z N ( 0 , 1 ) σ where Z is “standardized” version of RV X , and thus the relationship: − μ − μ ⎛ b ⎛ ⎞ ⎞ ⎛ ⎛ ⎞ ⎞ b a ≤ ≤ = Φ − Φ ⎜ ⎟ ⎜ ⎟ P ( a X b ) σ σ ⎝ ⎠ ⎝ ⎠ Probability Calculations for Normal Distribution II − μ − μ ⎛ ⎞ ⎛ ⎞ b a ≤ ≤ = Φ − Φ ⎜ ⎟ ⎜ ⎟ P ( a X b ) σ σ ⎝ ⎠ ⎝ ⎠ � Ex. Suppose that X ∼ N (3, 4), then − − ∞ ∞ − ⎛ ⎛ ⎞ ⎞ ⎛ ⎛ ⎞ ⎞ 6 6 3 3 3 3 P ( X ≤ 6) = P (- ∞≤ X ≤ 6) = P ( X ≤ 6) P ( ≤ X ≤ 6) Φ − Φ ⎜ ⎜ ⎟ ⎟ ⎜ ⎜ ⎟ ⎟ ⎝ ⎠ ⎝ ⎠ 2 2 = Φ (1.5) – Φ (- ∞ ) = 0.9332 – 0 = 0.9332 6

12/21/2006 Probability Calculations for Normal Distribution − μ − μ III ⎛ ⎞ ⎛ ⎞ b a ≤ ≤ = Φ − Φ ⎜ ⎟ ⎜ ⎟ P ( a X b ) σ σ ⎝ ⎠ ⎝ ⎠ � Ex. Suppose that X ∼ N (3, 4), then ⎛ − ⎞ ⎛ − ⎞ 5 . 4 3 . 0 2 . 0 3 . 0 P (2.0 ≤ X ≤ 5.4) = P (2 0 ≤ X ≤ 5 4) = Φ Φ − Φ Φ ⎜ ⎜ ⎟ ⎟ ⎜ ⎜ ⎟ ⎟ ⎝ ⎠ ⎝ ⎠ 2 . 0 2 . 0 = Φ (1.2) – Φ (-0.5) = 0.8849 – 0.3085 = 0.5764 Probability Calculations for Normal Distribution IV � In general, if X ∼ N ( µ , σ 2 ), P ( µ - c σ ≤ X ≤ µ+c σ ) = P (- c ≤ Z ≤ c ) where c = 1, 2, 3 7

12/21/2006 Probability Calculations Using the Normal Distribution: Examples (5.1.4) � Ex.37 pg.224: ⎛ − μ ⎞ ⎛ − ∞ − μ ⎞ 10 . 5 P ( X ≤ 10.5) = P (- ∞≤ X ≤ 10.5) = Φ ⎜ ⎟ − Φ ⎜ ⎟ σ σ σ σ ⎝ ⎝ ⎠ ⎠ ⎝ ⎝ ⎠ ⎠ − − ∞ − ⎛ ⎞ ⎛ ⎞ 10 . 5 11 11 = Φ − Φ ⎜ ⎟ ⎜ ⎟ ⎝ ⎠ ⎝ ⎠ 0 . 3 0 . 3 = Φ − − Φ −∞ = − = ( 1 . 67 ) ( ) 0 . 0475 0 0 . 0475 Outline � Probability Calculations Using the Normal Distribution (5.1) � Linear Combinations of Normal Random Variables (5.2) � Approximating Distributions with the Normal Distribution (5.3) � Distributions Related to the Normal � Distributions Related to the Normal Distribution (5.4) 8

12/21/2006 Linear Combinations of Normal Random Variables I (5.2) � Linear Function of a Normal RV If X ∼ N ( µ , σ 2 ) and a and b are constants, then Y = aX + b ∼ N ( aµ+b, a 2 σ 2 ) Linear Combinations of Normal Random Variables II (5.2) � The Sum of Two Independent Normal p RVs If X 1 ∼ N ( µ 1 , σ 1 2 ) and X 2 ∼ N ( µ 2 , σ 2 2 ) are independent RVs, then Y = X 1 + X 2 ∼ N ( µ 1 + µ 2 , σ 1 2 + σ 2 2 ) Note: If two normal RVs are not independent, then their sum is still normally distributed, but the variance of the sum depends on the covariance of the two RVs. 9

12/21/2006 Linear Combinations of Normal Random Variables III (5.2) Linear Combinations of Normal Random Variables IV (5.2) � Linear Combinations of Independent Linear Combinations of Independent Normal RVs If X i ∼ N ( µ i , σ i 2 ), 1 ≤ i ≤ n , are independent RVs and if a i , 1 ≤ i ≤ n and b are constants, then Y = a 1 X 1 + … + a n X n + b ∼ N ( µ , σ 2 ) where µ = a 1 µ 1 + … + a n µ n + b and σ 2 = a 1 2 + … + a n 2 σ 1 2 σ n 2 10

12/21/2006 Linear Combinations of Normal Random Variables V (5.2) � Averaging Independent Normal RVs g g p If X i ∼ N ( µ , σ 2 ), 1 ≤ i ≤ n , are independent RVs, then their average is distributed X ⎛ ⎞ , σ 2 ⎜ ⎟ ≈ μ X N ⎜ ⎟ ⎝ ⎠ n Note: the averaging reduces the variance to σ 2 / n , so that the average has a tends to X be closer to the mean value µ than the individual RV X i ’s Outline � Probability Calculations Using the Normal Distribution (5.1) � Linear Combinations of Normal Random Variables (5.2) � Approximating Distributions with the Normal Distribution (5.3) � Distributions Related to the Normal � Distributions Related to the Normal Distribution (5.4) 11

12/21/2006 Approximating Distributions with the Normal Distribution (5.3) � Normal distribution can be used to Normal distribution can be used to provide good approximation to the prob values of certain other distributions. � The cdf of a complicated distributions (e.g., with large n ), can be related to (e.g., with large n ), can be related to the cdf of a normal distribution (easier to calculate) The Normal Approximation to the Binomial Distribution I (5.3.1) � The prob values of a B ( n , p ) distribution can be approximated by those of a N ( np , np (1 – p ) distribution N ( np np (1 p ) distrib tion � If X ∼ B ( n , p ), then ⎛ ⎞ + − x 0 . 5 np ⎜ ⎟ ≤ ≈ Φ P ( X x ) ⎜ ⎟ − ⎝ np ( 1 p ) ⎠ and ⎛ ⎞ − − x x 0 0 . . 5 5 np np ⎜ ⎜ ⎟ ⎟ ≥ ≈ Φ P ( X x ) ⎜ ⎟ − ⎝ np ( 1 p ) ⎠ Under the conditions that np ≥ 5 and n (1 - p ) ≥ 5 See http://www.ruf.rice.edu/~lane/stat_sim/binom_demo.html 12

12/21/2006 The Normal Approximation to the Binomial Distribution II (5.3.1) Approximating P(X ≤ 5) Approximating P( 8 ≤ X ≤ probability from a normal probability from a normal 11) with a probability from a 11) with a probability from a distribution normal distribution Since np = 16x0.5 = 8 ≥ 5 and n (1 – p ) = 16x0.5 = 8 ⎛ ⎞ ⎛ ⎞ + − − − x 0 . 5 np x 0 . 5 np ⎜ ⎟ ⎜ ⎟ ≤ ≈ Φ ≥ ≈ Φ P ( X x ) P ( X x ) ⎜ ⎟ ⎜ ⎟ − − ⎝ np ( 1 p ) ⎠ ⎝ np ( 1 p ) ⎠ The Central Limit Theorem (5.3.2) � If X 1 , …, X n is a sequence of independent identically distributed RVs with a mean µ and a variance σ 2 , then the distribution of 2 + + L their average X X can be = 1 n X n approximated by a ⎛ ⎞ , σ 2 ⎜ ⎟ μ distribution. N ⎜ ⎟ ⎝ ⎠ n � Similarly, the distribution of the sum X 1 + … + X n can be approximated by a ( ) distribution μ n , σ Binomial case Binomial case 2 N n 13