Central Limit Theorem Will Perkins February 14, 2013 Approximating - PowerPoint PPT Presentation

Central Limit Theorem Will Perkins February 14, 2013

Approximating Binomial Probabilities What is � Pr[ Bin ( n , 1 / 2) = n / 2 + x n / 2] Exact: � � n 2 − n = � n / 2 + x n / 2 Then use Stirling’s formula.

Reproving the WLLN Theorem Let X 1 , X 2 , . . . be iid with E X i = µ . Then X 1 + X 2 + · · · + X n D − → µ n Proof: Let U n = X 1 + X 2 + ··· + X n . n Calculate φ U n ( t ) and show it converges to e it µ as n → ∞ . φ U n ( t ) = φ X 1 ( t / n ) n

Reproving the WLLN We know that φ X 1 (0) = 1 and that it is a uniformly continuous function, so this suggests using a Taylor expansion, since t / n is very close to 0 for large n . φ X 1 ( t / n ) = φ X 1 (0) + t n φ ′ X 1 (0) + o ( t / n ) = 1 + it µ/ n + o ( t / n ) Now we raise this to the n th power, and use a familiar limit: φ U n ( t ) = φ X 1 ( t / n ) n = (1 + it µ/ n + o ( t / n )) n → e it µ as n → ∞ . e it µ is the characteristic function of the constant µ , so we’ve shown that D U n − → µ

The Central Limit Theorem Theorem Let X 1 , X 2 , . . . be iid random variables with mean µ and variance σ 2 . Then X 1 + · · · + X n − n µ D √ − → N (0 , 1) σ 2 n as n → ∞ .

Proof We’ll use a similar proof to the proof of the WLLN. Let U n = X 1 + · · · + X n − n µ √ σ 2 n Calculate: φ U n ( t ) = φ X 1 − µ ( t /σ √ n ) n

Proof Taylor expansion around 0: φ X 1 − µ ( t /σ √ n ) = φ X 1 − µ (0)+ X 1 − µ (0)+ t 2 t σ √ n φ ′ 2 σ 2 n φ ′′ � t 2 / n � X 1 − µ (0)+ o = 1 + 0 − t 2 2 n + o ( t 2 / n ) And so, raising this to the n th power gives: φ U n ( t ) = (1 − t 2 2 n + o ( t 2 / n )) n → e − t 2 / 2 as n → ∞ , and e − t 2 / 2 is the characteristic function of a N (0 , 1) RV.

History of the CLT Abraham deMoivre, 1718, The Doctrine of Chance . At this point there was no Gaussian distribution (and no Gauss), and no Fourier transform (no Fourier). DeMoivre spent 3 years in prison in France for religious reasons, then moved to England at age 21 where he met Isaac Newton. He was interested in calculating the odds in gambling games, and in particular wanted an approximation to Binomial probabilities.

Lindeberg-Feller CLT For non-identically distributed rv’s. X i ’s independent with mean 0 and finite variances. Let s 2 n = σ 2 1 + · · · + σ 2 n . Let F n be the distribution function of X n . The Lindeberg condition is that for all ǫ > 0: n 1 � x 2 dF j ( x ) = 0 � lim s 2 n →∞ | x | >ǫ s j n i =1 Then X 1 + · · · + X n D − → N (0 , 1) s n

CLT for Triangular Arrays What’s a triangular array?  X 1 , 1  X 2 , 1 X 2 , 2     X 3 , 1 X 3 , 2 X 3 , 3     X 4 , 1 X 4 , 2 X 4 , 3 X 4 , 4   . . . . . . Let S n = � n j =1 S n , j (Sum of row n ). What are conditions on the random variables so that a properly centered and normalized S n converges to a normal distribution as n → ∞ ?

CLT for Triangular Arrays 1 Independence: assume all rv’s in the array are indpendent. 2 Centering: assume E X i , j = 0 for all i , j . n , j ] → σ 2 > 0 as 3 Variances converge: assume � n j =1 E [ X 2 n → ∞ . 4 No single variance is too large: For all ǫ > 0, n � E [ X 2 lim n , j || X n , j | > ǫ ] = 0 n →∞ j =1 D → N (0 , σ 2 ) as n → ∞ . Then S n −

Examples 1 Show that the theorem for triangular arrays is a generalization of the standard CLT. 2 Show that if we have independent rv’s with uniformly bounded means and variances then a CLT holds.

Gaussian Probabilities Let Z ∼ N (0 , 1). Then 1 Pr[ − 1 ≤ Z ≤ 1] ≈ . 68 (within 1 standard deviation) 2 Pr[ − 2 ≤ Z ≤ 2] ≈ . 95 (within 2 standard deviations) 3 Pr[ − 3 ≤ Z ≤ 3] ≈ . 99 (within 3 standard deviations) Other probabilities can be estimated using the fact that Z is symmetric, and by scaling and centering non-standard Normals.