STAT 7032 Probability CLT part Wlodek Bryc Created: Friday, Jan 2, - PowerPoint PPT Presentation

STAT 7032 Probability CLT part Wlodek Bryc Created: Friday, Jan 2, 2014 Printed: April 15, 2020 File: Grad-Prob-2020-slides.TEX

Facts to use ϕ ( t ) = E exp( itX ) ◮ For standard normal distribution ϕ ( t ) = e − t 2 / 2 ◮ The following are equivalent: D ◮ X n − → X ◮ ϕ n ( t ) → ϕ ( t ) for all t ∈ R . ◮ If X is square integrable with mean zero and variance σ 2 then � ≤ E (min { 1 � � � ϕ ( t ) − (1 − σ 2 t 2 6 | tX | 3 , ( tX ) 2 } ) 2 ) (1) � � Proof: ϕ ( t ) = Ee − itX . This relies on two integral identities applied to x = tX ( ω ) under � x � e ix − (1 + ix − x 2 � � � ≤ | x 3 | � i � 0 ( x − s ) 2 e is ds � the integral: 2 ) � = � � 2 6 �� x � � � e ix − (1 + ix − x 2 0 ( x − s )( e is − 1) ds � ≤ x 2 � � 2 ) � = � �

Last time we used inequality | z n 1 − z n 2 | ≤ n | z 1 − z 2 | complex numbers of modulus at most 1 which we now generalize. Lemma If z 1 , . . . , z m and w 1 , . . . , w m are complex numbers of modulus at most 1 then m � | z 1 . . . z m − w 1 . . . w m | ≤ | z k − w k | (2) k =1 Proof. Write the left hand side of (2) as a telescoping sum: z 1 . . . z m − w 1 . . . w m = z 1 . . . z m − w 1 z 2 . . . z m + w 1 z 2 . . . z m − w 1 w 2 . . . z m · · · + w 1 w 2 . . . w m − 1 z m − w 1 w 2 . . . w m m � = w 1 . . . w k − 1 ( z k − w k ) z k +1 . . . z m k =1

Lindeberg’s theorem For each n we have a triangular array of random variables that are independent in each row X 1 , 1 , X 1 , 2 , . . . , X 1 , r 1 X 2 , 1 , X 2 , 2 , . . . , X 2 , r 2 . . . X n , 1 , X n , 2 , . . . , X n , rn . . . and we set S n = X n , 1 + · · · + X n , r n . We assume that random variables are square-integrable with mean zero, and we use the notation r n E ( X n , k ) = 0 , σ 2 nk = E ( X 2 n , k ) , s 2 � σ 2 n = (3) nk k =1 Definition (The Lindeberg condition) We say that the Lindeberg condition holds if r n 1 � � X 2 ∀ ε> 0 lim nk dP = 0 (4) s 2 n →∞ n | X nk | >ε s n k =1

Remark (Important Observation) Under the Lindeberg condition, we have σ 2 nk n →∞ max lim = 0 (5) s 2 k ≤ r n n Proof. � � � σ 2 X 2 X 2 nk dP ≤ ε s 2 X 2 nk = nk dP + n + nk dP | X nk |≤ ε s n | X nk | >ε s n | X nk | >ε s n So σ 2 ≤ ε + 1 � nk X 2 max max nk dP s 2 s 2 k ≤ r n k ≤ r n n n | X nk | >ε s n r n ≤ ε + 1 � � X 2 nk dP s 2 | X nk | >ε s n n k =1

Theorem (Lindeberg CLT) Suppose that for each n the sequence X n 1 . . . X n , r n is independent with mean zero. If the Lindeberg condition holds for all ε > 0 then D S n / s n − → Z. Example (Suppose X 1 , X 2 , . . . , are iid mean m variance σ 2 > 0 . Then S n = k =1 ( X k − m ) D � n 1 σ √ n − → Z .) ◮ Triangular array: X n , k = X k − m √ n σ and s n = 1. ◮ The Lindeberg condition is n ( X k − m ) 2 1 � � lim dP | X k − m | >εσ √ n σ 2 n n →∞ k =1 1 � ( X 1 − m ) 2 dP = 0 = lim | X 1 − m | >εσ √ n σ 2 n →∞ by Lebesgue dominated convergence theorem.

Proof of Lindeberg CLT I Without loss of generality we may assume that s 2 n = 1 so that � r n k =1 σ 2 nk = 1. ◮ Denote ϕ nk = E ( e itX nk ). By (1) we have � � � ϕ nk ( t ) − (1 − 1 � 2 t 2 σ 2 � � min {| tX nk | 2 , | tX nk | 3 } � nk ) � ≤ E � � � � | tX nk | 3 dP + | tX nk | 2 dP ≤ | X nk |≤ ε | X nk | >ε � � � ≤ t 3 ε X 2 nk dP + t 2 X 2 nk dP ≤ t 3 εσ 2 nk + t 2 X 2 nk dP | X nk | dP ≤ ε | X nk | >ε | X nk | >ε | z 1 . . . z m − w 1 . . . w m | ≤ � m ◮ Using (2), k =1 | z k − w k | we see that for n large enough so that 1 2 t 2 σ 2 nk < 1 � r n � � � � (1 − 1 2 t 2 σ 2 � ϕ S n ( t ) − nk ) � � � � � k =1 r n r n � � � 3 2 2 2

Proof of Lindeberg CLT II Since ε > 0 is arbitrary and t ∈ R is fixed, this shows that � r n � � � � (1 − 1 2 t 2 σ 2 lim � ϕ S n ( t ) − nk ) � = 0 � � n →∞ � � k =1 � � e − t 2 / 2 − � r n � k =1 (1 − 1 2 t 2 σ 2 It remains to verify that lim n →∞ nk ) � = 0. � � To do so, we apply the previous proof to the triangular array Z n , k = σ n , k Z k of independent normal random variables. Note that r n nk / 2 = e − t 2 / 2 e − t 2 σ 2 � ϕ � rn k =1 Z nk ( t ) = k =1 We only need to verify the Lindeberg condition for { Z nk } .

Proof of Lindeberg CLT III � � Z 2 nk dP = σ 2 x 2 f ( x ) dx nk | Z nk | >ε | x | >ε/σ nk k σ 2 So for ε > 0 we estimate (recall that � nk = 1) r n r n � � � � Z 2 σ 2 x 2 f ( x ) dx nk dP ≤ nk | Z nk | >ε | x | >ε/σ nk k =1 k =1 � x 2 f ( x ) dx ≤ max 1 ≤ k ≤ r n | x | >ε/σ nk � x 2 f ( x ) dx = | x | >ε/ max k σ nk The right hand side goes to zero as n → ∞ , because by max 1 ≤ k ≤ r n σ nk → 0 by (5). QED

Lyapunov’s theorem Theorem Suppose that for each n the sequence X n 1 . . . X n , r n is independent with mean zero. If the Lyapunov’s condition r n 1 E | X nk | 2+ δ = 0 � lim (7) s 2+ δ n →∞ n k =1 D holds for some δ > 0 , then S n / s n − → Z Proof. We use the following bound to verify Lindeberg’s condition: r n r n 1 1 � � � � X 2 | X nk | 2+ δ dP nk dP ≤ ε δ s 2+ δ s 2 | X nk | >ε s n | X nk | >ε s n n n k =1 k =1 r n 1 � E | X nk | 2+ δ ≤ ε δ s 2+ δ n k =1

Corollary Suppose X k are independent with mean zero, variance σ 2 and that sup k E | X k | 2+ δ < ∞ . Then S n / √ n D − → σ Z. Proof. Let C = sup k E | X k | 2+ δ . WLOG σ > 0. Then s n = σ √ n and � n 1 k =1 E ( | X k | 2+ δ ) ≤ Cn C σ 2+ δ n 1+ δ/ 2 = σ 2+ δ n δ/ 2 → 0, so Lyapunov’s s 2+ δ n condition is satisfied. Corollary Suppose X k are independent, uniformly bounded, and have mean Var ( S n ) D � zero. If � n Var ( X n ) = ∞ , then S n / − → N (0 , 1) . Proof. Suppose | X n | ≤ C for a constant C . Then n E | X n | 3 ≤ C s 2 1 = C � n → 0 s 3 s 3 s n n n k =1

The end Lets stop here ◮ Homework 11, due Monday - two exercises from Ch 11 of the notes. ◮ There is also a sizeable list of exercises from past prelims ◮ Things to do on Friday: ◮ CLT without Lindeberg condition, when normalization is not by variance ◮ Multivariate characteristic functions and multivariate normal distribution. Thank you