Biostatistics 602 - Statistical Inference March 19th, 2013 - PowerPoint PPT Presentation

can also be represented that W n is close to Consistency implies that the probability of W n close to . Recap . are unknown as its asymptotic properties. When the sample size n approaches infinity, the behaviors of an estimator Asymptotic Evaluation of Point Estimators Summary . Asymptotic Efficiency Asymptotic Normality . . . . . . . . . . . . .. . . .. . . .. .. Definition - Consistency P . Pr W n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . 1 as n goes to approaches to . When W n n .. lim Pr W n n lim . ) means that, given any (converges in probability to P W n . . . .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 4 / 33 .. .. .. . . .. . . . .. . . . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . Let W n = W n ( X 1 , · · · , X n ) = W n ( X ) be a sequence of estimators for τ ( θ ) . We say W n is consistent for estimating τ ( θ ) if W n → τ ( θ ) under P θ for every θ ∈ Ω .

can also be represented that W n is close to Consistency implies that the probability of W n close to . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . .. Summary . .. .. . .. . . . When the sample size n approaches infinity, the behaviors of an estimator Asymptotic Evaluation of Point Estimators lim March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . 1 as n goes to approaches to . When W n lim . P W n P . . Definition - Consistency . are unknown as its asymptotic properties. .. . . . . .. . .. .. . . .. . .. .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . . 4 / 33 . .. . . . .. . . . . . . . . . . . . . . . . . . . . . Let W n = W n ( X 1 , · · · , X n ) = W n ( X ) be a sequence of estimators for τ ( θ ) . We say W n is consistent for estimating τ ( θ ) if W n → τ ( θ ) under P θ for every θ ∈ Ω . → τ ( θ ) (converges in probability to τ ( θ ) ) means that, given any ϵ > 0 . n →∞ Pr ( | W n − τ ( θ ) | ≥ ϵ ) = 0 n →∞ Pr ( | W n − τ ( θ ) | < ϵ ) = 1

Consistency implies that the probability of W n close to . .. Asymptotic Normality Recap . . . . . . . . . . . .. . . . . . .. . .. .. . . .. Asymptotic Efficiency Summary . lim March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . 1 as n goes to approaches to lim P Asymptotic Evaluation of Point Estimators W n P . . Definition - Consistency . are unknown as its asymptotic properties. When the sample size n approaches infinity, the behaviors of an estimator . . .. . . .. . . . . . .. . .. .. . . .. . . .. . . . .. . .. . .. . . .. . . .. . . 4 / 33 . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Let W n = W n ( X 1 , · · · , X n ) = W n ( X ) be a sequence of estimators for τ ( θ ) . We say W n is consistent for estimating τ ( θ ) if W n → τ ( θ ) under P θ for every θ ∈ Ω . → τ ( θ ) (converges in probability to τ ( θ ) ) means that, given any ϵ > 0 . n →∞ Pr ( | W n − τ ( θ ) | ≥ ϵ ) = 0 n →∞ Pr ( | W n − τ ( θ ) | < ϵ ) = 1 When | W n − τ ( θ ) | < ϵ can also be represented that W n is close to τ ( θ ) .

. .. . .. . . .. . . . Recap . .. . . .. . . . . . . . . . . . . Asymptotic Normality . P March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang lim lim P W n . Asymptotic Efficiency . Definition - Consistency . are unknown as its asymptotic properties. When the sample size n approaches infinity, the behaviors of an estimator Asymptotic Evaluation of Point Estimators Summary . .. .. . . . . .. . .. .. . .. . . . .. . . .. . . .. . 4 / 33 .. . . .. . . .. . . . . .. .. . .. . . . . . . . . . . . . . . . . . . . . . . . . . Let W n = W n ( X 1 , · · · , X n ) = W n ( X ) be a sequence of estimators for τ ( θ ) . We say W n is consistent for estimating τ ( θ ) if W n → τ ( θ ) under P θ for every θ ∈ Ω . → τ ( θ ) (converges in probability to τ ( θ ) ) means that, given any ϵ > 0 . n →∞ Pr ( | W n − τ ( θ ) | ≥ ϵ ) = 0 n →∞ Pr ( | W n − τ ( θ ) | < ϵ ) = 1 When | W n − τ ( θ ) | < ϵ can also be represented that W n is close to τ ( θ ) . Consistency implies that the probability of W n close to τ ( θ ) approaches to 1 as n goes to ∞ .

• Chebychev’s Inequality Need to show that both Bias W n and Var W n converges to zero . . . . .. . . .. . . .. . . .. . . .. .. Recap . . . . . . . . . . MSE W n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Var W n W n Bias E W n .. Pr W n Pr W n Tools for proving consistency Summary . Asymptotic Efficiency Asymptotic Normality . . .. .. . .. . . .. . . . . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . . .. . . .. 5 / 33 . . . . . . . . . . . . . . . . . . . . . • Use definition (complicated)

Need to show that both Bias W n and Var W n converges to zero . .. . . .. . . . . . .. . . .. . .. .. . Asymptotic Efficiency March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Tools for proving consistency Summary . Asymptotic Normality .. Recap . . . . . . . . . . . .. . . . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. .. . . .. 5 / 33 . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . • Use definition (complicated) • Chebychev’s Inequality Pr (( W n − τ ( θ )) 2 ≥ ϵ 2 ) Pr ( | W n − τ ( θ ) | ≥ ϵ ) = E [ W n − τ ( θ )] 2 ≤ ϵ 2 = Bias 2 ( W n ) + Var ( W n ) MSE ( W n ) = ϵ 2 ϵ 2

. . . . .. . . .. . . .. . . .. . . .. . . Asymptotic Efficiency March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Tools for proving consistency Summary . Asymptotic Normality .. Recap . . . . . . . . . . . .. . . .. .. . . . . .. . . .. . . .. . . .. . . .. .. . . . . .. .. . . .. 5 / 33 . .. . . . . .. . . . . . . . . . . . . . . . . . . . . . • Use definition (complicated) • Chebychev’s Inequality Pr (( W n − τ ( θ )) 2 ≥ ϵ 2 ) Pr ( | W n − τ ( θ ) | ≥ ϵ ) = E [ W n − τ ( θ )] 2 ≤ ϵ 2 = Bias 2 ( W n ) + Var ( W n ) MSE ( W n ) = ϵ 2 ϵ 2 Need to show that both Bias ( W n ) and Var ( W n ) converges to zero

. . . . .. . . .. . . .. . . .. . . .. .. . . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . . Theorem 10.1.3 Theorem for consistency .. Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . . .. .. .. . . . .. . . . . . .. . . .. . . . .. .. .. . .. . . .. . . . . . .. . . .. . 6 / 33 . . . . . . . . . . . . . . . . . . . . . If W n is a sequence of estimators of τ ( θ ) satisfying • lim n − > ∞ Bias ( W n ) = 0 . • lim n − > ∞ Var ( W n ) = 0 . for all θ , then W n is consistent for τ ( θ )

. .. .. . . .. . . . . . .. . . .. . .. . .. . Theorem 5.5.2 March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang P . . . . Weak Law of Large Numbers Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . . .. . . . . . .. . . .. . . . .. .. .. . . .. . . .. . . . . . .. . . .. . 7 / 33 . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n be iid random variables with E ( X ) = µ and Var ( X ) = σ 2 < ∞ . Then X n converges in probability to µ . → µ . i.e. X n

b n is also a consistent sequence of estimators of If W n is consistent for and g is a continuous function, then g W n is . Asymptotic Normality . . Theorem 10.1.5 . Consistent sequence of estimators Summary . Asymptotic Efficiency . . . . . . . . . . Recap . . .. . . .. . . .. sequences of constants satisfying . . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . consistent for g . . . . . . . . Continuous Map Theorem . . a n W n Then U n . .. . .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. 8 / 33 . . . . . . . . . . . . . . . . . . . . . Let W n is a consistent sequence of estimators of τ ( θ ) . Let a n , b n be 1 lim n →∞ a n = 1 2 lim n →∞ b n = 0 .

If W n is consistent for and g is a continuous function, then g W n is . . . Consistent sequence of estimators Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . .. . . .. . .. Theorem 10.1.5 sequences of constants satisfying . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . consistent for g . . . . . . . . Continuous Map Theorem . . . . . .. . . .. .. . . .. . . .. . . . .. . .. . . .. . . .. . . . 8 / 33 . .. . . .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Let W n is a consistent sequence of estimators of τ ( θ ) . Let a n , b n be 1 lim n →∞ a n = 1 2 lim n →∞ b n = 0 . Then U n = a n W n + b n is also a consistent sequence of estimators of τ ( θ ) .

. . Recap . . . . . . . . . . . .. . . .. . Asymptotic Efficiency .. . . .. . . .. . Asymptotic Normality . .. . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . . Continuous Map Theorem . . Summary . . sequences of constants satisfying . . Theorem 10.1.5 . Consistent sequence of estimators . .. . . . . . .. . . .. . .. . . . .. . . .. . . . .. 8 / 33 .. .. . . .. . . .. . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Let W n is a consistent sequence of estimators of τ ( θ ) . Let a n , b n be 1 lim n →∞ a n = 1 2 lim n →∞ b n = 0 . Then U n = a n W n + b n is also a consistent sequence of estimators of τ ( θ ) . If W n is consistent for θ and g is a continuous function, then g ( W n ) is consistent for g ( θ ) .

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . . .. . . Recap Asymptotic Efficiency . 1 Propose a consistent estimator of the median. March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang c where c is constant. 2 Propose a consistent estimator of Pr X . . . . . i.i.d. . . Problem . Example - Exponential Family Summary .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . 9 / 33 . . . . .. . . .. . . . .. . . .. . . .. .. . . . . . . . . . . . . . . . . . . . . . Suppose X 1 , · · · , X n ∼ Exponential ( β ) .

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . . .. . . Recap Asymptotic Efficiency . 1 Propose a consistent estimator of the median. March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang c where c is constant. 2 Propose a consistent estimator of Pr X . . . . . i.i.d. . . Problem . Example - Exponential Family Summary .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . 9 / 33 . . . . .. . . .. . . . .. . . .. . . .. .. . . . . . . . . . . . . . . . . . . . . . Suppose X 1 , · · · , X n ∼ Exponential ( β ) .

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . . 1 Propose a consistent estimator of the median. . i.i.d. Asymptotic Normality . . Problem . Example - Exponential Family Summary . Asymptotic Efficiency . .. .. .. . .. . . .. . . . . . .. . . .. . . . .. . . . .. . . .. . . .. 9 / 33 . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Suppose X 1 , · · · , X n ∼ Exponential ( β ) . 2 Propose a consistent estimator of Pr ( X ≤ c ) where c is constant.

c X is consistent for . Asymptotic Normality Recap . . . . . . . . . . . .. . . .. . . . .. . . .. . .. .. Asymptotic Efficiency Summary . Pr X March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang g c e c e e By continuous mapping Theorem, g X . is continuous function of c e , As X is consistent for c . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . 10 / 33 .. . . .. . . .. . . .. . . .. . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . Consistent estimator of Pr ( X ≤ c ) ∫ c 1 β e − x / β dx Pr ( X ≤ c ) = 0

As X is consistent for c X is consistent for . . . . . . . . . . . . .. . . .. . . Asymptotic Normality .. . . .. . .. .. . Recap . Asymptotic Efficiency Pr X March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang g c e c e .. By continuous mapping Theorem, g X . is continuous function of c e , Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 10 / 33 . . . . .. .. . .. . .. . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . Consistent estimator of Pr ( X ≤ c ) ∫ c 1 β e − x / β dx Pr ( X ≤ c ) = 0 1 − e − c / β =

c X is consistent for . . .. . . .. . . .. .. . . .. .. . .. . . . .. c March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang g c e Pr X . . . . . . . . . . e By continuous mapping Theorem, g X Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . .. .. . . . .. . . . . . .. . . .. . . . 10 / 33 .. . . . .. . . . .. . .. .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Consistent estimator of Pr ( X ≤ c ) ∫ c 1 β e − x / β dx Pr ( X ≤ c ) = 0 1 − e − c / β = As X is consistent for β , 1 − e − c / β is continuous function of β .

. .. . . .. . .. .. . . .. . . .. . . . . Asymptotic Normality March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Summary . Asymptotic Efficiency Recap . . . . . . . . . . . . .. . . .. .. . . . . . .. . .. .. . . .. . . .. . . .. . . . . . .. .. . . .. 10 / 33 . .. . . . . .. . . . . . . . . . . . . . . . . . . . . . Consistent estimator of Pr ( X ≤ c ) ∫ c 1 β e − x / β dx Pr ( X ≤ c ) = 0 1 − e − c / β = As X is consistent for β , 1 − e − c / β is continuous function of β . By continuous mapping Theorem, g ( X ) = 1 − e − c / X is consistent for Pr ( X ≤ c ) = 1 − e − c / β = g ( β )

. .. . .. . . .. . . . Recap . .. . . .. . . . . . . . . . . . . Asymptotic Normality . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang is consistent for p by Law of Large Numbers. c I X i i n Asymptotic Efficiency Y i i n n Y i.i.d. Summary . .. .. . . .. . . .. . . .. . .. . . . .. . . .. . . .. 11 / 33 . . .. . . .. . . .. . . .. .. . . . . .. . . . . . . . . . . . . . . . . . . . . . . Consistent estimator of Pr ( X ≤ c ) - Alternative Method Define Y i = I ( X i ≤ c ) . Then Y i ∼ Bernoulli ( p ) where p = Pr ( X ≤ c ) .

. .. .. . . .. . . . . . .. .. . .. . . . .. . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang is consistent for p by Law of Large Numbers. n n n Y . i.i.d. Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . .. . .. . . . . . .. . . .. . . . 11 / 33 .. . . . . . .. .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Consistent estimator of Pr ( X ≤ c ) - Alternative Method Define Y i = I ( X i ≤ c ) . Then Y i ∼ Bernoulli ( p ) where p = Pr ( X ≤ c ) . 1 Y i = 1 ∑ ∑ = I ( X i ≤ c ) i =1 i =1

. .. .. . . .. . . . . . .. . . .. .. . . .. . Theorem 10.1.6 - Consistency of MLEs March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang i.i.d. Suppose X i . . . . Consistency of MLEs Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . . .. . . . . . .. . . .. . . . .. .. .. . . .. . . .. . . . . . .. . . .. . 12 / 33 . . . . . . . . . . . . . . . . . . . . . ∼ f ( x | θ ) . Let ˆ θ be the MLE of θ , and τ ( θ ) be a continuous function of θ . Then under ”regularity conditions” on f ( x | θ ) , the MLE of τ ( θ ) (i.e. τ (ˆ θ ) ) is consistent for τ ( θ ) .

. . Recap . . . . . . . . . . . .. . . .. . Asymptotic Efficiency .. . . .. . . .. . Asymptotic Normality . .. : ”asymptotic mean” March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . n We denote W n : ”asymptotic variance” • • Summary d where d . . Definition: Asymptotic Normality . Asymptotic Normality . .. . . .. . . .. . . .. . .. . . . .. . . .. . . . . .. . .. . . .. . . .. . . .. . .. . . .. . . 13 / 33 . . . . . . . . . . . . . . . . . . . . . A statistic (or an estimator) W n ( X ) is asymptotically normal if √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ → stands for ”converge in distribution”

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. .. . .. . Recap . where March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . n We denote W n d d Asymptotic Normality . . Definition: Asymptotic Normality . Asymptotic Normality Summary . Asymptotic Efficiency . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . . . .. . . .. . . .. . .. .. . .. . . 13 / 33 . . . . . . . . . . . . . . . . . . . . . . . A statistic (or an estimator) W n ( X ) is asymptotically normal if √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ → stands for ”converge in distribution” • τ ( θ ) : ”asymptotic mean” • ν ( θ ) : ”asymptotic variance”

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . n d where . Asymptotic Normality . Definition: Asymptotic Normality . Asymptotic Normality Summary . Asymptotic Efficiency . .. .. .. . . . . .. . . . .. . .. . . .. . . . .. . . . .. . . .. . . . .. 13 / 33 . . .. . . .. .. . . . . . . . . . . . . . . . . . . . . . . A statistic (or an estimator) W n ( X ) is asymptotically normal if √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ → stands for ”converge in distribution” • τ ( θ ) : ”asymptotic mean” • ν ( θ ) : ”asymptotic variance” ( ) τ ( θ ) , ν ( θ ) We denote W n ∼ AN

1 Y n X n . Theorem 5.5.17 - Slutsky’s Theorem . d n X n X i.i.d. Assume X i . . Central Limit Theorem . . Central Limit Theorem Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . . . aX March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang a X d Y n 2 X n . . d .. . . a , where a is a constant, P X , Y n d If X n . . . . . .. . .. .. . . .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . 14 / 33 . . . . . . . . . . . . . . . . . . . . . ∼ f ( x | θ ) with finite mean µ ( θ ) and variance σ 2 ( θ ) . µ ( θ ) , σ 2 ( θ ) ( ) ∼ AN

1 Y n X n . Theorem 5.5.17 - Slutsky’s Theorem . d n X i.i.d. Assume X i . . Central Limit Theorem . . Central Limit Theorem Summary . Asymptotic Efficiency Asymptotic Normality Recap .. . .. . . . . . aX March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang a X d Y n 2 X n . . d . . . a , where a is a constant, P X , Y n d If X n . . . . .. . . . . . . . . . . . . . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . .. . . .. . 14 / 33 .. . . .. . . .. . . .. . . .. . . . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ f ( x | θ ) with finite mean µ ( θ ) and variance σ 2 ( θ ) . µ ( θ ) , σ 2 ( θ ) ( ) ∼ AN ⇔ √ n N (0 , σ 2 ( θ )) ( X − µ ( θ ) ) →

1 Y n X n . Asymptotic Efficiency Assume X i . . Central Limit Theorem . Central Limit Theorem Summary . Asymptotic Normality X Recap . . . . . . . . . . . .. .. . .. . . .. i.i.d. d n . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang a X d Y n 2 X n . aX . d . . P d If X n . . Theorem 5.5.17 - Slutsky’s Theorem . . . .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . . 14 / 33 . . .. . . .. .. . . . .. . . .. . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . ∼ f ( x | θ ) with finite mean µ ( θ ) and variance σ 2 ( θ ) . µ ( θ ) , σ 2 ( θ ) ( ) ∼ AN ⇔ √ n N (0 , σ 2 ( θ )) ( X − µ ( θ ) ) → → X , Y n → a , where a is a constant,

. Recap . . Central Limit Theorem . Central Limit Theorem Summary . Asymptotic Efficiency Asymptotic Normality . . . . . . . . . . i.i.d. . .. .. . .. . . .. . Assume X i X .. d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang a X d Y n 2 X n . . n . P d If X n . . Theorem 5.5.17 - Slutsky’s Theorem . d . . . .. . . . .. . . .. . . . . . .. . . .. . . .. . . . .. .. . . .. . . .. . .. .. . . .. . . .. . . . .. . 14 / 33 . . . . . . . . . . . . . . . . . . . . . . ∼ f ( x | θ ) with finite mean µ ( θ ) and variance σ 2 ( θ ) . µ ( θ ) , σ 2 ( θ ) ( ) ∼ AN ⇔ √ n N (0 , σ 2 ( θ )) ( X − µ ( θ ) ) → → X , Y n → a , where a is a constant, 1 Y n · X n → aX

. . . Central Limit Theorem Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . . .. . . .. Central Limit Theorem . . P March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang d . . d . . d Assume X i If X n . . Theorem 5.5.17 - Slutsky’s Theorem . d n X i.i.d. . . .. .. . . . .. . . .. . . . . . .. . . .. . . .. . . .. 14 / 33 . . .. . .. . . . .. . .. . . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . ∼ f ( x | θ ) with finite mean µ ( θ ) and variance σ 2 ( θ ) . µ ( θ ) , σ 2 ( θ ) ( ) ∼ AN ⇔ √ n N (0 , σ 2 ( θ )) ( X − µ ( θ ) ) → → X , Y n → a , where a is a constant, 1 Y n · X n → aX 2 X n + Y n → X + a

p p . . Y i.i.d. Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. n . .. . . .. . . .. n i . c March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p n Var Y E Y I X i Y i i n n is consistent for p . Therefore, c I X i i n n . .. .. .. . .. . . .. . . .. . . . .. . .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . . . .. . . .. . 15 / 33 . . . . . . . . . . . . . . . . . . . . . Example - Estimator of Pr ( X ≤ c ) Define Y i = I ( X i ≤ c ) . Then Y i ∼ Bernoulli ( p ) where p = Pr ( X ≤ c ) .

p p . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . .. Summary . .. . .. .. . . . Y i.i.d. c March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p n Var Y E Y I X i . i n n is consistent for p . Therefore, n n n n .. . . . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . 15 / 33 . . . .. . . . . . . . . . . . . . . . . . . . . . Example - Estimator of Pr ( X ≤ c ) Define Y i = I ( X i ≤ c ) . Then Y i ∼ Bernoulli ( p ) where p = Pr ( X ≤ c ) . 1 Y i = 1 ∑ ∑ = I ( X i ≤ c ) i =1 i =1

. . .. . . .. .. . .. . . . . . . . . . . . .. . . .. . . .. . Recap . is consistent for p . Therefore, March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n n n n Asymptotic Normality n n n Y i.i.d. Summary . Asymptotic Efficiency . . .. . . .. . . . .. . .. .. . . .. . . .. . . . 15 / 33 .. .. . . . .. . . .. . . . . .. . .. . . . . . . . . . . . . . . . . . . . . . . . . Example - Estimator of Pr ( X ≤ c ) Define Y i = I ( X i ≤ c ) . Then Y i ∼ Bernoulli ( p ) where p = Pr ( X ≤ c ) . 1 Y i = 1 ∑ ∑ = I ( X i ≤ c ) i =1 i =1 1 ( E ( Y ) , Var ( Y ) ) ∑ I ( X i ≤ c ) ∼ AN i =1 ( p , p (1 − p ) ) = AN =

. . . . .. . . .. . .. . . . .. . .. .. . .. . . . . . . . . . . .. n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang d n X d n X X n Recap By Central Limit Theorem, n Example Summary . Asymptotic Efficiency Asymptotic Normality . . . .. . .. . . .. . . . . . .. . . .. . . . 16 / 33 .. . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n be iid samples with finite mean µ and variance σ 2 . Define 1 ∑ S 2 ( X i − X ) 2 n = n − 1 i =1

. . . . .. . . .. . .. . . .. .. . . .. . .. . . . . . . . . . . .. n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang d n X d n X X n Recap By Central Limit Theorem, n Example Summary . Asymptotic Efficiency Asymptotic Normality . . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . 16 / 33 . .. . . .. . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n be iid samples with finite mean µ and variance σ 2 . Define 1 ∑ S 2 ( X i − X ) 2 n = n − 1 i =1 µ, σ 2 ( ) ∼ AN

. . . .. . . .. . .. .. .. . .. . . .. . . . .. X n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang d n X d n By Central Limit Theorem, . . . . . . . . . . n Example Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . .. . .. . . . . . . .. . .. . . .. . . . .. . .. .. . . .. . . .. . . . 16 / 33 . .. .. . . . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n be iid samples with finite mean µ and variance σ 2 . Define 1 ∑ S 2 ( X i − X ) 2 n = n − 1 i =1 µ, σ 2 ( ) ∼ AN ⇔ √ n ( X − µ ) N (0 , σ 2 ) →

. .. .. . . .. . .. . . . .. . . .. . . . .. . By Central Limit Theorem, March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang d d n X n n . Example Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . . .. . . .. . . . .. . . .. . . . 16 / 33 .. .. . . .. . . .. . . . . . . .. . .. . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n be iid samples with finite mean µ and variance σ 2 . Define 1 ∑ S 2 ( X i − X ) 2 n = n − 1 i =1 µ, σ 2 ( ) ∼ AN ⇔ √ n ( X − µ ) N (0 , σ 2 ) → √ n ( X − µ ) ⇔ → N (0 , 1) σ

. . Recap . . . . . . . . . . . .. . . .. . Asymptotic Efficiency .. . . .. . . .. . Asymptotic Normality . .. . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . P S n n X Therefore, By Slutsky’s Theorem P Summary S n P S n P We showed previously S n S n S n Example (cont’d) . .. . . .. . . .. . . .. . .. . . . .. . . .. . . . . .. . .. . . .. . . .. . . .. . .. . . .. . . 17 / 33 . . . . . . . . . . . . . . . . . . . . . √ n ( X − µ ) √ n ( X − µ ) σ = σ

. .. . .. . . .. . . . Recap . .. . . .. . . . . . . . . . . . . Asymptotic Normality . Therefore, By Slutsky’s Theorem March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang . P S n n X P Asymptotic Efficiency P P n S n S n Example (cont’d) Summary . .. .. . . . .. .. . . .. . .. . . . .. . . .. . . .. . 17 / 33 . . . .. . . .. . . .. . . .. .. . .. . . . . . . . . . . . . . . . . . . . . . . . . √ n ( X − µ ) √ n ( X − µ ) σ = σ → σ 2 ⇒ S n We showed previously S 2 → σ ⇒ σ / S n → 1 .

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . P March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang P S n Therefore, By Slutsky’s Theorem P P Asymptotic Normality n S n S n Example (cont’d) Summary . Asymptotic Efficiency . .. .. .. . . . . .. . . . .. . .. . . .. . . . .. . . . .. . . .. . . .. . . 17 / 33 .. . . .. .. . . . . . . . . . . . . . . . . . . . . . . . √ n ( X − µ ) √ n ( X − µ ) σ = σ → σ 2 ⇒ S n We showed previously S 2 → σ ⇒ σ / S n → 1 . √ n ( X − µ ) → N (0 , 1) .

. .. .. . . .. . . . . . .. . .. .. . . . .. . Theorem 5.5.24 - Delta Method March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n . . . . Delta Method Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. .. . .. . . . . . .. . . .. . . . . .. . . . .. . . . .. . 18 / 33 .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . ( ) θ, ν ( θ ) Assume W n ∼ AN . If a function g satisfies g ′ ( θ ) ̸ = 0 , then ( g ( θ ) , [ g ′ ( θ )] 2 ν ( θ ) ) g ( W n ) ∼ AN

p p y , then X g X By central limit Theorem, n p X n d p p p n X n i.i.d. y Delta Method - Example Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . .. Define g y . . p March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p p p p p n p X g p g p X X By Delta Method, y y y g y g X . .. . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . .. . . .. . . .. . . 19 / 33 . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) .

p p y , then X g X . Define g y n p X n d i.i.d. Delta Method - Example Summary Asymptotic Efficiency . Asymptotic Normality Recap . . . . . . . . . . . .. . .. .. . . y X . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p p p p p p g X . p g p g p X X By Delta Method, y y y g y .. . . .. .. . . .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . . .. .. . . . .. . . .. . . .. . . .. . . .. . 19 / 33 . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) . By central limit Theorem, √ n ( X n − p ) → N (0 , 1) √ p (1 − p )

g X . Asymptotic Efficiency y Define g y n d i.i.d. Delta Method - Example Summary . Asymptotic Normality X Recap . . . . . . . . . . . .. .. . .. . . .. y , then X g X . . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p p p p p p g y p g p g p X X By Delta Method, y y y . . .. .. .. . . .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . . 19 / 33 . .. . .. . . .. . . .. . . .. . . . . .. . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) . By central limit Theorem, √ n ( X n − p ) → N (0 , 1) √ p (1 − p ) ( p , p (1 − p ) ) ⇔ X n ∼ AN

. . i.i.d. Delta Method - Example Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. n . . .. . . .. . . .. d g y . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p p p p p p y p g p g p X X g X By Delta Method, y y . .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . . 19 / 33 . .. . .. . . .. . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) . By central limit Theorem, √ n ( X n − p ) → N (0 , 1) √ p (1 − p ) ( p , p (1 − p ) ) ⇔ X n ∼ AN Define g ( y ) = y (1 − y ) , then X (1 − X ) = g ( X ) .

. . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . .. Delta Method - Example .. . .. . . .. . . Summary i.i.d. . p March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p p p p n d p p g p g p X X g X By Delta Method, n .. . . . . . .. . . .. . . .. . .. .. . . .. . . .. . . .. 19 / 33 .. . . . .. . . .. . . . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) . By central limit Theorem, √ n ( X n − p ) → N (0 , 1) √ p (1 − p ) ( p , p (1 − p ) ) ⇔ X n ∼ AN Define g ( y ) = y (1 − y ) , then X (1 − X ) = g ( X ) . g ′ ( y ) = ( y − y 2 ) ′ = 1 − 2 y

. .. . .. . . .. . .. . Recap . .. . . .. . . . . . . . . . . . . Asymptotic Normality . p March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n p p p p Asymptotic Efficiency n By Delta Method, n d i.i.d. Delta Method - Example Summary . .. . . . . . .. . .. . .. . .. . . . .. . . .. . . .. 19 / 33 .. . .. . . . .. . . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) . By central limit Theorem, √ n ( X n − p ) → N (0 , 1) √ p (1 − p ) ( p , p (1 − p ) ) ⇔ X n ∼ AN Define g ( y ) = y (1 − y ) , then X (1 − X ) = g ( X ) . g ′ ( y ) = ( y − y 2 ) ′ = 1 − 2 y ( g ( p ) , [ g ′ ( p )] 2 p (1 − p ) ) g ( X ) = X (1 − X ) ∼ AN

. .. .. . .. .. . . . . . .. . . .. . . . .. . d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n By Delta Method, n i.i.d. . Delta Method - Example Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . . .. . . .. . . . .. . . .. . . . 19 / 33 .. . .. . . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . ∼ Bernoulli ( p ) where p ̸ = 1 X 1 , · · · , X n 2 , we want to know the asymptotic distribution of X (1 − X ) . By central limit Theorem, √ n ( X n − p ) → N (0 , 1) √ p (1 − p ) ( p , p (1 − p ) ) ⇔ X n ∼ AN Define g ( y ) = y (1 − y ) , then X (1 − X ) = g ( X ) . g ′ ( y ) = ( y − y 2 ) ′ = 1 − 2 y ( g ( p ) , [ g ′ ( p )] 2 p (1 − p ) ) g ( X ) = X (1 − X ) ∼ AN ( p (1 − p ) , (1 − 2 p ) 2 p (1 − p ) ) = AN

. . Recap . . . . . . . . . . . .. . . .. . Asymptotic Efficiency .. . . .. . . .. . Asymptotic Normality . .. 1 Central Limit Theorem March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang 3 Delta Method (Theorem 5.5.24) . . 2 Slutsky Theorem . . Summary . Tools to show asymptotic normality n W n for all d n W n Asymptotic Normality .. . . . .. . . .. . . .. . .. . . . .. . . .. . . . . .. .. .. . . .. . . .. . . . . .. . . .. . . 20 / 33 . . . . . . . . . . . . . . . . . . . . . Given a statistic W n ( X ) , for example X , s 2 X , e − X

. .. . .. . . .. . . . Recap . .. .. . .. . . . . . . . . . . . . Asymptotic Normality . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang 3 Delta Method (Theorem 5.5.24) . . 2 Slutsky Theorem 1 Central Limit Theorem Asymptotic Efficiency . . Tools to show asymptotic normality n d Asymptotic Normality Summary . .. . . . . . .. .. . . .. . .. . . . .. . . .. . . .. 20 / 33 . . . . .. . . .. . .. .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Given a statistic W n ( X ) , for example X , s 2 X , e − X √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ ( τ ( θ ) , ν ( θ ) ) ⇐ ⇒ W n ∼ AN

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . . .. . . Recap Asymptotic Efficiency . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang 3 Delta Method (Theorem 5.5.24) . . 2 Slutsky Theorem . . 1 Central Limit Theorem . . Tools to show asymptotic normality n d Asymptotic Normality Summary .. .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. 20 / 33 .. . . . . . .. . . .. . . .. . .. . . . .. . . . . . . . . . . . . . . . . . . . . . Given a statistic W n ( X ) , for example X , s 2 X , e − X √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ ( τ ( θ ) , ν ( θ ) ) ⇐ ⇒ W n ∼ AN

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . . .. . . Recap Asymptotic Efficiency . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang 3 Delta Method (Theorem 5.5.24) . . 2 Slutsky Theorem . . 1 Central Limit Theorem . . Tools to show asymptotic normality n d Asymptotic Normality Summary .. .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. 20 / 33 .. . . . . . .. . . .. . . .. . .. . . . .. . . . . . . . . . . . . . . . . . . . . . Given a statistic W n ( X ) , for example X , s 2 X , e − X √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ ( τ ( θ ) , ν ( θ ) ) ⇐ ⇒ W n ∼ AN

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . . .. . . Recap Asymptotic Efficiency . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang 3 Delta Method (Theorem 5.5.24) . . 2 Slutsky Theorem . . 1 Central Limit Theorem . . Tools to show asymptotic normality n d Asymptotic Normality Summary .. .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. 20 / 33 .. . . . . . .. . . .. . . .. . .. . . . .. . . . . . . . . . . . . . . . . . . . . . Given a statistic W n ( X ) , for example X , s 2 X , e − X √ n ( W n − τ ( θ )) → N (0 , ν ( θ )) for all θ ( τ ( θ ) , ν ( θ ) ) ⇐ ⇒ W n ∼ AN

For example, in order to get the asymptotic distribution of n X i , X i , then E Y Var Y . Using Central Limit Theorem Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . . .. . . .. . . .. n i n Y March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n Var X E X n Y i . i n n X i i n n define Y i . .. .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . .. 21 / 33 . . .. . . .. . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . µ ( θ ) , σ 2 ( θ ) ( ) X ∼ AN where µ ( θ ) = E ( X ) , and σ 2 ( θ ) = Var ( X ) .

X i , then E Y Var Y . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . .. Summary . . .. . .. .. . . . n Using Central Limit Theorem Y i March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n Var X E X n Y i . n n X i i n n define Y i n .. . . . . .. . . .. . . .. . .. .. . . .. . . .. . . . .. . . . . .. . . .. . . .. . . .. . .. . . .. . 21 / 33 . . . . . . . . . . . . . . . . . . . . . µ ( θ ) , σ 2 ( θ ) ( ) X ∼ AN where µ ( θ ) = E ( X ) , and σ 2 ( θ ) = Var ( X ) . For example, in order to get the asymptotic distribution of 1 i =1 X 2 ∑ n i ,

E Y Var Y . .. . .. . . .. . . . Recap . .. . . .. . . . . . . . . . . . . Asymptotic Efficiency Asymptotic Normality n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n Var X E X n n . i n n n n Using Central Limit Theorem Summary . .. .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. 21 / 33 .. . . . .. . . .. . . . .. . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . µ ( θ ) , σ 2 ( θ ) ( ) X ∼ AN where µ ( θ ) = E ( X ) , and σ 2 ( θ ) = Var ( X ) . For example, in order to get the asymptotic distribution of 1 i =1 X 2 ∑ n i , define Y i = X 2 i , then 1 1 ∑ X 2 ∑ = Y i = Y i =1 i =1

. .. . .. . . .. .. . . Recap . .. . . .. . . . . . . . . . . . . Asymptotic Normality . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n Var X E X n n Asymptotic Efficiency i n n n n Using Central Limit Theorem Summary . .. . . . . . .. . .. . .. . .. . . . .. . . .. . . .. 21 / 33 . .. . . . .. .. . . .. . . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . µ ( θ ) , σ 2 ( θ ) ( ) X ∼ AN where µ ( θ ) = E ( X ) , and σ 2 ( θ ) = Var ( X ) . For example, in order to get the asymptotic distribution of 1 i =1 X 2 ∑ n i , define Y i = X 2 i , then 1 1 ∑ X 2 ∑ = Y i = Y i =1 i =1 ( E Y , Var ( Y ) ) ∼ AN

. . .. . .. .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . i March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n n n n Asymptotic Normality n n n Using Central Limit Theorem Summary . Asymptotic Efficiency . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . .. . .. . . .. . . .. . . . 21 / 33 . .. .. . . . . . . . . . . . . . . . . . . . . . . . . µ ( θ ) , σ 2 ( θ ) ( ) X ∼ AN where µ ( θ ) = E ( X ) , and σ 2 ( θ ) = Var ( X ) . For example, in order to get the asymptotic distribution of 1 i =1 X 2 ∑ n i , define Y i = X 2 i , then 1 1 ∑ X 2 ∑ = Y i = Y i =1 i =1 ( E Y , Var ( Y ) ) ∼ AN E X 2 , Var ( X 2 ) ( ) ∼ AN

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . .. .. . Recap . 1 Y n X n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang d . . d . Asymptotic Normality . P d When X n Using Slutsky Theorem Summary . Asymptotic Efficiency . . .. . . . .. . . .. . .. .. . . .. . . .. . . . 22 / 33 . .. . . . .. . . .. . . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . → X , Y n → a , then → aX 2 X n + Y n → X + a .

. .. .. . . .. . . . . . .. .. . .. . . .. .. Summary March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n Using Delta Method (Theorem 5.5.24) . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . . . . . . . . . .. . . .. . . .. . . .. .. . . .. . . .. . .. . .. 23 / 33 . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . ( ) θ, ν ( θ ) Assume W n ∼ AN . If a function g satisfies g ′ ( θ ) ̸ = 0 , then ( ) g ( θ ) , [ g ′ ( θ )] 2 ν ( θ ) g ( W n ) ∼ AN

. Asymptotic Normality i.i.d. . . Problem . Example Summary . Asymptotic Efficiency Recap Solution . . . . . . . . . . . .. . . .. . . .. . . . 2 By the invariance property of MLE, MLE of March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n X 3 By central limit theorem, we know that . . is X . . . . is X . 1 It can be easily shown that MLE of . . . . . . . . .. .. . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. 24 / 33 . .. . .. . . .. . . .. . . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ ̸ = 0 Find the asymptotic distribution of MLE of µ 2 .

. . Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . .. . . .. . . .. .. Example Problem .. is X . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n X 3 By central limit theorem, we know that . . 2 By the invariance property of MLE, MLE of . . . . . . Solution . i.i.d. . . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . 24 / 33 .. .. . . .. . . . . . . . . . . . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ ̸ = 0 Find the asymptotic distribution of MLE of µ 2 . 1 It can be easily shown that MLE of µ is X .

. . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . .. Example . . .. . . .. .. . Summary . . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n X 3 By central limit theorem, we know that . . . Problem . . . Solution . i.i.d. . . .. . . . .. .. . . .. . . .. . .. .. . . .. . . .. . . . . . .. . . .. . . .. . . .. . . . . . .. . . .. 24 / 33 . . . . . . . . . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ ̸ = 0 Find the asymptotic distribution of MLE of µ 2 . 1 It can be easily shown that MLE of µ is X . 2 . 2 By the invariance property of MLE, MLE of µ 2 is X

. .. Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . . Summary . .. . .. .. . . .. . Example . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n 3 By central limit theorem, we know that . . . . . . . Solution . i.i.d. . . Problem . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . . 24 / 33 . . .. .. . . . . . . . . . . . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ ̸ = 0 Find the asymptotic distribution of MLE of µ 2 . 1 It can be easily shown that MLE of µ is X . 2 . 2 By the invariance property of MLE, MLE of µ 2 is X µ, σ 2 ( ) X ∼ AN

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . X March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n g g y Asymptotic Normality g y . . . Solution (cont’d) Summary . Asymptotic Efficiency .. . .. . . . .. . . .. . .. . . . .. . . .. . . .. 25 / 33 . . . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . 4 Define g ( y ) = y 2 , and apply Delta Method.

. . . . .. . . .. . .. . . . .. . . .. .. .. . . . . . . . . . . .. X March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n g g . Recap . . Solution (cont’d) Summary . Asymptotic Efficiency Asymptotic Normality . . . .. . .. . . .. . . . . . .. . . .. . . . 25 / 33 .. .. .. . .. . . . . . . . .. . . .. .. . . . . . . . . . . . . . . . . . . . . . . . 4 Define g ( y ) = y 2 , and apply Delta Method. g ′ ( y ) = 2 y

. .. .. . . .. . . . . . .. . .. .. . . . .. . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n X . . . Solution (cont’d) Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . .. . .. . . . . . .. . . .. . . . 25 / 33 .. .. .. .. . . . . . . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . 4 Define g ( y ) = y 2 , and apply Delta Method. g ′ ( y ) = 2 y g ( µ ) , [ g ′ ( µ )] 2 σ 2 ( ) 2 ∼ AN

. .. .. . . .. . . . . .. .. . . .. . . . .. . . March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n X . . . Solution (cont’d) Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. . . .. .. . . .. . . . . . .. . . .. . . . .. .. .. . . .. . . . . . .. . . . .. . . 25 / 33 .. . . . . . . . . . . . . . . . . . . . . . 4 Define g ( y ) = y 2 , and apply Delta Method. g ′ ( y ) = 2 y g ( µ ) , [ g ′ ( µ )] 2 σ 2 ( ) 2 ∼ AN µ 2 , (2 µ ) 2 σ 2 ( ) ∼ AN

If two estimators W n and V n satisfy The asymptotic relative efficiency (ARE) of V n with respect to W n is ARE V n W n If ARE V n W n , then V n is asymptotically more . Asymptotic Normality . their asymptotic variance. If both estimators are consistent and asymptotic normal, we can compare Asymptotic Relative Efficiency (ARE) Summary . Asymptotic Efficiency . Recap . . . . . . . . . . . .. . . .. . . .. Definition 10.1.16 : Asymptotic Relative Efficiency . . d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang efficient than W n . for every V W V n V n . W d n W n . . . . .. . . .. .. . .. . . .. . . .. . . . .. . .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . 26 / 33 . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . .

The asymptotic relative efficiency (ARE) of V n with respect to W n is ARE V n W n If ARE V n W n , then V n is asymptotically more Recap . . . . . . . . . . . .. . . .. . . . Asymptotic Efficiency .. . . .. .. . .. Asymptotic Normality Asymptotic Relative Efficiency (ARE) . d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang efficient than W n . for every V W d Summary . . Definition 10.1.16 : Asymptotic Relative Efficiency . their asymptotic variance. If both estimators are consistent and asymptotic normal, we can compare . . . .. . . . . . .. . . .. . .. .. . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. . .. . . . .. 26 / 33 . . . . . . . . . . . . . . . . . . . . . √ n [ W n − τ ( θ )] If two estimators W n and V n satisfy → N (0 , σ 2 W ) √ n [ V n − τ ( θ )] → N (0 , σ 2 V )

If ARE V n W n , then V n is asymptotically more . . . . . . . . . . . . . .. . . .. . . .. Asymptotic Normality . .. .. . .. . Recap . Asymptotic Efficiency d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang efficient than W n . for every V W d .. . . Definition 10.1.16 : Asymptotic Relative Efficiency . their asymptotic variance. If both estimators are consistent and asymptotic normal, we can compare Asymptotic Relative Efficiency (ARE) Summary . . . . .. . . .. . . .. . .. . . . .. . . .. . . . . .. . .. . . .. . . .. . . .. 26 / 33 . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . √ n [ W n − τ ( θ )] If two estimators W n and V n satisfy → N (0 , σ 2 W ) √ n [ V n − τ ( θ )] → N (0 , σ 2 V ) The asymptotic relative efficiency (ARE) of V n with respect to W n is ARE ( V n , W n ) = σ 2 σ 2

. .. . .. . . .. . . . Recap . .. . .. .. . . . . . . . . . . . . Asymptotic Normality . d March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang efficient than W n . V W d . Asymptotic Efficiency . Definition 10.1.16 : Asymptotic Relative Efficiency . their asymptotic variance. If both estimators are consistent and asymptotic normal, we can compare Asymptotic Relative Efficiency (ARE) Summary . .. . . . . . .. . . .. . .. . . . .. . . .. . . .. .. 26 / 33 . . . . . .. . .. . .. .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . √ n [ W n − τ ( θ )] If two estimators W n and V n satisfy → N (0 , σ 2 W ) √ n [ V n − τ ( θ )] → N (0 , σ 2 V ) The asymptotic relative efficiency (ARE) of V n with respect to W n is ARE ( V n , W n ) = σ 2 σ 2 If ARE ( V n , W n ) ≥ 1 for every θ ∈ Ω , then V n is asymptotically more

. .. Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . . Summary . .. . . .. . . .. . Example . i March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Determine which one is more asymptotically efficient estimator. X e V n I X i n . n W n Our estimators are i.i.d. Let X i . . Problem .. . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . . .. . . .. 27 / 33 . . . . . . . . . . . . . . . . . . . . . ∼ Poisson ( λ ) . consider estimating Pr ( X = 0) = e − λ

. . Recap . . . . . . . . . . . .. . . .. . Asymptotic Efficiency .. . . .. . .. .. . Asymptotic Normality . .. n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Determine which one is more asymptotically efficient estimator. X e V n n W n Summary Our estimators are i.i.d. Let X i . . Problem . Example . . . . . . . .. . . .. . .. . . . .. . . .. . . . .. .. .. .. . . .. . . .. . . 27 / 33 . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ Poisson ( λ ) . consider estimating Pr ( X = 0) = e − λ 1 ∑ = I ( X i = 0) i =1

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . .. .. . . Recap Asymptotic Efficiency . W n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Determine which one is more asymptotically efficient estimator. V n n n Our estimators are . i.i.d. Let X i . . Problem . Example Summary .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. . . . . . .. . . .. . .. . . . .. . . .. 27 / 33 . .. . . . . . . . . . . . . . . . . . . . . . ∼ Poisson ( λ ) . consider estimating Pr ( X = 0) = e − λ 1 ∑ = I ( X i = 0) i =1 e − X =

. . . . . . . . . . . . . .. . . .. . .. Asymptotic Normality . . .. . .. .. . . Recap Asymptotic Efficiency . W n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang Determine which one is more asymptotically efficient estimator. V n n n Our estimators are . i.i.d. Let X i . . Problem . Example Summary .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. . . . . . .. . . .. . .. . . . .. . . .. 27 / 33 . .. . . . . . . . . . . . . . . . . . . . . . ∼ Poisson ( λ ) . consider estimating Pr ( X = 0) = e − λ 1 ∑ = I ( X i = 0) i =1 e − X =

g X and g y . . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . Solution - Asymptotic Distribution of V n .. . . .. . . .. . . Summary X . g March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e n g X E X Var X n e V n y . By Delta Method e y , then V n e Define g y n .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 28 / 33 . . . . . . . . . . . . . . . . . . . . . V n ( X ) = e − X , by CLT,

. .. Asymptotic Normality Recap . . . . . . . . . . . .. . . . . . .. . . .. . . .. Asymptotic Efficiency Summary . g March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e n g X Solution - Asymptotic Distribution of V n e V n y . By Delta Method e g X and g y y , then V n e Define g y .. . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 28 / 33 . . . . . . . . . . . . . . . . . . . . . V n ( X ) = e − X , by CLT, X ∼ AN ( E X , Var X / n ) ∼ AN ( λ, λ / n )

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . g March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e n g Asymptotic Normality X e V n Solution - Asymptotic Distribution of V n Summary . Asymptotic Efficiency .. . .. . . . .. . . .. . .. .. . . .. . . .. . . . 28 / 33 . . . .. . . .. . . .. . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . V n ( X ) = e − X , by CLT, X ∼ AN ( E X , Var X / n ) ∼ AN ( λ, λ / n ) Define g ( y ) = e − y , then V n = g ( X ) and g ′ ( y ) = − e − y . By Delta Method

. . . . .. . . .. . . .. . . .. . . .. .. . . n March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e Solution - Asymptotic Distribution of V n .. Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . . .. .. .. . . . .. . . . . . .. . . .. . . .. 28 / 33 . . .. . .. . .. .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . . V n ( X ) = e − X , by CLT, X ∼ AN ( E X , Var X / n ) ∼ AN ( λ, λ / n ) Define g ( y ) = e − y , then V n = g ( X ) and g ′ ( y ) = − e − y . By Delta Method ( g ( λ ) , [ g ′ ( λ )] 2 λ ) V n = e − X ∼ AN

. .. .. . . .. . . . . .. .. . . .. . . .. .. Summary March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n n Solution - Asymptotic Distribution of V n . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . . . . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . .. . .. 28 / 33 . . . . .. . .. . . . . . . . . . . . . . . . . . . . . . V n ( X ) = e − X , by CLT, X ∼ AN ( E X , Var X / n ) ∼ AN ( λ, λ / n ) Define g ( y ) = e − y , then V n = g ( X ) and g ′ ( y ) = − e − y . By Delta Method ( g ( λ ) , [ g ′ ( λ )] 2 λ ) V n = e − X ∼ AN ( ) e − λ , e − 2 λ λ ∼ AN

. . . . . . . . . . . n n W n Solution - Asymptotic Distribution of W n Summary . Asymptotic Efficiency Asymptotic Normality Recap . I X i .. . . .. . . .. . . i Z n . E Z March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e e n Var Z Z n Z i W n By CLT, e e Var Z e Pr X E Z Bernoulli E Z .. .. . .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . .. .. . . . .. . . .. . . .. . .. . . . .. . . .. . 29 / 33 . . . . . . . . . . . . . . . . . . . . . Define Z i = I ( X i = 0)

. . W n Solution - Asymptotic Distribution of W n Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . .. n . . .. . . .. . .. .. n Z i . Var Z March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e e n E Z Bernoulli E Z Z n W n By CLT, e e Var Z e Pr X E Z . . .. .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . . 29 / 33 . . .. . . .. . . .. . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Define Z i = I ( X i = 0) 1 ∑ = I ( X i = 0) = Z n i =1

. .. Solution - Asymptotic Distribution of W n Summary . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . . n . .. . . .. . .. .. . W n n .. Var Z March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e e n E Z Z i Z n W n By CLT, e e Var Z e Pr X E Z . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . . .. 29 / 33 . .. . . .. . . . . . . . . . . . . . . . . . . . . . Define Z i = I ( X i = 0) 1 ∑ = I ( X i = 0) = Z n i =1 ∼ Bernoulli ( E ( Z ))

. . . Asymptotic Efficiency Asymptotic Normality Recap . . . . . . . . . . . .. . .. Solution - Asymptotic Distribution of W n . . .. . .. .. . . Summary W n . Var Z March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e e n E Z n Z n W n By CLT, e e Var Z Z i n .. . . . .. . .. . . .. . . .. . .. .. . . .. . . .. . . . 29 / 33 . .. . . .. . . .. . . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . Define Z i = I ( X i = 0) 1 ∑ = I ( X i = 0) = Z n i =1 ∼ Bernoulli ( E ( Z )) Pr ( X = 0) = e − λ E ( Z ) =

. . Recap . . . . . . . . . . . .. . . .. . Asymptotic Efficiency .. . .. .. . . .. . Asymptotic Normality . .. Var Z March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e e n E Z Summary Z n W n By CLT, Z i n n W n Solution - Asymptotic Distribution of W n . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 29 / 33 .. . . .. . . .. . . . .. . . . .. . .. . .. . . . . . . . . . . . . . . . . . . . . . Define Z i = I ( X i = 0) 1 ∑ = I ( X i = 0) = Z n i =1 ∼ Bernoulli ( E ( Z )) Pr ( X = 0) = e − λ E ( Z ) = e − λ (1 − e − λ ) Var ( Z ) =

. . .. . . .. . . .. . . . . . . . . . . . .. . . .. . . .. . Recap . By CLT, March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang n e e e Z i Asymptotic Normality n n W n Solution - Asymptotic Distribution of W n Summary . Asymptotic Efficiency . .. .. . . .. . . . .. . .. .. . . .. . . .. . . . 29 / 33 . .. . . .. . . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Define Z i = I ( X i = 0) 1 ∑ = I ( X i = 0) = Z n i =1 ∼ Bernoulli ( E ( Z )) Pr ( X = 0) = e − λ E ( Z ) = e − λ (1 − e − λ ) Var ( Z ) = W n = Z n ∼ AN ( E ( Z ) , Var ( Z )/ n )