biostatistics 602 statistical inference
play

Biostatistics 602 - Statistical Inference February 26th, 2013 - PowerPoint PPT Presentation

Jan 24, 2024 •262 likes •1.41k views

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang February 26th, 2013 Hyun Min Kang Rao-Blackwell Theorem Lecture 13

  1. log f X x i . Exponential Family Recap . . . . . .. . . .. . . . .. . . .. . . .. Rao-Blackwell Summary . i February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang t x i w log h x log c n Proof i n x log L n n n E . .. .. . .. . . . .. . . .. . . .. . . .. . . .. . . . 7 / 27 .. .. . .. . . .. . . . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . [ ] 1 ∑ E [ t ( X 1 )] = · · · = E [ t ( X n )] = τ ( θ ) t ( X i ) = i =1 So, 1 ∑ n i =1 t ( x i ) is an unbiased estimator of τ ( θ ) .

  2. . .. .. . . .. . . . . . .. . . .. . . . .. . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n n E . Proof Summary . Rao-Blackwell Exponential Family Recap . . . . .. .. . .. .. . . .. . . . . . .. . . .. . . . .. .. . . . .. . . .. . . 7 / 27 .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . [ ] 1 ∑ E [ t ( X 1 )] = · · · = E [ t ( X n )] = τ ( θ ) t ( X i ) = i =1 So, 1 ∑ n i =1 t ( x i ) is an unbiased estimator of τ ( θ ) . ∑ log L ( θ | x ) = log f X ( x i | θ ) i =1 ∑ = [ log c ( θ ) + log h ( x ) + w ( θ ) t ( x i )] i =1

  3. t x i is the best unbiased estimator of • And it attains the Cramer-Rao lower bound. • Because E n nw n Proof (cont’d) Summary . Rao-Blackwell Exponential Family Recap . i . . . . . .. . . .. . .. .. . n c t x i x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c , log L .. w c c i n n • w c . . . . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. . 8 / 27 .. . . .. . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1

  4. t x i is the best unbiased estimator of • And it attains the Cramer-Rao lower bound. • Because E . . Rao-Blackwell Exponential Family Recap . . . . . .. . Proof (cont’d) .. .. . . .. . . .. . Summary n n x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c , log L n w c c i n n • .. . . . .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 8 / 27 . . . .. . .. . . .. . . .. . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1

  5. • And it attains the Cramer-Rao lower bound. • Because E . . . . . . . .. . .. .. . Exponential Family .. . . .. . . .. . Recap . Rao-Blackwell , February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c x .. log L n • n n n Proof (cont’d) Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 8 / 27 .. . .. . . .. . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) 1 ∑ n i =1 t ( x i ) is the best unbiased estimator of − c ( θ ) w ′ ( θ )

  6. • Because E . . . . . . . .. . .. .. . Exponential Family .. . . .. . . .. . Recap . Rao-Blackwell , February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . w c c x .. log L n • n n n Proof (cont’d) Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 8 / 27 .. .. . . .. . . . . .. . . .. .. . . . .. . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) 1 ∑ n i =1 t ( x i ) is the best unbiased estimator of − c ( θ ) w ′ ( θ ) • And it attains the Cramer-Rao lower bound.

  7. . .. .. . . .. . . . . . .. . . .. . . . .. . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n • n n . Proof (cont’d) Summary . Rao-Blackwell Exponential Family Recap . . . . .. .. . . .. . . .. . . .. .. . . .. . . .. . . . 8 / 27 .. . .. . . .. . . . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . [ c ′ ( θ ) ∂ log L ( θ | x ) ] ∑ c ( θ ) + 0 + w ′ ( θ ) t ( x i ) = ∂θ i =1 [ }] c ′ ( θ ) 1 { ∑ nw ′ ( θ ) t ( x i ) − − = c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) 1 ∑ n i =1 t ( x i ) is the best unbiased estimator of − c ( θ ) w ′ ( θ ) • And it attains the Cramer-Rao lower bound. [ ∂ c ′ ( θ ) ] ∂θ log L ( θ | x ) = 0 , τ ( θ ) = − c ( θ ) w ′ ( θ ) . • Because E

  8. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. . . .. . Exponential Family . .. Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n t X i i n n E t X Summary In other words, unbiased estimator for its expected value. n . . Fact . Cramer-Rao Theorem on Exponential Family . .. . . . . . .. . . .. . .. . . . .. . . .. . . . .. .. . .. . . .. . . .. . . .. . 9 / 27 .. . . .. . . . . . . . . . . . . . . . . . . . . . . . f X ( x | θ ) = c ( θ ) h ( x ) exp [ w ( θ ) t ( x )] If X 1 , · · · , X n are iid samples from f X ( x | θ ) , 1 ∑ n i =1 t ( X i ) is the best

  9. . . . . . . . .. . . .. . .. Exponential Family . . .. . .. .. . . Recap Rao-Blackwell . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n t X i i n Var . unbiased estimator for its expected value. In other words, n . . Fact . Cramer-Rao Theorem on Exponential Family Summary .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. . . . . . .. . . .. . . .. . 9 / 27 .. .. .. . . . . . . . . . . . . . . . . . . . . . . . . . f X ( x | θ ) = c ( θ ) h ( x ) exp [ w ( θ ) t ( x )] If X 1 , · · · , X n are iid samples from f X ( x | θ ) , 1 ∑ n i =1 t ( X i ) is the best E [ t ( X )] = τ ( θ )

  10. . . .. . . .. . . .. .. . . . . .. . . .. . . .. . Recap . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n Var unbiased estimator for its expected value. In other words, . Exponential Family . Fact . Cramer-Rao Theorem on Exponential Family Summary . Rao-Blackwell . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . . . .. . . .. . . .. . . . .. . . .. .. 9 / 27 . . . . . . . . . . . . . . . . . . . . . . f X ( x | θ ) = c ( θ ) h ( x ) exp [ w ( θ ) t ( x )] If X 1 , · · · , X n are iid samples from f X ( x | θ ) , 1 ∑ n i =1 t ( X i ) is the best E [ t ( X )] = τ ( θ ) [ ] [ τ ′ ( θ )] 2 1 ∑ t ( X i ) = I n ( θ ) i =1

  11. . .. Rao-Blackwell Exponential Family Recap . . . . . .. . . . Summary . .. . .. .. . . .. . Proof . nw February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang t x i i n n , W x where a n W x a x log L w c c n . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . 10 / 27 .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . [ ] c ′ ( θ ) 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) = t ( X i ) + c ( θ ) w ′ ( θ ) i =1

  12. . . Recap . . . . . .. . . .. . Rao-Blackwell .. .. . .. . . .. . Exponential Family . .. , W x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang t x i i n n nw Summary where a W x a x log L n n Proof . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 10 / 27 . .. . .. . . . .. .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . [ ] c ′ ( θ ) 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) = t ( X i ) + c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) τ ( θ ) = − c ( θ ) w ′ ( θ )

  13. . . . .. . .. .. . .. .. . . .. . . .. . . . Proof February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n Summary . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . . .. . . 10 / 27 . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . [ ] c ′ ( θ ) 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) = t ( X i ) + c ( θ ) w ′ ( θ ) i =1 c ′ ( θ ) τ ( θ ) = − c ( θ ) w ′ ( θ ) ∂ ∂θ log L ( θ | x ) = a ( θ )[ W ( x ) − τ ( θ )] where a ( θ ) = nw ′ ( θ ) , W ( x ) = 1 ∑ n i =1 t ( x i )

  14. . Recap x log L E n n Summary . Rao-Blackwell Exponential Family . . . . E . .. . . .. . . .. . I n nw .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n t X i i n Var n w n t X i i n n Var nw t X i i n . .. . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. 11 / 27 . . .. . . .. . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1

  15. . . Summary . Rao-Blackwell Exponential Family Recap . . . . . .. . n .. . . .. . . .. . n E .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n t X i i n n Var w n t X i i n n Var nw n . .. . .. . . . .. . . .. . . . . . .. . . .. . . .. . . . .. 11 / 27 . . . .. . . . . .. . . .. . .. .. . .. . .. . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1

  16. . . Rao-Blackwell Exponential Family .. Recap . . . . . .. . .. Summary . . .. . . .. . . . n . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n t X i i n n Var w n n n Var n n E .. 11 / 27 . .. .. . . .. . . .. . . . . .. . . . .. .. . . . .. .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1 [ { }] 1 nw ′ ( θ ) ∑ = t ( X i ) − τ ( θ ) i =1

  17. . . Recap . . . . . .. . . .. . .. .. . . .. . . .. . Exponential Family Rao-Blackwell .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang I n w n n n . n Var n n E n n Summary . 11 / 27 . .. .. . . . .. . . . . . .. . .. . . . .. . .. . . .. . . .. . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1 [ { }] 1 nw ′ ( θ ) ∑ = t ( X i ) − τ ( θ ) i =1 [ ] 1 } 2 Var n 2 { ∑ w ′ ( θ ) = t ( X i ) i =1

  18. . . .. . . .. . . .. . . . . . .. . . .. . . .. . Recap . Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n n n Exponential Family n E n n Summary . .. . Rao-Blackwell .. . . .. . . .. . . .. .. . . .. . . .. . . . 11 / 27 . . . . . . .. . . .. .. . . .. .. . . .. . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [ ] 1 ∂ ∑ nw ′ ( θ ) ∂θ log L ( θ | x ) t ( X i ) − τ ( θ ) = i =1 [{ ∂  ) 2  } 2 ] ( 1  ( nw ′ ( θ )) 2 ∑ ∂θ log L ( θ | x ) = I n ( θ ) = E t ( X i ) − τ ( θ )  i =1 [ { }] 1 nw ′ ( θ ) ∑ = t ( X i ) − τ ( θ ) i =1 [ ] 1 } 2 Var n 2 { ∑ w ′ ( θ ) = t ( X i ) i =1 } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ )

  19. . . .. . . .. . . .. . . . . . .. . . .. . .. .. . Recap . I n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang nw I n I n I n nw Exponential Family I n w n E Summary . Rao-Blackwell . . .. . . . .. . . .. . .. .. . . .. . . .. . . . 12 / 27 . . .. . . .. . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ )

  20. . .. .. . . .. . . . . . .. .. . .. . . . .. . I n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang nw I n I n I n nw . E Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . .. . .. . . . . . .. . . .. . . . 12 / 27 .. . . .. . . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ ) } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ )

  21. . . . .. . . .. . .. .. . . .. . . .. . . . E February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang nw I n I n Summary . . Rao-Blackwell Exponential Family Recap . . . . . .. . .. .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . 12 / 27 . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ ) } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ ) I n ( θ ) · I n ( θ ) ] 2 [ nw ′ ( θ ) = [ τ ′ ( θ )] 2

  22. . . . . .. .. . .. . . .. . . .. . . .. . . Rao-Blackwell February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E Summary . Exponential Family .. Recap . . . . . .. . . .. . . . . . .. . . .. . . .. . . .. . . .. .. . . . . .. .. . . .. 12 / 27 . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . Obtaining I n ( θ ) [{ ∂ } 2 ] ∂θ log L ( θ | x ) = I n ( θ ) } 2 [ τ ′ ( θ )] 2 n 2 { w ′ ( θ ) = I n ( θ ) I n ( θ ) · I n ( θ ) ] 2 [ nw ′ ( θ ) = [ τ ′ ( θ )] 2 ) 2 ( I n ( θ ) = τ ′ ( θ ) | nw ′ ( θ ) τ ′ ( θ ) | I n ( θ ) =

  23. • It helps to confirm an estimator is the best unbiased estimator of • If an unbiased estimator of • There may be unbiased estimators of . .. . Rao-Blackwell Exponential Family Recap . . . . . . . Summary .. . . .. . . .. . Summary 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . is no longer a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller valid lower bound. I n . 2 When ”regularity conditions” are not satisfied, . . estimator. CR-bound, it does NOT mean that it is not the best unbiased has variance greater than the if it happens to attain the CR-bound. .. .. . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. .. . . .. . . .. . . . .. . .. . . .. . . 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) .

  24. • If an unbiased estimator of • There may be unbiased estimators of . . . Rao-Blackwell Exponential Family Recap . . . . . .. . Summary .. . . .. . . .. . Summary . . is no longer a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller valid lower bound. I n .. 2 When ”regularity conditions” are not satisfied, . . estimator. CR-bound, it does NOT mean that it is not the best unbiased has variance greater than the if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao .. . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . . .. . . .. . .. .. . . .. . . 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ )

  25. • There may be unbiased estimators of . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. Summary . .. . . .. . . . . Summary is no longer a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller valid lower bound. I n . 2 When ”regularity conditions” are not satisfied, . . estimator. CR-bound, it does NOT mean that it is not the best unbiased if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . .. . . . . .. . . .. . . .. . .. . . . .. . . .. 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ ) • If an unbiased estimator of τ ( θ ) has variance greater than the

  26. • There may be unbiased estimators of . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . valid lower bound. February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . I n than that have variance smaller is no longer a Summary . . estimator. CR-bound, it does NOT mean that it is not the best unbiased if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . . . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . . . .. . . .. . . .. . .. . . .. . . .. . 13 / 27 . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ ) • If an unbiased estimator of τ ( θ ) has variance greater than the 2 When ”regularity conditions” are not satisfied, [ τ ′ ( θ )] 2 I n ( θ )

  27. . . .. . . .. . . .. . . . . . .. . . .. .. . .. . Recap . estimator. February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang valid lower bound. is no longer a . . CR-bound, it does NOT mean that it is not the best unbiased Exponential Family if it happens to attain the CR-bound. 1 If ”regularity conditions” are satisfied, then we have a Cramer-Rao . . Summary Summary . Rao-Blackwell . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . . . .. . . .. . . .. . 13 / 27 . .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . . bound for unbiased estimators of τ ( θ ) . • It helps to confirm an estimator is the best unbiased estimator of τ ( θ ) • If an unbiased estimator of τ ( θ ) has variance greater than the 2 When ”regularity conditions” are not satisfied, [ τ ′ ( θ )] 2 I n ( θ ) • There may be unbiased estimators of τ ( θ ) that have variance smaller than [ τ ′ ( θ )] 2 I n ( θ ) .

  28. • How do we find the best unbiased estimator? • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator . . .. . . .. . . .. .. . . .. . . .. . Recap . 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . . . . . . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family .. . . .. .. . .. . . .. . . . . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . .

  29. • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator . . . .. . . .. . . .. .. . .. . . .. . . . . . . . 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . .. . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator?

  30. • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator . . . .. . . .. . . .. .. . .. . . .. . . . . . . . 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . .. . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator?

  31. • Find a ’better’ unbiased estimator . . . .. . . .. . . .. .. . .. . . .. . . . .. 1 Using Cramer-Rao bound February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . . . . . . . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap .. . . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . . .. . . .. 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator? • Use complete and sufficient statistic.

  32. . .. .. . . .. . . . . . .. . . .. . .. . .. . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang 2 Using Rao-Blackwell theorem . . 1 Using Cramer-Rao bound . . Methods for finding best unbiased estimator Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . . .. . . . . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . .. . . .. . 14 / 27 . . . . . . . . . . . . . . . . . . . . . • How do we find the best unbiased estimator? • Use complete and sufficient statistic. • Find a ’better’ unbiased estimator

  33. • Var X • E g X Y • If X and Y are independent, E g X Y . . . . .. . . .. .. . . . . .. . . .. .. Recap . . . . (Theorem 4.4.7) February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E g X . g x f x Y dx is a function of Y . x Var E X Y . E Var X Y X and Y are two random variables Important Facts Summary . Rao-Blackwell Exponential Family .. . .. . . . .. . . .. . .. . . . .. . . .. . . .. 15 / 27 . . . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3)

  34. • E g X Y • If X and Y are independent, E g X Y . . . .. . . .. . . . .. . . .. . . .. .. . X and Y are two random variables February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E g X . g x f x Y dx is a function of Y . x Important Facts . Summary . Rao-Blackwell Exponential Family Recap . . . . . .. .. . .. .. . . .. . . . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . . .. . 15 / 27 . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3) • Var ( X ) = E [ Var ( X | Y )] + Var [ E ( X | Y )] (Theorem 4.4.7)

  35. • If X and Y are independent, E g X Y . . . .. . . .. . .. .. . . .. . . .. . . . Summary February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang E g X . X and Y are two random variables Important Facts . . Rao-Blackwell Exponential Family Recap . . . . . .. . .. .. .. .. . . .. . . . . . .. . . .. . . . . .. .. . .. . . .. . . . . . .. . . .. . 15 / 27 . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3) • Var ( X ) = E [ Var ( X | Y )] + Var [ E ( X | Y )] (Theorem 4.4.7) ∫ • E [ g ( X ) | Y ] = x ∈X g ( x ) f ( x | Y ) dx is a function of Y .

  36. . .. .. . . .. . . . . . .. . . .. . . .. .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang X and Y are two random variables Important Facts Summary Rao-Blackwell . Exponential Family Recap . . . . . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . .. .. . . .. . . .. . . 15 / 27 . . . . . . . . . . . . . . . . . . . . . • E ( X ) = E [ E ( X | Y )] (Theorem 4.4.3) • Var ( X ) = E [ Var ( X | Y )] + Var [ E ( X | Y )] (Theorem 4.4.7) ∫ • E [ g ( X ) | Y ] = x ∈X g ( x ) f ( x | Y ) dx is a function of Y . • If X and Y are independent, E [ g ( X ) | Y ] = E [ g ( X )] .

  37. X n . Consider . . . Rao-Blackwell Exponential Family Recap . . . . . .. . Seeking for a better unbiased estimator .. . . .. . . .. . . Summary Suppose T X is any function of X . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W Var E W T Var X ) (unbiased for E W X E E W X T T E E W X T T .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 16 / 27 . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) .

  38. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W Var E W T Var Seeking for a better unbiased estimator ) (unbiased for E W X E E W X T T E E W X T T .. . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 16 / 27 . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider

  39. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. . . .. .. Exponential Family . .. Var E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W T Summary Var ) (unbiased for E W X E E W X T T E Seeking for a better unbiased estimator . . . . .. . . .. . . .. . .. . . . .. . . .. . . . . 16 / 27 . .. . . .. . . .. . . .. . .. .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T )

  40. . . . .. . . .. . .. .. . . .. . .. .. . . . .. Var E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Var W T . . . . Var Seeking for a better unbiased estimator Summary . Rao-Blackwell Exponential Family Recap . . . .. .. . . . .. . . . . . .. . . .. . . . 16 / 27 .. . . .. . . . .. . .. . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) )

  41. . . . . .. . . .. . . .. . .. .. . . .. .. . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W E Var W T Seeking for a better unbiased estimator .. Summary . Rao-Blackwell Exponential Family Recap . . . . . . . .. .. .. . . . . . . . . .. . . .. . . . .. .. .. . .. . . .. . . . 16 / 27 . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) ) Var ( φ ( T )) = Var [ E ( W | T )]

  42. . .. .. . . .. . . . . .. .. . . .. . . .. .. Summary February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Var W Seeking for a better unbiased estimator . . Rao-Blackwell Exponential Family Recap . . . . . .. . . . . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. . .. . .. 16 / 27 . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) ) Var ( φ ( T )) = Var [ E ( W | T )] Var ( W ) − E [ Var ( W | T )] =

  43. . .. .. . . .. . . .. . . .. . . .. . . .. .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang (smaller variance than W ) Seeking for a better unbiased estimator Summary Rao-Blackwell . Exponential Family Recap . . . . . .. . . . . . . . .. . . . . . .. . . .. . . .. .. . . .. . . .. .. . . .. 16 / 27 . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . Suppose W ( X ) is an unbiased estimator of τ ( θ ) . That is, E [ W ( X )] = τ ( θ ) . Suppose T ( X ) is any function of X = ( X 1 , · · · , X n ) . Consider φ ( T ) = E ( W ( X ) | T ) E [ E ( W ( X ) | T )] = E [ W ( X )] = τ ( θ ) E [ φ ( T )] = (unbiased for τ ( θ ) ) Var ( φ ( T )) = Var [ E ( W | T )] Var ( W ) − E [ Var ( W | T )] = ≤ Var ( W )

  44. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary .. E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T may not be an estimator. , which means that T may depend on . E W T T A better unbiased estimator? 2 . . T is equal or better than W X . T is an estimator, then 1 If . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . .. . . .. . . .. . 17 / 27 . . . . . . . . . . . . . . . . . . . . . Does this mean that φ ( T ) is a better estimator than W ( X ) ?

  45. . . . . . . . .. . . .. . .. Exponential Family . . .. . . .. . . Recap Rao-Blackwell . E W T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T may not be an estimator. , which means that T may depend on . E W T . T 2 . . . . A better unbiased estimator? Summary .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . 17 / 27 . . . . .. . . .. . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Does this mean that φ ( T ) is a better estimator than W ( X ) ? 1 If φ ( T ) is an estimator, then φ ( T ) is equal or better than W ( X ) .

  46. . . . . .. . . .. . . .. . . .. . . .. .. . . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . . . A better unbiased estimator? .. Summary . Rao-Blackwell Exponential Family Recap . . . . . . .. .. .. . . . .. . . . . . .. . . .. . . . .. .. .. . .. . . .. . . . 17 / 27 . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Does this mean that φ ( T ) is a better estimator than W ( X ) ? 1 If φ ( T ) is an estimator, then φ ( T ) is equal or better than W ( X ) . 2 φ ( T ) = E [ W | T ] = E [ W | T , θ ] . φ ( T ) may depend on θ , which means that φ ( T ) may not be an estimator.

  47. • E • Var • But . Recap . . . . . .. . . .. . Rao-Blackwell . .. . .. .. . . .. Exponential Family Example 1 . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X Var (unbiased) Summary T X E X X E X X E X X i.i.d. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . . 18 / 27 .. . . .. . . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1

  48. • E • Var • But . . . . . . .. . . .. . . Exponential Family .. .. . .. . . .. . Recap . Rao-Blackwell T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X Var (unbiased) .. T X E X X i.i.d. Example 1 Summary . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 18 / 27 .. . . . .. . .. . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 2 E ( X 1 | X 1 ) + 1 1 = 2 E ( X 2 | X 1 )

  49. • E • Var • But . .. . . .. . . .. . . . . . . .. . . .. . . . Rao-Blackwell Recap T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X Var (unbiased) Exponential Family T X i.i.d. Example 1 Summary . . .. .. . . . . .. .. . . .. . .. . . . .. . . .. . . .. 18 / 27 . . .. . . . .. . . .. . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 2 E ( X 1 | X 1 ) + 1 1 = 2 E ( X 2 | X 1 ) 2 X 1 + 1 1 = 2 E ( X 2 )

  50. • E • Var • But . .. . . .. . .. .. . . . . . . .. . . .. . . . Exponential Family Recap Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X T . (unbiased) T i.i.d. Example 1 Summary . Rao-Blackwell .. . . . . . .. . .. . .. . .. . . . .. . . .. . . .. 18 / 27 .. . . . .. . . .. . . .. . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 1 2 E ( X 1 | X 1 ) + 1 = 2 E ( X 2 | X 1 ) 2 X 1 + 1 1 = 2 E ( X 2 ) 1 2 X 1 + 1 = 2 θ

  51. • Var • But . . .. . .. .. . . .. .. . . .. . . .. . . . . . . . Var February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. X X T .. i.i.d. Example 1 Summary . Rao-Blackwell Exponential Family Recap . . . .. .. . . .. . . . . . . .. . . .. . . . 18 / 27 .. . . . .. . . .. . . .. . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 2 E ( X 1 | X 1 ) + 1 1 = 2 E ( X 2 | X 1 ) 1 2 X 1 + 1 = 2 E ( X 2 ) 2 X 1 + 1 1 = 2 θ • E [ φ ( T )] = 1 2 θ + 1 2 θ = θ (unbiased)

  52. • But . . .. .. . . .. . .. .. . . .. . . .. . . . Summary February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is NOT an estimator. i.i.d. Example 1 . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. . . . .. . . .. . . .. . . . .. . . .. 18 / 27 . . . . .. . . . .. . .. .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 1 2 E ( X 1 | X 1 ) + 1 = 2 E ( X 2 | X 1 ) 1 2 X 1 + 1 = 2 E ( X 2 ) 1 2 X 1 + 1 = 2 θ • E [ φ ( T )] = 1 2 θ + 1 2 θ = θ (unbiased) • Var [ φ ( T )] = 1 4 < Var ( 1 2 ( X 1 + X 2 )) = 1 2

  53. . .. .. . . .. . . . . . .. . . .. . . .. .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang i.i.d. Example 1 Summary Rao-Blackwell . Exponential Family Recap . . . . . .. . . .. . . . . .. . . .. . . .. . . .. . . .. . . . .. . . .. .. . . .. 18 / 27 .. . .. . . . . . . . . . . . . . . . . . . . . . . . . . ∼ N ( θ, 1) . W ( X ) = 1 Let X 1 , · · · , X n 2 ( X 1 + X 2 ) is an unbiased estimator of θ . Consider conditioning it on T ( X ) = X 1 . [ 1 ] φ ( T ) = E [ W | T ] = E 2( X 1 + X 2 ) | X 1 1 2 E ( X 1 | X 1 ) + 1 = 2 E ( X 2 | X 1 ) 1 2 X 1 + 1 = 2 E ( X 2 ) 2 X 1 + 1 1 = 2 θ • E [ φ ( T )] = 1 2 θ + 1 2 θ = θ (unbiased) • Var [ φ ( T )] = 1 4 < Var ( 1 2 ( X 1 + X 2 )) = 1 2 • But φ ( T ) is NOT an estimator.

  54. E X n X X n X • E • Var E X X E W T T i.i.d. Example 2 Summary . Rao-Blackwell . Exponential Family Recap . . . . . .. . . .. . . .. E X X n E X X Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T . (unbiased) T X n nX n E nX X n E X .. . .. . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . . .. . .. . . .. . . .. . . . .. . .. . . .. . . 19 / 27 . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X .

  55. E X X E X n X X n X • E • Var i.i.d. Example 2 Summary . Rao-Blackwell Exponential Family Recap . . . . . . .. . . .. . . .. .. . E X X E X n Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T . (unbiased) T X n nX n E nX X n .. . . . .. . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . . 19 / 27 .. . .. . . .. .. . . .. . . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X )

  56. X n X • E • Var Example 2 Summary . Rao-Blackwell Exponential Family Recap . . . . . .. n . . .. . . .. .. . .. i.i.d. . E X Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T . (unbiased) T X n nX n E nX X n . . .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 19 / 27 .. .. . .. . . .. . . . . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) =

  57. • E • Var . . Rao-Blackwell Exponential Family Recap . . . . . .. . Example 2 . .. . . .. .. . .. . Summary n i.i.d. Var X February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n n T .. (unbiased) T X n nX n E nX X n . . . .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . 19 / 27 . . . . .. . . .. . . .. . .. . .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) =

  58. • Var . . . .. . . .. . .. Recap . .. .. . . .. . . . . . . Rao-Blackwell Exponential Family n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • Var W n Var X . T n n n i.i.d. Example 2 Summary . .. . . . . . .. .. . . .. . .. . . . .. . . .. . . .. 19 / 27 . . .. .. . . . . . .. . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) = E ( nX | X ) = = nX n = X • E [ φ ( T )] = θ (unbiased)

  59. . . . .. . . .. . .. .. . . .. . . .. . . . .. n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang T is an estimator. • n n n . . . . i.i.d. Example 2 Summary . Rao-Blackwell Exponential Family Recap . .. . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . . .. . .. . . .. . 19 / 27 . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) = E ( nX | X ) = = nX n = X • E [ φ ( T )] = θ (unbiased) • Var [ φ ( T )] = Var ( X ) = 1 n < Var ( W ) = 1

  60. . .. .. . . .. . . .. . . .. . . .. . . . .. . n February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang n n n i.i.d. . Example 2 Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . . .. . .. . . . .. . . .. . . . . .. . . . .. . . .. . . 19 / 27 .. . . . . .. .. . . . . . . . . . . . . . . . . . . . . . . Let X 1 , · · · , X n ∼ N ( θ, 1) . W ( X ) = X 1 is an unbiased estimator of θ . Consider conditioning it on X . φ ( T ) = E [ W | T ] = E ( X 1 | X ) E ( X 1 | X ) + E ( X 2 | X ) + · · · + E ( X n | X ) = E ( X 1 + · · · + X n | X ) = E ( nX | X ) = = nX n = X • E [ φ ( T )] = θ (unbiased) • Var [ φ ( T )] = Var ( X ) = 1 n < Var ( W ) = 1 • φ ( T ) is an estimator.

  61. . . . Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . . .. . . .. . . .. Theorem 7.3.17 . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . for all T Define 2 Var . . T 1 E . . E W T . Then the followings hold. T .. . .. . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. . .. . . .. . . .. . . . .. . .. . . .. . . 20 / 27 . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ .

  62. . . . Rao-Blackwell Exponential Family Recap . . . . . .. . .. Rao-Blackwell Theorem . . .. . . .. . . Summary . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . for all T Theorem 7.3.17 2 Var . . T 1 E . . . .. .. . . . .. . . .. . . .. . .. .. . . .. . . .. . . . .. . . . . .. . . .. . . .. . .. . . . .. . . .. 20 / 27 . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold.

  63. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . for all T Rao-Blackwell Theorem 2 Var . . . . . Theorem 7.3.17 . . .. .. . . .. . . .. . . .. . .. .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . . . .. . . .. . 20 / 27 . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ )

  64. . .. . .. . . .. . . . Recap . .. . . .. . .. . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . T is a uniformly better unbiased estimator of That is, . . Rao-Blackwell . . . Theorem 7.3.17 . Rao-Blackwell Theorem Summary . .. . . . . . .. . . .. . .. . . . .. . . .. . . .. .. 20 / 27 . . . . .. . . .. . .. .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ ) 2 Var [ φ ( T ) | θ ] ≤ Var ( W | θ ) for all θ .

  65. . . . . .. . . .. . .. . . . .. . . .. . .. . . . . .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . . . . . Recap Theorem 7.3.17 . Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family . .. . .. . .. . . .. . . . .. . .. . . .. . . . . . . .. . . .. . . .. . 20 / 27 . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Let W ( X ) be any unbiased estimator of τ ( θ ) , and T be a sufficient statistic for θ . Define φ ( T ) = E [ W | T ] . Then the followings hold. 1 E [ φ ( T ) | θ ] = τ ( θ ) 2 Var [ φ ( T ) | θ ] ≤ Var ( W | θ ) for all θ . That is, φ ( T ) is a uniformly better unbiased estimator of τ ( θ ) .

  66. . . Var E W T T 2 Var . . . . Proof of Rao-Blackwell Theorem Summary Rao-Blackwell E Var W T Exponential Family Recap . . . . . .. . . .. . . Var W Var W .. T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on x . Therefore, (better than W ). Because T is a sufficient statistic, f x T does not depend on W x f x T d x x E W X T E W T T T is indeed an estimator. 3 Need to show . . .. . . . . . .. . . .. . . .. . .. . . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . .. . . .. . . .. . . 21 / 27 . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased)

  67. . . . Proof of Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . . .. . . .. . . .. . . . T February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on x . Therefore, (better than W ). Because T is a sufficient statistic, f x T does not depend on W x f x T d x x E W X T E W T T T is indeed an estimator. 3 Need to show . .. . .. .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . . 21 / 27 .. . .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W )

  68. . . . Proof of Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . . .. . . .. . . .. . . . x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on T (better than W ). . Therefore, Because T is a sufficient statistic, f x T does not depend on W x f x T d x x E W X T E W T T . . .. . .. .. .. . . .. . . .. . . . . . .. . . .. . . .. . . . 21 / 27 . .. . .. . . .. . . .. . . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator.

  69. . .. Rao-Blackwell Exponential Family Recap . . . . . .. . . . Summary . .. . . .. . . .. . Proof of Rao-Blackwell Theorem . x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on T . . Therefore, Because T is a sufficient statistic, f x T does not depend on . . (better than W ). . . . . .. .. . .. . . . .. . . .. . . .. . . .. . . .. . . . 21 / 27 .. . .. . .. . . . .. . .. . . .. . . . .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X

  70. . .. Exponential Family Recap . . . . . .. . . . . . .. . . .. . . .. Rao-Blackwell Summary . x February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an , and W x f x T d x does not depend on T Proof of Rao-Blackwell Theorem Therefore, . . (better than W ). . . . . . .. .. . .. . . . .. . . .. . . .. . . .. . . .. . . . 21 / 27 .. .. . .. . . . .. . . . . . .. . . . .. . .. . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X Because T is a sufficient statistic, f ( x | T ) does not depend on θ .

  71. . .. . .. . . .. . . . Recap . .. . . .. . . . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . estimator of T is indeed an and . Rao-Blackwell (better than W ). . . . . Proof of Rao-Blackwell Theorem Summary . .. .. . . . . .. .. . .. . .. . . . .. . . .. . . .. . 21 / 27 . . . . . .. . .. .. . . . . .. . . .. . .. . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X Because T is a sufficient statistic, f ( x | T ) does not depend on θ . Therefore, ∫ φ ( T ) = x ∈X W ( x ) f ( x | T ) d x does not depend on θ ,

  72. . . . .. . . .. . .. .. . .. .. . . .. . . . .. . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang . . (better than W ). . . . . . . . Proof of Rao-Blackwell Theorem Summary . Rao-Blackwell Exponential Family Recap . . . .. . .. . . . . . . .. . .. . . .. . . . .. . . .. . . . . . .. . .. .. . .. . 21 / 27 .. . . . . . . . . . . . . . . . . . . . . . . . 1 E [ φ ( T )] = E [ E ( W | T )] = E ( W ) = τ ( θ ) (unbiased) 2 Var [ φ ( T )] = Var [ E ( W | T )] = Var ( W ) − E [ Var ( W | T )] ≤ Var ( W ) 3 Need to show φ ( T ) is indeed an estimator. E ( W | T ) = E [ W ( X ) | T ] φ ( T ) = ∫ = W ( x ) f ( x | T ) d x x ∈X Because T is a sufficient statistic, f ( x | T ) does not depend on θ . Therefore, ∫ φ ( T ) = x ∈X W ( x ) f ( x | T ) d x does not depend on θ , and φ ( T ) is indeed an estimator of θ .

  73. . . . . . . . . . . Proof . . . Consider Theorem 7.3.19 . Uniqueness of UMVUE Summary . Rao-Blackwell Exponential Family Recap . . . . . .. Suppose W and W are two best unbiased estimators of estimator W .. Cov W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W Var W W Var W W Var W W W Var Var W W W E E W . W . . .. . . .. . . .. . . .. . . .. . .. .. . . .. . . .. . . .. . . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . .. . . .. 22 / 27 . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique.

  74. . Exponential Family . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . Rao-Blackwell Recap . . . . . . .. . . .. . . .. Proof . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W W E W Cov W Var W Var W W W Var Var W W W E . .. .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . .. . . . .. . . .. . . .. . . .. 22 / 27 . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) .

  75. . . . . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . Rao-Blackwell Exponential Family Recap . . .. . . .. . . .. . . . Proof . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W W . Cov W Var W Var W W W Var Var W E . .. .. . .. .. . .. . .. . . .. . . . . . .. . . .. . . .. . . . 22 / 27 .. . . . .. . . . .. . .. .. . . . .. . .. . . . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2

  76. . . . Uniqueness of UMVUE Summary . Rao-Blackwell Exponential Family Recap . . . . .. . . .. .. . . .. . . .. Theorem 7.3.19 . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W . W Cov W Var W Var W Var E . . Proof . . .. .. . . .. . . . .. . . . . . .. . . .. . . .. . . .. 22 / 27 . . . .. .. . . .. . . .. . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2

  77. . . . Rao-Blackwell Exponential Family Recap . . . . . .. .. .. Uniqueness of UMVUE . . .. . . .. . . Summary . . Var W February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var W Var W Var W Theorem 7.3.19 Var E . . Proof . . . .. . . . . .. . . .. . . .. .. . .. .. . . .. . . .. . . . 22 / 27 . . . .. . . .. . . .. .. . . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 4 Var ( W 1 ) + 1 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 )

  78. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. . . .. . Exponential Family . .. E February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var W Var W Var . Summary . Proof . . . Theorem 7.3.19 . Uniqueness of UMVUE . .. . . .. . . .. . . .. . .. . . . . .. . . .. . . . 22 / 27 .. .. . . .. . . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 4 Var ( W 1 ) + 1 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 ) 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 √ ≤ Var ( W 1 ) Var ( W 2 ) 2

  79. . .. . .. . . .. . . . Recap . .. . . .. . . . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var E . Proof Rao-Blackwell . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . .. .. . .. . . .. . . .. . . . . . .. .. . . .. . . .. 22 / 27 . . . .. .. . . . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 ) 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 √ ≤ Var ( W 1 ) Var ( W 2 ) 2 = Var ( W 1 ) = Var ( W 2 )

  80. . .. . .. . . .. . . . Recap . .. . . .. . . . . . . Exponential Family . . February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang Therefore W is better or equal to W and W . Var E . Proof Rao-Blackwell . . . Theorem 7.3.19 . Uniqueness of UMVUE Summary . .. .. . .. . . .. . . .. . . . . . .. .. . . .. . . .. 22 / 27 . . . .. .. . . . . .. . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . . If W is a best unbiased estimator of τ ( θ ) , then W is unique. Suppose W 1 and W 2 are two best unbiased estimators of τ ( θ ) . Consider estimator W 3 = 1 2 ( W 1 + W 2 ) . ( 1 2 W 1 + 1 ) = 1 2 τ ( θ ) + 1 E ( W 3 ) = 2 τ ( θ ) = τ ( θ ) 2 W 2 ( 1 2 W 1 + 1 ) Var ( W 3 ) = 2 W 2 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 = 2 Cov ( W 1 , W 2 ) 1 4 Var ( W 1 ) + 1 4 Var ( W 2 ) + 1 √ ≤ Var ( W 1 ) Var ( W 2 ) 2 = Var ( W 1 ) = Var ( W 2 )

  81. . Exponential Family W Cov W Therefore, the equality must hold, requiring contradictory to the assumption. If strict inequality holds, W is better than W and W , which is Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Recap Var W . . . . . .. . . .. . . .. Var W By Cauchy-Schwarz inequality, this is true if and only if W .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a b E W aW Var W Var W Var W a Var W b aW Cov W W Cov W b . . .. .. . .. . . .. . . .. . . . .. . .. . . .. . . .. . . . . . .. . .. . . .. . . .. . . 23 / 27 . . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) .

  82. . Recap W Cov W Therefore, the equality must hold, requiring contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Exponential Family . . . . Var W . .. . . .. . . .. . Var W By Cauchy-Schwarz inequality, this is true if and only if W .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a b E W aW Var W Var W Var W a Var W b aW Cov W W Cov W b .. . . .. . .. . . .. . . .. . . . .. . .. . . .. . . .. . . . . . . .. . . .. . . .. . . .. 23 / 27 . . .. . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is

  83. . . contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Exponential Family Recap . . . . .. By Cauchy-Schwarz inequality, this is true if and only if W . . .. . . .. .. . .. Therefore, the equality must hold, requiring aW . a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a b E W b Var W Var W Var W a Var W b aW Cov W W Cov W . . .. .. . .. . . .. . . .. . . . . . .. . . .. . . .. . . . 23 / 27 .. . . .. . . .. .. . . .. .. . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 1 2 Cov ( W 1 , W 2 ) = 1 √ Var ( W 1 ) Var ( W 2 ) 2

  84. . . Summary . Rao-Blackwell Exponential Family Recap . . . . . .. . contradictory to the assumption. .. . . .. . .. .. . Proof of Theorem 7.3.19 (cont’d) Therefore, the equality must hold, requiring .. b February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b a a Cov W E W Var W Var W Var W a Var W b aW Cov W W . . . . . . .. . . .. . . .. . . .. . . .. . . .. . . . .. 23 / 27 . .. . . .. . . .. . . .. . . .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 2 Cov ( W 1 , W 2 ) = 1 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b

  85. . . Recap . . . . . .. . . .. . Rao-Blackwell .. . . .. .. . .. . Exponential Family . .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b b Summary a E W Var W Var W Var W Therefore, the equality must hold, requiring contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) . . . . .. . . . .. . . .. . . .. . . .. . . .. . . . 23 / 27 . . . .. . .. . . .. . .. .. . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 2 Cov ( W 1 , W 2 ) = 1 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 )

  86. . . .. . . .. . . .. . . . . . .. .. . .. . . .. . Recap . a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b b Exponential Family a E W Therefore, the equality must hold, requiring contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell . . .. .. . .. . . .. . . . .. . .. . . .. . . . . . .. . .. . . .. . . .. . . 23 / 27 . . . .. . . .. . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 2 Cov ( W 1 , W 2 ) = 1 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 ) = Var ( W 1 ) Var ( W 2 ) = Var ( W 1 )

  87. . . . .. . . .. . .. .. .. . .. . . .. . . . .. a February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. W . Therefore, the best unbiased must hold, and W b Therefore, the equality must hold, requiring . . . . contradictory to the assumption. Proof of Theorem 7.3.19 (cont’d) Summary . Rao-Blackwell Exponential Family Recap . . . .. . .. . . . . . . .. . .. . . .. . . . .. . . .. . . .. . . .. . . 23 / 27 .. . .. . . .. . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 1 2 Cov ( W 1 , W 2 ) = 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 ) = Var ( W 1 ) Var ( W 2 ) = Var ( W 1 ) E ( W 2 ) = a τ ( θ ) + b

  88. . .. . .. . . .. . .. .. . . .. . . .. . . . Proof of Theorem 7.3.19 (cont’d) February 26th, 2013 Biostatistics 602 - Lecture 13 Hyun Min Kang estimator is unique. Therefore, the equality must hold, requiring contradictory to the assumption. Summary . . Rao-Blackwell Exponential Family Recap . . . . . .. . . .. . .. . . .. . . . . . .. . . .. . . . .. .. .. . .. . . . .. . . 23 / 27 . . .. . .. . . . . . . . . . . . . . . . . . . . . . . . Var ( W 3 ) ≤ Var ( W 1 ) = Var ( W 2 ) . If strict inequality holds, W 3 is better than W 1 and W 2 , which is 1 2 Cov ( W 1 , W 2 ) = 1 √ Var ( W 1 ) Var ( W 2 ) 2 By Cauchy-Schwarz inequality, this is true if and only if W 2 = aW 1 + b Cov ( W 1 , W 2 ) = Cov ( W 1 , aW 1 + b ) = a Var ( W 1 ) = Var ( W 1 ) Var ( W 2 ) = Var ( W 1 ) E ( W 2 ) = a τ ( θ ) + b = τ ( θ ) a = 1 , b = 0 must hold, and W 2 = W 1 . Therefore, the best unbiased

Recommend

STATS 8: Introduction to Biostatistics Statistical Inference for the Relationship Between Two

STATS 8: Introduction to Biostatistics Statistical Inference for the Relationship Between Two

STATS 8: Introduction to Biostatistics Statistical Inference for the Relationship Between Two Variables Babak Shahbaba Department of Statistics, UCI Objective We now discuss hypothesis testing regarding possible relationships between two

341 views • 12 slides
Biostatistics 602 - Statistical Inference March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun

Biostatistics 602 - Statistical Inference March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference March 19th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang March 19th, 2013 Hyun Min Kang Asymptotic Evaluation of Point Estimators

1.58k views • 118 slides
Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14

Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 28th, 2013 Biostatistics 602 - Lecture 14 Hyun Min Kang February 28th, 2013 Hyun Min Kang Obtaining Best Unbiased Estimator

1.46k views • 96 slides
Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

. . . . .. . . .. . .. . . . .. . . .. . .. . . . . . . . . .. Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun Min Kang Apil 23rd, 2013 Hyun Min Kang Final Exam Review &amp;

1.78k views • 155 slides
Biostatistics 602 - Statistical Inference April 18th, 2013 Biostatistics 602 - Lecture 25 Hyun

Biostatistics 602 - Statistical Inference April 18th, 2013 Biostatistics 602 - Lecture 25 Hyun

. . . . .. . . .. . .. . . . .. . . .. .. .. . . . . .. Biostatistics 602 - Statistical Inference April 18th, 2013 Biostatistics 602 - Lecture 25 Hyun Min Kang April 18th, 2013 Hyun Min Kang Bayesian Test &amp; Practice

1.47k views • 134 slides
Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun

. . . . .. . . .. . .. . . . .. . . .. . .. . . . . . . . . .. Biostatistics 602 - Statistical Inference Apil 23rd, 2013 Biostatistics 602 - Lecture 26 Hyun Min Kang Apil 23rd, 2013 Hyun Min Kang Final Exam Review &amp;

1.82k views • 156 slides
Biostatistics 602 - Statistical Inference March 14th, 2013 Biostatistics 602 - Lecture 16 Hyun

Biostatistics 602 - Statistical Inference March 14th, 2013 Biostatistics 602 - Lecture 16 Hyun

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference March 14th, 2013 Biostatistics 602 - Lecture 16 Hyun Min Kang March 14th, 2013 Hyun Min Kang Evaluation of Bayes Estimator Lecture 16

982 views • 96 slides
Biostatistics 602 - Statistical Inference April 16th, 2013 Biostatistics 602 - Lecture 24 Hyun

Biostatistics 602 - Statistical Inference April 16th, 2013 Biostatistics 602 - Lecture 24 Hyun

. . . . .. . . .. . .. . . . .. . . .. . .. . . . . . .. Biostatistics 602 - Statistical Inference April 16th, 2013 Biostatistics 602 - Lecture 24 Hyun Min Kang April 16th, 2013 Hyun Min Kang E-M Algorithm &amp; Practice

1.91k views • 147 slides
Biostatistics 602 - Statistical Inference February 19th, 2013 Biostatistics 602 - Lecture 12

Biostatistics 602 - Statistical Inference February 19th, 2013 Biostatistics 602 - Lecture 12

. .. .. . . .. . . . . . .. . . .. . . . .. .. Biostatistics 602 - Statistical Inference February 19th, 2013 Biostatistics 602 - Lecture 12 Hyun Min Kang February 19th, 2013 Hyun Min Kang Cramer-Rao Theorem Lecture 12 .

1.18k views • 90 slides
Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . .

Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . .

. Summary February 7th, 2013 Biostatistics 602 - Lecture 09 Hyun Min Kang February 7th, 2013 Hyun Min Kang Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . . MLE Method of Moments Likelihood

1.44k views • 109 slides
Ancillary Statistics Lecture 04 Biostatistics 602 - Statistical Inference . . . .

Ancillary Statistics Lecture 04 Biostatistics 602 - Statistical Inference . . . .

. Summary January 22th, 2013 Biostatistics 602 - Lecture 04 Hyun Min Kang January 22th, 2013 Hyun Min Kang Ancillary Statistics Lecture 04 Biostatistics 602 - Statistical Inference . . . . Location-scale Family Ancillary Statistics

1.58k views • 84 slides
Evaluation of Point Estimators Lecture 11 Biostatistics 602 - Statistical Inference . .

Evaluation of Point Estimators Lecture 11 Biostatistics 602 - Statistical Inference . .

. . February 14th, 2013 Biostatistics 602 - Lecture 11 Hyun Min Kang February 14th, 2013 Hyun Min Kang Evaluation of Point Estimators Lecture 11 Biostatistics 602 - Statistical Inference . . Summary Cramer-Rao . Evaluation MLE Recap

2.02k views • 136 slides
Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical Inference . Summary . .

Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical Inference . Summary . .

. . February 5th, 2013 Biostatistics 602 - Lecture 08 Hyun Min Kang February 5th, 2013 Hyun Min Kang Data Reduction - Summary Lecture 08 Biostatistics 602 - Statistical Inference . Summary . . Review Exponential Family . . . . . . .

1.48k views • 94 slides
Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - Statistical Inference . . . .

Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - Statistical Inference . . . .

. Summary April 2nd, 2013 Biostatistics 602 - Lecture 21 Hyun Min Kang April 2nd, 2013 Hyun Min Kang Wald Test Asymptotics of LRT Lecture 21 Biostatistics 602 - Statistical Inference . . . . Wald Test Asymptotics of LRT Karlin-Rabin

1.46k views • 95 slides
Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

. . March 12th, 2013 Biostatistics 602 - Lecture 15 Hyun Min Kang March 12th, 2013 Hyun Min Kang Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate Family . Bayes Estimator Bayesian Statistics

1.35k views • 88 slides
Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased

Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased

. Summary March 26th, 2013 Biostatistics 602 - Lecture 19 Hyun Min Kang March 26th, 2013 Hyun Min Kang Likelihood Ratio Test Lecture 19 Biostatistics 602 - Statistical Inference . . . . Unbiased Test LRT Recap . . . . . . . . 1

1.76k views • 144 slides
Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary .

Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary .

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang February 12th, 2013 Hyun Min Kang Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary . . MLE Recap . . . . . . . . . 1

1.08k views • 84 slides
Basus Theorem Lecture 06 Biostatistics 602 - Statistical Inference . Summary . . Basus

Basus Theorem Lecture 06 Biostatistics 602 - Statistical Inference . Summary . . Basus

. . January 29th, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang January 29th, 2013 Hyun Min Kang Basus Theorem Lecture 06 Biostatistics 602 - Statistical Inference . Summary . . Basus Theorem Complete Statistics . . . . . .

1.23k views • 102 slides
Factorization Theorem Lecture 02 Biostatistics 602 - Statistical Inference . Summary . .

Factorization Theorem Lecture 02 Biostatistics 602 - Statistical Inference . Summary . .

. . January 15th, 2013 Biostatistics 602 - Lecture 02 Hyun Min Kang January 15th, 2013 Hyun Min Kang Factorization Theorem Lecture 02 Biostatistics 602 - Statistical Inference . Summary . . Factorization Theorem Recap . . . . . .

1.04k views • 102 slides
Hypothesis Testing Lecture 18 Biostatistics 602 - Statistical Inference . Summary . .

Hypothesis Testing Lecture 18 Biostatistics 602 - Statistical Inference . Summary . .

. . March 21th, 2013 Biostatistics 602 - Lecture 18 Hyun Min Kang March 21th, 2013 Hyun Min Kang Hypothesis Testing Lecture 18 Biostatistics 602 - Statistical Inference . Summary . . Hypothesis Testing Recap . . . . . . . . . .

1.24k views • 121 slides
Complete Statistics Lecture 05 Biostatistics 602 - Statistical Inference . Summary . .

Complete Statistics Lecture 05 Biostatistics 602 - Statistical Inference . Summary . .

. . January 24th, 2013 Biostatistics 602 - Lecture 05 Hyun Min Kang January 24th, 2013 Hyun Min Kang Complete Statistics Lecture 05 Biostatistics 602 - Statistical Inference . Summary . . Complete Statistics Ancillary Statistics . .

1.41k views • 112 slides
Exponential Family Lecture 07 Biostatistics 602 - Statistical Inference . . Summary .

Exponential Family Lecture 07 Biostatistics 602 - Statistical Inference . . Summary .

. . January 31st, 2013 Biostatistics 602 - Lecture 07 Hyun Min Kang January 31st, 2013 Hyun Min Kang Exponential Family Lecture 07 Biostatistics 602 - Statistical Inference . . Summary . Exponential Family . . . . . . . . . . . . . .

1.28k views • 97 slides
Field of Biostatistics White, A.M., Lu, N., He, H., Silenzio, V &amp; Tu, X.M. University of

Field of Biostatistics White, A.M., Lu, N., He, H., Silenzio, V &amp; Tu, X.M. University of

Statistical Inference of Social Media Data on Neighborhood Violence Prevention and Its Impact on the Field of Biostatistics White, A.M., Lu, N., He, H., Silenzio, V &amp; Tu, X.M. University of Rochester Medical Center This picture shows how

315 views • 18 slides
Minimal Sufficient Statistics Lecture 03 Biostatistics 602 - Statistical Inference . Summary .

Minimal Sufficient Statistics Lecture 03 Biostatistics 602 - Statistical Inference . Summary .

. . January 17th, 2013 Biostatistics 602 - Lecture 03 Hyun Min Kang January 17th, 2013 Hyun Min Kang Minimal Sufficient Statistics Lecture 03 Biostatistics 602 - Statistical Inference . Summary . . Minimal Sufficient Statistics

1.09k views • 95 slides

More recommend