Asymptotic Theory Part I Review of Asymptotic Theory James J. - PowerPoint PPT Presentation

Asymptotic Theory Part I Review of Asymptotic Theory James J. Heckman University of Chicago This draft, April 4, 2006 1

� 1 Inequalities for Random Variables In this section, we review a set of useful inequalities for random variables. • Markov’s Inequality If � ( � � 0) = 1 and � ( � ) � � , then for any � � 0 we have: � ( � � � ) � � ( � ) 2

• Chebyschev’s Inequality If �� 2 � � , then Pr [ | � | � � ] � �� 2 �� 2 • Schwarz Inequality � | �� | 2 � � | � | 2 � | � | 2 3

�� � 2 Convergence concepts for random variables In this section, we examine di � erent convergence concepts and the relationship between them. 2.1 Definitions • Almost sure convergence � � � 0 h i � ( � ) � � ( � ) i � Pr lim � �� | � � ( � ) � � ( � ) | � � = 1 4

� � ��� ������ � � � � • Convergence in rth moment � � ( � ) � if � ( � � � ) and � ( � � ) � � and lim � �� � [ | � � � � | � ] = 0 • Convergence in Probability � � � 0 � � ( � ) � � ( � ) i � lim � �� Pr [ | � � ( � ) � � ( � ) | � � ] = 0 In other words, for any �� � � 0 � � � 0 ( �� � ) such that � � � � 0 , Pr [ | � � � � | � � ] � �� If � � � � , a constant, we write plim � � = � . 5

� � � � � � � � � � � � ��� ������ � � � � � � � � � � � � � �� � � � � � � � � � �� � � � • Convergence in Distribution � � (rv or constant) i � � � � � � pointwise, where � is the cdf of � . 2.2 Comparing Convergence Concepts � � = = Also, if � is a constant, then � � = 6

� � � 3 Laws of Large Numbers 3.1 Weak Law of Large Numbers Let � � = 1 P � � =1 � � and � = lim � �� � ( � � ) be finite. If � � ( � ) � � , then we say that { � � } obeys the “Weak Law of Large Numbers (WLLN)”. 7

Chebyschev’s weak LLN: By a Chebyschev theorem, su � cient conditions for { � � } (which may be neither identically distributed nor independent) to obey the WLLN are: • � [ � � ] � � � � [ � � ] � � � � 6 = �� ��� ( � � � � � ) = 0 � P � 1 • lim � �� � =1 � ( � � ) = 0 � � 2 8

� �� � 3.2 Strong Law of Large Numbers Let � � = 1 P � � =1 � � and � = lim � �� � ( � � ) be finite. If � � ( � ) � � , then { � � } obeys the SLLN. Su � cient conditions for { � � } to obey the SLLN: • { � � } independent and P � 1 � 2 � ( � � ) � � (Kolmogorov � =1 LLN 1) • { � � } i.i.d., � ( � � ) exists and is equal to � (Kolmogorov LLN 2) 9

� � � � � � 4 Results for function of random vari- ables 4.1 Slutsky Theorem � � and � ( · ) continuous = � � ( � � ) � � ( � ) If � is a constant, we write plim � ( � � ) = � ( plim � � ) . So if � ( � � ) is an estimate of some parameter and we don’t know the distribution of � ( � � ) , then using this result we can approxi- mate it with the distribution of � ( � ) . 10

� � � � � � � � � � � � � 4.2 Some convergence in distribution results • � ( · ) continuous and � � � � = � � ( � � ) � � ( � ) • � � � � � and � � � � = � � where � � to � � means lim � �� Pr [ | � � ( � ) � � � ( � ) | � � ] = 0 . 11

� � � � � � � � � � � �� � 4.3 Man and Wald Theorem � � and � � � � = • � � + � � � � + � • � � � � • The limit of the joint distribution of ( � � � � � ) exists and equals the joint distribution of ( �� � ) . 12

� � � � � � � � � � � � � 4.4 Delta Method If { � � } is a sequence of nonstochastic numbers tending to � , � � where � is a constant, and � 00 ( · ) exists, then: � � ( � � � � ) � � 0 ( � ) � . � � [ � ( � � ) � � ( � )] � � (0 � � 2 If � 00 ( · ) exists and � ( � � � � ) � ) , then ¡ ¢ 0 � � 0 ( � ) 2 � 2 � [ � ( � � ) � � ( � )] . 13

� � � � � � 5 Central Limit Theorems • Lindberg-Levy CLT X � = 1 { � � } i.i.d., ¯ � � , � ( � � ) = � , � ( � � ) = � 2 � =1 � � (0 � � 2 ) = � ( � � � ) 14

� � �� � � � � �� � � � � � � � � � �� � � � • Liapounov CLT � � , � ( � � ) = � 2 � , � ( � 3 { � � } i.i.d., � ( � � ) = � ) � � , X 1 £ | � � � � � | 3 ¤ ¯ = � � , � 3 � = � . � =1 Then if: " � # � 1 � 2 " � # 1 � 3 X X � 2 lim = 0 � 3 � � =1 � =1 � � (0 � � 2 ) = � ( � � � ) where X X 1 1 � 2 = lim � 2 ¯ � and ¯ � = lim � =1 � =1 15

� � � � Definitions: � � (1) and � � (1) 6 � � = � � (1) if for every � � 0 , lim � �� � {| � � | � � } = 1 For a vector, X � is � � (1) if || X � || = � � (1) More generally, � � = � � ( � � ) if = � � (1) For the vector case we have X � = � � ( � � ) if || X � || = � � ( � � ) 16

Note a little more formal definition � � = � � (1) if for every � � 0 , and every � � 0 , there exists an integer � ( �� � ) such that if � � � ( �� � ) then � {| � � | � � } � 1 � � � � = � � (1) if for every � � 0 , there exists a constant � ( � ) and integer � ( � ) such that if � � � ( � ) then � {| � � | � � ( � ) } � 1 � � 17

� � For a vector X : X � = � � (1) if || X � || = � � (1) More generally, X � X � = � � ( � � ) if = � � (1) For a vector X � = � � ( � � ) if || X � || = � � ( � � ) � � sequences less than any arbitrary value. � � bounds sequence above. 18