hypothesis testing large sample
play

Hypothesis Testing: Large Sample Asymptotic Theory Part IV James - PowerPoint PPT Presentation

Hypothesis Testing: Large Sample Asymptotic Theory Part IV James J. Heckman University of Chicago Econ 312 This draft, April 18, 2006 This lecture consists of three sections: Three commonly used tests Composite hypothesis


  1. Hypothesis Testing: Large Sample Asymptotic Theory — Part IV James J. Heckman University of Chicago Econ 312 This draft, April 18, 2006

  2. � This lecture consists of three sections: • Three commonly used tests • Composite hypothesis • Application to linear regression model X ln $ = ln � ( � � | � ) , � =1 where the � � are i.i.d. 1

  3. 1 Three commonly used tests In this section we examine 3 commonly used (asymptotically equivalent) tests: 1. Wald Test 2. Rao, Score or Lagrange Multiplier (LM) Test 3. LR Test We motivate the derivation of the test statistic in each case and show asymptotic equivalence of the three tests. 2

  4. � � � � 1.1 Wald Test Consider a simple null hypothesis: � 0 : � = � 0 , � � : � 6 = � 0 . Natural test uses the fact 1 that: � (ˆ � � � 0 ) � � (0 � � � 1 � 0 ) � (ˆ � (ˆ � � � 0 ) � � 2 ( � ) ( � = number of so that � � � 0 ) 0 � � 0 parameters), where: � 1 ¸ ¸ � 2 ln $ � � 2 ln � ( � � | � ) � � 0 = � � = � � , ���� 0 ���� 0 for � � i.i.d. 1 See Lecture III on Asymptotic theory. 3

  5. � � � This is the Wald Test and is similar to the � test used in OLS hypothesis testing. We use the uniform convergence of ˆ � to � 0 to obtain plim � ˆ � = � � 0 , so that the test statistic for the Wald test: d � (ˆ � (ˆ � � 2 ( � ) � � � 0 ) 0 � ˆ � = � � � 0 ) in large samples. Note that the Wald test is based on the unrestricted MLE model. 4

  6. � �� �� �� � � �� �� Recall at true parameter vector � = � 0 , ¯ ¸ Z � ln $ ( � | � ) ¯ � � ln $ ( � | � ) ¯ = $ ( � | � 0 ) �� = 0 ¯ � = � 0 because Z Z � $ ( � | � ) $ ( � | � 0 ) �� = 1 = �� = 0 , but �� = � ln $ � $ $ ¯ Z � ln $ ( � | � ) ¯ ¯ = $ ( � | � 0 ) �� = 0 � ¯ � = � 0 5

  7. �� �� � � Di � erentiating again, ¯ Z � 2 ln $ ( � | � ) ¯ ¯ $ ( � | � 0 ) �� ¯ ���� 0 � = � 0 ¯ ¯ Z � ln $ ( � | � ) ¯ ¯ � ln $ ( � | � ) ¯ ¯ + $ ( � | � 0 ) �� ¯ ¯ �� 0 � = � 0 � = � 0 = 0 � Therefore, " à !# ¯ ¯ ¯ ¯ 1 � ln $ � ln $ ¯ ¯ � � 0 = � ¯ ¯ �� 0 � = � 0 � = � 0 6

  8. ��� �� � �� �� �� Cramer-Rao Lower Bound (Scalar Case) Consider an estimator � ( � ) of � : Z � ( � ( � )) = � ( � ) $ ( � ; � ) ��� Under regularity, Z �� ( � ( � )) � ( � ) � ln $ ( � ; � ) = μ ¶ � ( � ) � � ln $ = 7

  9. �� �� �� �� Z � E ( � ( � )) � ( � ) � $ = �� d � Z � ( � ) � ln $ = $ d � μ ¶ � ( � ) � � ln $ = Cov , because μ � ln $ ¶ E = 0 . 8

  10. �� �� �� � � From the Cauchy-Schwartz inequality, μ �� ( � ( � )) ¶ 2 μ � ln $ ¶ � � �� ( � ( � )) � �� | {z } � � 0 For full rank information matrix � � 0 , μ �� ( � ( � )) ¶ 2 � �� ( � ( � )) � � � 0 9

  11. �� If unbiased, � ln � ( � ( � )) � ( � ( � )) = �� = 1 � �� ( � ( � )) � 1 � � 0 There is a vector version: � �� ( � ( � )) � � � 1 � 0 � Cannot do better than MLE in terms of e � ciency. 10

  12. � � �� � �� 1.2 Rao Test (also LM or Score Test) The second Test — Rao Test (LM) is based on the restricted model. Observes that in a large enough sample � 0 (true para- meter value) should be a root of the likelihood equation: ¯ ¯ X ¯ ¯ 1 � ln $ � 1 � ln � ( � � � � ) ¯ ¯ = 0 � ¯ ¯ � =1 � 0 � 0 i.e., it imposes the null onto the model for all sample sizes. In contrast, the Wald Test gets its statistics from estimates of the unrestricted model (i.e., a model where the null is not im- posed on the estimates). The Likelihood Ratio Test compares the likelihood restricted with the likelihood unrestricted. 11

  13. � �� �� �� �� � � �� ¯ μ � ln � ( � � | � ) ¶ ¯ � ln $ ¯ = 0 and E = 0 , ¯ ˆ � M L but μ � ln � ( � � | � ) ¶ 1 � ln $ P � E = 0 ¯ ¯ 1 � ln $ P ¯ = � 0 . ¯ � 0 12

  14. �� �� �� � � � � � Now By C.L.T.; Lindberg-Levy for � � i.i.d. z }| { ¯ ¯ X ¯ ¯ 1 � ln $ 1 � ln � ( � � � � ) ¯ ¯ = � � (0 � � � 0 ) ¯ ¯ � =1 � 0 � 0 implies that the hypothesis could be tested by testing if score ¯ ¯ � ln $ ¯ = 0 , at the restricted parameter values. 2 ¯ � 0 2 For the distributional results, refer to earlier lectures on Asymptotic theory. 13

  15. � �� �� � � � � �� Thus this test uses the statistic: μ 1 μ 1 ¶ 0 ¶ � ln $ � ln $ � � 1 = ˆ � 0 � 0 � � 2 ( � ) . d �� � � 2 ( � ) in large samples can be shown using plim � ˆ � = � � 0 . 14

  16. �� �� �� The Rao test is also called the Score Test or the Lagrange Multiplier test. This is because this test can be motivated from the solution to a constrained maximization of the log- likelihood subject to constraint � = � 0 . We get: ln $ � � 0 ( � � � 0 ) Lagrangian: � ln $ FOC: � � = 0 � = � ln $ � At null: = 0 Thus one can test on � (the Lagrange Multiplier) or on the μ � ln $ ¶ Score . 15

  17. � �� � �� � � Asymptotic Equivalence of Wald and LM tests To estab- lish the asymptotic relationships between the two tests, we use Taylor’s theorem to write: ¯ ¯ X � ln � X � ln � ¯ ¯ 1 1 ¯ ¯ = ¯ ˆ ¯ | {z } � 0 =0 by construction μX � 2 ln � ¶ + 1 � � (ˆ � � � 0 ) � ���� 0 where � � is an intermediate value with k � 0 k � k � � k � k ˆ � k � 16

  18. � � �� � � � � � �� � � � � � �� �� � � From above, we get the duality relationships between score and parameter vectors: ¯ μX � 2 ln � ¶ � 1 X � ln � ¯ (ˆ ¯ � � � 0 ) = � ¯ ���� 0 � 0 ¯ μ μX � 2 ln � ¶ ¶ � 1 X � ln � ¯ � 1 1 � (ˆ ¯ � � � 0 ) = ¯ ���� 0 � 0 Noting that plim � � = � 0 , substituting into the Wald statistic we get: μ 1 ¶ 0 ¯ μ 1 ¶¯ ¯ ¯ � ln $ � ln $ ¯ ¯ � 0 � � 0 � � 1 � � 1 plim � = = plim �� ¯ ¯ � 0 � 0 � 0 Thus, asymptotically the Wald and Rao tests are equivalent. 17

  19. � �� 1.3 Likelihood Ratio Test The third commonly used test is the Likelihood Ratio Test and it uses both the restricted and the unrestricted models. Taylor expanding the Likelihood function around the point ˆ � we get: ¯ ¯ � ) + � ln $ ln $ (ˆ ( � 0 � ˆ ¯ ln $ ( � 0 ) = � ) ¯ ˆ | {z } | {z } showed start with the restricted =0 by construction μ � 2 ln $ ¶ +1 2( � 0 � ˆ � � ( � 0 � ˆ � ) 0 � ) � ���� 0 where � � is an intermediate value with k � 0 k � k � � k � k ˆ � k . 18

  20. � � �� � � ³ ´ ³ ´ ˆ ˆ ln $ 0 ln $ � ln $ � ln $ � � 0 ln $ 0 + � ln $ 0 ( � � � � 0 ) ln $ � = | {z } =0 � 2 $ 0 ( � � � � 0 ) + � � � � 0 ���� 0 2 ( � � � � 0 ) 0 � 2 $ 0 2 (ln $ � � ln $ 0 ) = ���� 0 ( � � � � 0 ) = � 2 (ln $ � � ln $ 0 ) | {z } 19

  21. � � � � � � � μ � 2 2 $ 0 ¶ � ( � � � � 0 ) 0 = ( � � � � 0 ) ( � ) ���� 0 where μ � 2 2 $ 0 ¶ p ( � ) ���� 0 � � � 0 and ¡ ¢ d 0 � � � 1 ( � � � � 0 ) . � 0 20

  22. � � � � � � � � � � � � � � � � � � μ � 1 ¶ � 2 ln $ � 2[ln $ (ˆ � (ˆ � (ˆ � � � 0 ) 0 � ) � ln $ ( � 0 )] = � � � 0 ) ���� 0 " # $ (ˆ � ) � (ˆ � (ˆ � � � 0 ) 0 � � 0 � 2 ln � � � 0 ) . $ ( � 0 ) We may also write " # $ (ˆ � ) � � 0 � � 0 � � � 2 ( � ) ; 2 ln $ ( � 0 ) ³ ´ ¡ ¢ ˆ 0 � � � 1 , and � � . � � � 0 � 0 21

  23. � � � � � � � �� Based on the above derivation, we get the statistic for the likelihood ratio test: à ˆ ! �� = � 2 ln ˆ where ˆ � � is the restricted maximum likelihood function and ˆ � � is the unrestricted maximum likelihood function, and as shown above: � � 2 ( � ) � Note that ˆ ˆ � � (as an unrestricted maximized function � � � would be greater than or equal to the restricted maximized function), so that always LR � 0 . 22

  24. � � �� � ���� � ��� � � � � Asymptotic Equivalence of LR and Wald tests From the derivation above, it is clear that asymptotically, the LR test statistic converges to the Wald test statistic, i.e., 0 � � 0 � (ˆ � (ˆ plim �� = � � � 0 ) � � � 0 ) = plim � | {z } | {z } d d where � � � (0 � � � 0 ) . We saw earlier that the LM and Wald tests were asymptotically equivalent. Thus along with the above asymptotic equivalence of LR and Wald we get that asymptotically all three commonly used tests are equivalent, i.e., asymptotically : 23

Recommend


More recommend