ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Second-Order Effect of Estimated Weights Recall the general mean-variance specification E( Y | x ) = f ( x , β ) , var( Y | x ) = σ 2 g ( β , θ , x ) 2 . The large sample approximate distribution of the GLS estimator is √ n � � ˆ L 0 , σ 2 � � β GLS − β 0 − → N 0 Σ WLS where � − 1 � n →∞ n − 1 X T WX Σ WLS = lim , if the working variance function is the true variance function. 1 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response The folklore theorem says the asymptotic distribution is the same no matter if the weights are known ( β and θ are held equal to their true values), or if the weights are estimated ( β and θ are estimated by √ n -consistent estimators ˆ β and ˆ θ ). regardless of the number of iterations of the GLS algorithm (the same for all values of C ). Such results hold only to this order of approximation ( first order ). The standard errors obtained this way tend to understate the variability associated with ˆ β GLS , particularly for smaller n . 2 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response This level of approximation suggests that ≈ 1 � � ˆ n σ 2 var β GLS 0 Σ WLS Or say, � 1 � = 1 � � ˆ n σ 2 var β GLS 0 Σ WLS + o p . n We can sometimes go further to a more refined approximation: � 1 = 1 0 Σ WLS + 1 � � � ˆ n σ 2 var n 2 V + o , β GLS n 2 where the second-order term n − 2 V may capture the effect of using estimated weights and the choice of number of iterations. 3 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Generally, arguments to establish such second order results are very tedious. So we will pursue some simple, special cases. Throughout, we assume the variance function g ( · ) is correctly specified, as our focus is on understanding the performance of the first order and second order results when the model is correct. The second order results can provide some useful theoretical insight. However, they do not translate into improvements that may be used in practice, as the necessary calculations are much too difficult to be implemented easily, even for those simple special cases. Later, we will consider the bootstrap as an alternative way of effecting the same sort of improvement “automatically” under certain circumstances. 4 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Case I When g ( · ) does not depend on β [Rothenberg, 1984] Assumptions g ( · ) does not depend on β ; The variance parameter θ is a scalar, and is estimated by · ˆ ∼ N ( θ 0 , τ 2 / n ); θ Linear model E ( Y j | x j ) = x T j β , and Gaussian distribution of Y j given x j . 5 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Then � 1 � = 1 0 Σ WLS + 1 � � ˆ n σ 2 var β GLS n 2 V + o , n 2 and V is an increasing function of τ 2 . Heuristic argument shows that V = τ 2 Σ D , but Σ D is hard to calculate. The number of iterations C appears not to matter here. 6 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response The precision of estimation of β by ˆ β GLS is dictated by the precision of estimation of θ . In particular, the more precise ˆ θ is, the more precise ˆ β GLS is, to second order. The role of estimation of θ only shows up in the second order term n − 2 V ; for large n , this term is dominated by the leading term, and its effect is negligible. For small n , however, the effect may be more pronounced. The form of V may be very difficult to derive, so it may not be practical to use this additional correction term to calculate improved standard errors of ˆ β GLS . 7 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Case II When g ( · ) depends on β [Carroll, Wu, and Ruppert, 1988] Weaker assumptions g ( · ) may depend on β , i.e., a general g ( β , θ , x ); The variance parameter θ does not have to be a scalar, and is estimated by a “reasonable” estimator ˆ θ ; Mean model need not be linear, and errors need not be Gaussian. ( C ) Then the C -step estimator ˆ GLS satisfies β � 1 = 1 0 Σ WLS + 1 � ( C ) � � ˆ n σ 2 var n 2 V ( C ) + o , β GLS n 2 where the second-order term V ( C ) may depend on C . 8 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response In general, V (3) = V (4) = · · · = V ( ∞ ). In addition, V (2) = V (3) = · · · = V ( ∞ ) if either g ( · ) does not depend on β and the errors are symmetrically distributed; (0) ˆ GLS is ˆ β β OLS . In addition, V (1) = V (2) = · · · = V ( ∞ ) if both the above conditions hold. 9 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response When V (1) � = V (3), there is no general ordering; both V (1) > V (3) and V (1) < V (3) are possible. � ǫ 2 � If g ( · ) does not depend on β and var = 2 + κ for all j , then for j all C and for some matrix V ∗ , V ( C ) = (2 − κ ) V ∗ . 10 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response There is no “optimal” number of iterations of the GLS algorithm, in a second order sense: It could well be that iterating past C = 1 could be detrimental! The usual practice of taking C = ∞ could be suboptimal in some situations. After C = 3 in general, or after C = 2 or 1, additional iteration has, to second order, no effect. The form of V ( C ) is always complicated, and again bootstrap variance estimation is preferred. Main point From a second order perspective, how one estimates θ does matter ! This has motivated research into determining the “best” way to estimate θ under different circumstances. 11 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Recall This approach is based on the first-order approximation that √ n � � ˆ β GLS − β 0 converges in law to some limiting Gaussian distribution. The central limit theorem does not need to be written in this way. All we really need in practice is that ˆ · ∼ N ( β 0 , Σ n ) β GLS where Σ n is some matrix we can calculate. 12 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Outline of bootstrap Depends on the assumption that ǫ j = Y j − f ( x j , β ) g ( β , θ , x j ) , 1 ≤ j ≤ n , are i.i.d. (note: not true for Poisson regression, for example). At step C + 1, get residuals ( C +1) � � x j , ˆ Y j − f β s j = � . � ( C +1) , ˆ ( C ) , x j ˆ g β θ 13 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response For b = 1 , 2 , . . . , B : � � s b 1 , s b 2 , . . . , s b sample with replacement from the finite n population { s 1 , s 2 , . . . , s n } Form the bootstrap responses � ( C +1) � � ( C +1) , ˆ ( C ) , x j � x j , ˆ ˆ Y b s b j = f + g j . β β θ b from these responses, using the Get the bootstrap estimate ˆ β C + 1 iterate. 14 / 15 Second-Order Effect of Estimated Weights
ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Summarize these B bootstrap estimates as B β boot = 1 b , ˆ � ˆ β B b =1 B 1 b − ˆ b − ˆ � T � � � ˆ � ˆ ˆ Σ boot = . β β boot β β boot B − 1 b =1 A heuristic argument shows that, when the second-order approximation works, ˆ Σ boot estimates the sum of both the first-order ( C +1) � � ˆ and second-order terms in var . β That is, it does “automatic” adjustment for using estimated weights. 15 / 15 Second-Order Effect of Estimated Weights
Recommend
More recommend
