Quadratic versus Linear Estimating Equations GLS estimating - PowerPoint PPT Presentation

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Quadratic versus Linear Estimating Equations GLS estimating equations � σ 2 g 2   f β j 0 n � − 1 � � 1 /σ 0 � Y j − f j � j � = 0 . ( Y j − f j ) 2 − σ 2 g 2 2 σ 2 g 2 2 σ 4 g 4   0 0 j j ν θ j j j =1 Estimating equations for β are linear in Y j . Estimating equations for β only require specification of the first two moments. GLS is optimal among all linear estimating equations. 1 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Gaussian ML estimating equations 2 σ 2 g 2 � σ 2 g 2  f β j j ν β j  � − 1 � n � 1 /σ 0 � Y j − f j � j � = 0 . ( Y j − f j ) 2 − σ 2 g 2  2 σ 2 g 2  2 σ 4 g 4 0 0 j j j ν θ j j =1 Estimating equations for β are quadratic in Y j . Estimating equations for β require specification of the third and fourth moments as well. Specifically, if we let ǫ j = Y j − f ( x j , β ) σ g ( β , θ , x j ) , then we need to know ǫ 3 ǫ 2 � � = ζ ∗ � � = 2 + κ ∗ E and var j . j j j 2 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Questions j , how much is ˆ If we know the true values ζ ∗ j and κ ∗ β improved using the quadratic estimating equations versus using the linear estimating equations? If we use working values (for example ζ j = κ j = 0, corresponding to normality) that are not the true values (i.e., ζ ∗ j and κ ∗ j ), is there any improvement in using the quadratic estimating equations? If we use working variance functions that are not the true variance functions, is there any improvement in using the quadratic estimating equations? 3 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response General form of quadratic estimating equations:  2 σ 2 g 2  f β , j j ν β , j � σ 2 g 2 n � − 1 ζ j σ 3 g 3 � j j � 1 �   ζ j σ 3 g 3 (2 + κ j ) σ 4 g 4 2 σ 2 g 2   0 σ j j j j =1 ν θ , j � Y j − f j � × = 0 . ( Y j − f j ) 2 − σ 2 g 2 j 4 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Large sample distribution for all parameters jointly : ˆ   β − β 0 √ n L � 0 , A − 1 BA − 1 � σ − σ 0 ˆ − → N .   ˆ θ − θ 0 Here n 1 � 0 , j V − 1 D T A = lim 0 , j D 0 , j , n n →∞ j =1 n 1 � D T 0 , j V − 1 0 , j var ( s 0 , j | x j ) V − 1 B = lim 0 , j D 0 , j , n n →∞ j =1 5 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Also � σ 2 g 2 ζ j σ 3 g 3 � j j V j = , ζ j σ 3 g 3 (2 + κ j ) σ 4 g 4 j j T   2 σ 2 g 2 j ν β , j f β , j D j = 1  � �  2 σ 2 g 2  σ  0 j ν θ , j and V 0 , j and D 0 , j are evaluated at the true β 0 and θ 0 . 6 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Also � σ 2 g 2 ζ ∗ j σ 3 g 3 � j j var ( s 0 , j | x j ) = , ζ ∗ j σ 3 g 3 � 2 + κ ∗ � σ 4 g 4 j j j the true variance matrix. Note that if the working values for ζ j and κ j are the same as the true values, var ( s 0 , j | x j ) = V 0 , j , so B = A , and the large sample distribution simplifies to ˆ   β − β 0 √ n L � 0 , A − 1 � ˆ σ − σ 0 − → N .   ˆ θ − θ 0 7 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response To deduce the limiting distribution of n 1 / 2 (ˆ β − β 0 ), it would be necessary to carry out the indicated matrix inversion and multiplications and extract the upper left p × p submatrix of the result. It is possible to show that, just as GLS is optimal among linear estimating equations, ˆ σ and ˆ β (as well as ˆ θ ) are optimal among quadratic estimating equations, provided the working values for ζ j and κ j are the true values. Next we consider a special case to gain better ideas of comparison between linear and quadratic estimating equations. 8 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response If we take ǫ 3 ǫ 2 � � � � E = 0 and var = 2 + κ for all j , j j while in truth ǫ 3 = ζ ∗ ǫ 2 = 2 + κ ∗ � � � � E and var for all j j j then we have n 1 / 2 � � L ˆ � 0 , σ 2 0 Γ − 1 ∆Γ − 1 � β − β 0 − → N . 9 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Here Γ = lim n →∞ Γ n X T WX + 4 σ 2 � � 2 + κ R T PR n →∞ n − 1 0 = lim WLS + 4 σ 2 = Σ − 1 0 2 + κ Σ β and ∆ = lim n →∞ ∆ n X T WX + 4 σ 2 � 0 (2 + κ ∗ ) �� R T PR + 2 σ 0 ξ ∗ � X T W 1 / 2 PR + R T PW 1 / 2 X = lim (2 + κ 2 ) 2 + κ n →∞ WLS + 4 σ 2 0 (2 + κ ∗ ) Σ β + 2 σ 0 ζ ∗ � � = Σ − 1 T β + T T β (2 + κ ) 2 2 + κ 10 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response and     ν T τ T β 01 θ 01 . . . . R = Q =  .   .      ν T τ T β 0 n θ 0 n n × p n × ( q +1) and P = I − Q ( Q T Q ) − 1 Q T . 11 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response Recall that, if the first two moments are correctly specified, then √ n � � L ˆ � 0 , σ 2 � β GLS − β 0 − → N 0 Σ WLS . First we note that the properties of ˆ β GLS do not depend on those of σ and ˆ θ , whereas the properties of ˆ ˆ β ML do. Next we compare ˆ β from the linear and quadratic equations in various scenarios under this special case. 12 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response When the data are really normal That is, we choose ζ = 0 and κ = 0 while ζ ∗ = 0 and κ ∗ = 0. Then Γ = Σ − 1 WLS + 2 σ 2 0 Σ β = Σ − 1 ∆ = Σ − 1 and ML . ML So √ n � � ˆ L 0 , σ 2 � � β ML − β 0 − → N 0 Σ ML . We can show that Σ GLS − Σ ML is nonnegative definite, as ( Σ − 1 WLS + 2 σ 2 0 Σ β ) − 1 = Σ ML � − 1 Σ WLS . Σ WLS + σ − 2 0 / 2 Σ − 1 � = Σ WLS − Σ WLS β 13 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response That is, ˆ β ML is more efficient compared to ˆ β GLS when the data truly are normal and we use the normal theory ML estimating equations for β . The source of improvement is from Σ β , which arises from taking advantage of the additional information β available in the variance function g ( · ). 14 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response When the data are only symmetrically distributed That is, we choose ζ = 0 and κ = 0 while ζ ∗ = 0 and κ ∗ > 0. Then Γ = Σ − 1 WLS + 2 σ 2 ∆ = Σ − 1 WLS + (2 + κ ) σ 2 0 Σ β 0 Σ β . and √ n � � L ˆ 0 , σ 2 Σ Q = Γ − 1 ∆Γ � � β ML − β 0 − → N 0 Σ Q , with We can show that Σ GLS − Σ Q is nonnegative definite, if κ ∗ ≤ 2. That is, the optimality of ˆ β ML no longer applies uniformly, when the data are only symmetric and we use the normal theory ML estimating equations for β . 15 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response When the data are symmetrically distributed ...and we correctly specify both ζ and κ . That is, we choose ζ = ζ ∗ = 0 and κ = κ ∗ . Then 4 σ 2 4 σ 2 Γ = Σ − 1 0 ∆ = Σ − 1 0 WLS + WLS + 2 + κ ∗ Σ β and 2 + κ ∗ Σ β and √ n � � ˆ L 0 , σ 2 � � β ML − β 0 − → N 0 Σ C , with � − 1 4 σ 2 � Σ − 1 0 Σ C = WLS + 2 + κ ∗ Σ β 16 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response We can show that Σ GLS − Σ C is nonnegative definite. That is, ˆ β ML is more efficient compared to ˆ β GLS when we know the data are symmetric and we are able to specify correctly a value for the excess kurtosis. 17 / 26 Quadratic versus Linear Estimating Equations

ST 762 Nonlinear Statistical Models for Univariate and Multivariate Response When the variance function g ( · ) does not depend on β In this case, Σ β = 0 and T β = 0. Then Γ = ∆ = Σ − 1 WLS and √ n � � ˆ L 0 , σ 2 � � β ML − β 0 − → N 0 Σ WLS . That is, there is nothing to be gained by using a quadratic estimating equation over a linear one, because there is no additional information on β to be gained from g ( · ). 18 / 26 Quadratic versus Linear Estimating Equations