HODGES-LEHMANN INVERSE LIKELIHOOD ESTIMATES (HLES) KJELL DOKSUM - PowerPoint PPT Presentation

HODGES-LEHMANN INVERSE LIKELIHOOD ESTIMATES (HLE’S) KJELL DOKSUM DEPT. OF STATISTICS UNIVERSITY OF WISCONSIN-MADISON COLUMBIA UNIVERSITY 4TH LEHMANN SYMPOSIUM RICE UNIV. MAY 11, 2011 KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 1/31

Figure: Javier Rojo KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 2/31

ACKNOWLEDGEMENTS AKI OZEKI UNIV. OF WISCONSIN KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 3/31

OUTLINE 1 SOME LIKELIHOODS 2 ASYMPTOTIC DISTRIBUTIONS OF HLE’s 3 MINIMAX RESULTS 4 ONE STEP ESTIMATORS KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 4/31

WHY HL-ESTIMATORS? 1 IN LINEAR REGRESSION MODELS WITH ERROR ∼ F , THE HL NORMAL SCORES ESTIMATE IS ASYMPTOTICALLY MORE EFFICIENT THAN THE LEAST SQUARES ESTIMATE, UNIFORMLY IN F . 2 SCHOLZ’S THEOREM. FOR EACH ONE SAMPLE ESTIMATE THAT CAN BE WRITTEN AS A LINEAR COMBINATION OF ORDER STATISTICS, THERE IS A HL-ESTIMATE THAT IS ASYMPTOTICALLY MORE EFFICIENT. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 5/31

SOME LIKELIHOODS X = DATA = ( Y , Z ), Y ∈ R , Z ∈ R p . θ = ( β ∈ R p , Λ ∈ F ) = PARAMETER � LIKELIHOOD = p ( x i ; θ ) i λ ( y i ; β | z i ) � COX LIK = � j ≥ i λ ( y i ; β | z j ) i p ( x i ; θ ), � p ( x i ; θ ) = 1. � EMPIRICAL LIK = i KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 6/31

SOME LIKELIHOODS PROFILE EMPIRICAL (PE) LIKELIHOOD � L PE ( β ) = sup { p ( x i ; θ ); Λ } (1) ˆ β PE = arg max L PE ( β ) THIS ESTIMATE ˆ β PE IS A FUNCTION OF THE RANK R 1 , · · · , R n OF Y 1 , · · · , Y n KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 7/31

SOME LIKELIHOODS HOEFFDING LIK ≡ L H ( r ( y ); β ) = P ( R = r ) = �� � p ( V ( r i ) ; θ | z i ) 1 , n ! E 0 p ( V ( r i ) ; θ 0 | z i ) i WHERE r i ≡ r ( y i ) ≡ RANK ( y i ). V (1) < · · · < V ( n ) ARE p ( v ; θ 0 | z ) ORDER STATISTICS. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 8/31

RANK LIKELIHOOD ESTIMATOR EXAMPLE : Y i = z T i β + ǫ i , ǫ i ∼ F , IID. FORWARD RANK MLE = ARG MAX L H ( r ( y ); β ) = KP MLE KP= KALBFLEISCH-PRENTICE (1973) ˆ β KP SOLVES ∇ β L H ( r ( y ); β ) = 0 KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 9/31

RANK LIKELIHOOD ESTIMATOR BECAUSE RANK (Λ( y i )) = RANK ( y i ). FOR Λ ր , ˆ β KP APPLIES TO SEMIPARA. TRANS. MODEL. ˆ β KP IS A FUNCTION OF THE RANKS OF Y i , AS IS THE COX ESTIMATE. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 10/31

HODGES-LEHMANN INVERSE MLE (1963) DEFINITION : IN THE LINEAR MODEL, ˆ β HL SOLVES ∇ β L H ( r ( y − z T β ∗ ); β ) | β =0 = 0 1ST COMPUTE ∇ β L H ( r ( y ; β )) | β =0 , THEN CONSTRUCT AN ESTIMATING EQUATION IN β ∗ BY REPLACING y WITH y − z T β ∗ . HERE y − z T β IS THE ”INVERSE” OF y = z t β + ǫ . KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 11/31

HODGES-LEHMANN INVERSE MLE (1963) HL INVERSE LIK. EST: ALIGN RANK OF RESIDUALS WITH THE ”BASELINE” RANKS USING HOEFFDING LIKELIHOOD. EXAMPLE : TWO SAMPLE CASE, LOGISTIC SHIFT MODEL, F 2 ( x ) = F 1 ( x − ∆), ∇ ∆ L H | ∆=0 = WILCOXON STAT ˆ ∆ HL = med ( X 2 j − X 1 i ), KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 12/31

GENERAL HODGES-LEHMANN INVERSE MLE MODEL : Y = h ( ǫ, z , β ), LET g ( y ; z , β ) BE THE SOLUTION (INVERSE) FOR ǫ OF h ( ǫ, z , β ) = y . ˆ β HL SOLVES ∇ β L H [ g ( y ; z , β ∗ ); β ] | β =0 = 0 KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 13/31

GENERAL HODGES-LEHMANN INVERSE MLE IN THE EXAMPLE Y i = z T i β + ǫ , ∇ β L H [ r ( y − z T β ∗ ); β ] | β =0 ARE p LINEAR RANK STATISTICS n T nj ( β ∗ ) = � z j ) a n ( r i ( β ∗ )) , ( z ij − ¯ j = 1 , · · · , p i =1 WHERE r i ( β ∗ ) = RANK ( y i − z T i β ∗ ), AND � r � a n ( r ) = a , n +1 a ( u ) = − f ′ f ( F − 1 ( u )) KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 14/31

GENERAL HODGES-LEHMANN INVERSE MLE SCALE MODEL : Y = ǫ exp[ z T β ], ǫ ∼ F ǫ = Y / exp[ z T β ], a n ( RANK ( y i / exp[ z T i β ])), � r � a n ( r ) = a 1 , n +1 a 1 ( u ) = − F − 1 ( u ) f ′ f ( F − 1 ( u )) − 1 HERE ˆ β HL SOLVES i β ∗ ]); β ] | β =0 = 0 ∇ β L H [ r ( y i / exp[ z T WHICH IS EQUIVALENT TO n � z j ) a n ( r i ( β ∗ )) = 0 , ( z ij − ¯ j = 1 , · · · , p i =1 KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 15/31

GENERAL HODGES-LEHMANN INVERSE MLE LINEAR MODELS : r EX1 : ǫ ∼ LOGISTIC = ⇒ a n ( r ) = n +1 ⇒ a n ( r ) = Φ − 1 � r � EX2 : ǫ ∼ NORMAL = , n +1 NORMAL SCORES SCALE MODEL : � r � EX3 : ǫ ∼ EXP = ⇒ a n ( r ) = − log 1 − , n +1 ˆ β HL IS THE LOGISTIC SCORES ESTIMATOR KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 16/31

ASYMPTOTICS THEOREM (JAECKEL 1972) : IN THE Y i = z T i β + ǫ MODEL, 1 THE HL ESTIMATE ˆ β HL IS A MAXIMIZER OF n � exp[ − ( y i − z T i β ) · a n ( RANK ( y i − z T S ( β ) = i β ))] i =1 2 HERE log[ S ( β )] is NONNEGATIVE, CONTINUOUS AND CONCAVE. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 17/31

ASYMPTOTICS IN THE LINEAR MODEL, LET ϕ ( u , f 0 ) = − f ′ f 0 ( F − 1 0 ( u )). THEN, 0 � � 1 � 2 0 [ ϕ ( u , f 0 ) − ¯ φ ] 2 du √ n (ˆ Σ − 1 ) β HL − β ) → N (0 , � 1 0 ϕ ( u , f 0 ) ϕ ( u , f ) du � 1 WHERE ¯ 0 ϕ ( u , f 0 ) du ,Σ = LIM n →∞ n − 1 Z T Z , φ = Z = CENTERED DESIGN MATRIX, ǫ i ∼ F , f = F ′ HERE f 0 ( · ) GENERATES L H ( · ) AND ˆ β HL . f ( · ) IS THE TRUE DENSITY of ǫ . KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 18/31

ASYMPTOTIC LINEAR MODEL EXAMPLES : 1 F 0 ()=LOGISTIC � � √ n (ˆ 1 Σ − 1 ) β HL − β ) → N (0 , � 1 0 f 2 ( u ) du ) 2 12( 2 F 0 ()= NORMAL (0 , σ 2 ) � � √ n (ˆ Σ − 1 β HL − β ) → N (0 , ) � 1 0 Φ − 1 ( u ) φ ( u , f ) du ) 2 ( KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 19/31

ASYMPTOTIC INEQUALITY HODGES-LEHMANN (56) CONJECTURE. CHERNOFF-SAVAGE (58) THEOREM. IF ˆ β HL IS BASED ON SCORES DERIVED BY TAKING f 0 = N (0 , 1), AND IF ˆ β MLE IS THE MLE FOR THE MODEL WITH ǫ ∼ N (0 , σ 2 ), THEN ASYMPTOTIC VARIANCE F (ˆ β HL ) ≤ ASYMPTOTIC VARIANCE F (ˆ β MLE ) WHERE F = TRUE DIST. OF ǫ . EQUALITY ONLY WHEN F = N (0 , σ 2 ). KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 20/31

IN THE AFT MODEL WITH ǫ ∼ F , THE HL EXPONENTIAL SCORES STATISTIC SATISFIES � 2 � √ n (ˆ 1 Σ − 1 ) β HL − β ) → N (0 , � 1 0 t λ ( t ) dF ( t ) WHERE λ ( t ) = f ( t ) / [1 − F ( t )]. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 21/31

NAIVE MINIMAX THEORY RESULT : THE COX ESTIMATE IS ASYMPTOTICALLY MINIMAX FOR THE PROPORTIONAL HAZARD (PH) MODEL: λ ( y ; z ) = λ 0 ( y ) e z T β PROOF : STEP A : THE COX ESTIMATE IS OPTIMAL FOR THE EXPONENTIAL MODEL, β R E ( β, ˆ β ) = R E ( β, ˆ INF ˆ β C ) (2) STEP B : THE PH MODEL CAN BE WRITTEN AS Λ 0 ( Y ) ∼ EXP-DISTR( z T β ) WHERE Λ 0 ր IS THE BASELINE HAZARD FUNCTION. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 22/31

NAIVE MINIMAX THEORY THE COX ESTIMATE ˆ β C IS INVARIANT, β C ( y ) = ˆ ˆ β C (Λ 0 ( y )), SO IT HAS CONSTANT RISK, R F ( β, ˆ β C ) = R E ( β, ˆ sup β C ) , (3) F F ( y | z ) ∈ PH STEP C : SINCE THE EXP MODEL IS PH, R F ( β, ˆ β ) ≥ R E ( β, ˆ sup β ) , (4) F F ( y | z ) ∈ PH STEP D : (2),(3),(4) ⇒ R F ( β, ˆ R F ( β, ˆ sup β C ) = inf sup β ) . QED . ˆ F β F KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 23/31

NAIVE MINIMAX THEORY NON-NAIVE PROOF: PAGE 332 of BICKEL, KLAASSEN, RITOV, WELLNER. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 24/31

NAIVE MINIMAX THEORY RESULT : THE HL EXP SCORES EST IS A MINIMAX FOR THE IHR ACCELERATED FAILURE TIME MODEL (IHRAFT) Y = Y 0 exp( z T β ) , Y 0 ∼ F , WITH F ∈ IHR = INCR. HAZARD RATE PROOF : STEP A : THE HL EXP. SC. ESTIMATE IS OPTIMAL FOR THE EXPONENTIAL MODEL, β R E ( β, ˆ β ) = R E ( β, ˆ INF ˆ β HL ) (5) KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 25/31

NAIVE MINIMAX THEORY STEP B : THE EXP MODEL IS LEAST FAVORABLE FOR ˆ β HL R F ( β, ˆ β HL ) = R E ( β, ˆ sup β HL ) , (6) F F ( y | z ) ∈ IHRAFT KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 26/31

NAIVE MINIMAX THEORY STEP C : SINCE THE EXP MODEL IS IHRAFT, R F ( β, ˆ β ) ≥ R E ( β, ˆ sup β ) , (7) F F ( y | z ) ∈ IHRAFT STEP D : (5),(6),(7) ⇒ R F ( β, ˆ R F ( β, ˆ sup β HL ) = inf sup β ) . QED . ˆ F β F TO PROVE STEP B, USE DOKSUM (1967); ARGUMENT BASED ON VAN ZWET ORDERINGS. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 27/31

ASYMPTOTIC miniMAX RESULT CONSIDER f 0 = LOGISTIC, SO r i a n ( r i ) = (8) n + 1 THEN ˆ β HL IS ASYMPTOTICALLY miniMAX OVER THE CLASS OF DISTRIBUTIONS WITH (VAN ZWET TYPE) LIGHTER TAILS THAN THE LOGISTIC DISTRIBUTION. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 28/31

ONE STEP ESTIMATIORS LET ˆ τ BE A CONSISTENT ESTIMATOR OF 1 τ = (9) � 1 0 φ ( u , f 0 ) φ ( u , f ) du AND LET ˆ β LSE BE THE LSE OF β . DEFINE τ · ( Z T Z ) − 1 · T n ( RANK ( Y − Z T ˆ β HL = ˆ ˆ β LSE + ˆ β LSE )) (10) THEN, √ n (ˆ β HL − β ) → N (0 , Γ) (11) JURECKOVA(69), KRAFT AND VAN EEDEN(72), HETTMANSPERGER, MCKEAN, TSIATIS, ETC. KJELL DOKSUM DEPT. OF STAT. AT UW-MADISON HLE’s 29/31