L-estimators, R-estimators, Redescending M gr. Jakub Petr asek - PowerPoint PPT Presentation

L-estimators, R-estimators, Redescending Estimators L-estimators, R-estimators, Redescending M gr. Jakub Petr´ asek Estimators Revision Seminar in Stochastic Modelling in Economics and Finance L-estimators R-estimators B-Robustness M gr. Jakub Petr´ asek and V-Robustness Redescending Estimators Covariance November 2, 2009 Matrices Estimators Bibliography

L-estimators, R-estimators, Redescending Estimators Outline M gr. Jakub Petr´ asek Revision L-estimators 1 Revision R-estimators B-Robustness and 2 L-estimators V-Robustness Redescending Estimators 3 R-estimators Covariance Matrices Estimators 4 B-Robustness and V-Robustness Bibliography 5 Redescending Estimators 6 Covariance Matrices Estimators

L-estimators, R-estimators, Model Set-up Redescending Estimators M gr. Jakub Petr´ asek Revision We consider random sample X 1 , ..., X n from sample space X . L-estimators We do not assume that observations belong to some parametric R-estimators model { F θ , θ ∈ Θ } , instead we work with the class F ( X ), B-Robustness and which describes all possible probability distributions on X . V-Robustness As an estimator we consider real-valued statistics Redescending Estimators Covariance T n = T n ( X 1 , ..., X n ) , Matrices Estimators assymptotically we can replace it by a functional Bibliography T ( X 1 , ... ) = T ( G ) , G ∈ F ( X ) . We work with Fisher consistent estimators, i.e. T ( F ) = θ . 1

L-estimators, R-estimators, Tools Redescending Estimators M gr. Jakub Petr´ asek Revision L-estimators Definition (Influence function) R-estimators B-Robustness T ((1 − t ) F + t ∆ x ) − T ( F ) and IF ( x ; T , F ) = lim . V-Robustness t t → 0 Redescending Estimators IF describes the sensitivity of an estimator to contamination at the Covariance point x . Matrices Estimators Lemma (Variance of an estimator) Bibliography � IF ( x ; T , F ) 2 d F ( x ) . V ( T , F ) = 2

L-estimators, R-estimators, Redescending Estimators M gr. Jakub Petr´ asek Theorem (Rao-Cram´ er bound) Revision L-estimators 1 V ( T , F ) ≥ J ( θ ∗ ) , R-estimators B-Robustness and where � � ∂ � 2 V-Robustness J ( θ ∗ ) = Redescending ∂θ [ln f θ ( x )] θ ∗ d F ∗ ( x ) Estimators Covariance Matrices is Fisher information and the lower bound is attached if and Estimators ∂ only if IF ( x ; T , F ) is proportional to ∂θ [ln f θ ( x )] θ ∗ . Bibliography 3

L-estimators, R-estimators, Robustness Measures Redescending Estimators M gr. Jakub Petr´ asek Revision Definition (Gross-error sensitivity) L-estimators γ ∗ ( T , F ) = sup R-estimators x | IF ( x ; T , F ) | . B-Robustness and Estimator is B-robust if γ ∗ < ∞ . V-Robustness Redescending Estimators Definition (Local-shift sensitivity) Covariance Matrices Estimators λ ∗ ( T , F ) = sup | IF ( x ; T , F ) − IF ( y ; T , F ) | / | y − x | . Bibliography x � = y Definition (Rejection point) ρ ∗ ( T , F ) = inf { r > 0; IF ( x ; T , F ) = 0 , | x | > r } . 4

L-estimators, R-estimators, Robustness Measures Redescending Estimators M gr. Jakub Petr´ asek Revision L-estimators R-estimators Definition (Qualitative Robustness) B-Robustness and π ( F , G ) < δ ⇒ π ( L F ( T n ) , L G ( T n )) < ǫ, V-Robustness Redescending where π denotes Prohorov metric. Estimators Covariance Matrices Estimators Definition (Breakdown point) Bibliography The smallest proportion of observations that can destroy the estimator. 5

L-estimators, R-estimators, M-estimators Redescending Estimators M gr. Jakub Derived from Maximum-Likelihood estimators. Solves Petr´ asek minimization problem Revision � L-estimators argmin ρ ( X i ; θ ) , R-estimators θ i B-Robustness and if ρ is differentiable, the M-estimator is defined by the equation V-Robustness � Redescending ψ ( X i ; θ ) = 0 . Estimators Covariance i Matrices Estimators Example Bibliography ρ ( x , θ ) = − ln f θ ( x ) defines Maximum-likelihood estimator. ψ ( x ) = min { max { x , − b } , b } defines Huber estimator for standard normal distribution. One needs to handle with optimization to get the M-estimator, another types were proposed. 6

L-estimators, R-estimators, L-estimators - location Redescending Estimators M gr. Jakub Petr´ asek Definition Revision n � T n ( X 1 , ..., X n ) = a i X n , ( i ) , (2.1) L-estimators R-estimators i =1 B-Robustness where and � i / n � 1 V-Robustness a i = h ( x ) d x , h ( x ) d x = 1 . Redescending Estimators ( i − 1) / n 0 Covariance Matrices We define the empirical quantile function Estimators Bibliography n ( y ) = X n , ( i ) , for i − 1 < y ≤ i G − 1 n n and the theoretical counterpart is G − 1 ( y ) = inf { x , G ( x ) ≥ y } . Assumption G is strictly increasing and absolutely continuous. 7

L-estimators, R-estimators, Influence function - derivation Redescending Estimators M gr. Jakub The corresponding functional to the estimator (2.1) is of the Petr´ asek form Revision � 1 � ∞ L-estimators G − 1 ( y ) h ( y ) d y = T ( G ) = xh ( G ( x )) d G ( x ) R-estimators 0 −∞ B-Robustness and Let us denote G t ( y ) = (1 − t ) F ( y ) + t I [ y ≥ x ] , then V-Robustness Redescending  F − 1 � � Estimators  u , u ≤ (1 − t ) F ( x )   Covariance 1 − t Matrices G − 1 ( u ) = x , (1 − t ) F ( x ) < u ≤ (1 − t ) F ( x ) + t Estimators t F − 1 � �    u − t Bibliography , u > (1 − t ) F ( x ) + t 1 − t so  u 1  1 − t )) , u ≤ (1 − t ) F ( x ) d G − 1 ( u ) (1 − t ) 2 f ( F − 1 ( u t = u − t 1 d t  1 − t )) , u > (1 − t ) F ( x ) + t (1 − t ) 2 f ( F − 1 ( u − t 8

L-estimators, R-estimators, Influence function - derivation Redescending Estimators M gr. Jakub Petr´ asek � 1 h ( u ) d G − 1 d T ( G t ) ( u ) Revision t = d u d t d t L-estimators 0 R-estimators we substitute into the integral above and set t = 0 to get B-Robustness and V-Robustness � F ( x ) Redescending 1 Estimators IF ( x ; T , F ) = uh ( u ) f ( F − 1 ( u )) d u Covariance 0 � 1 Matrices 1 Estimators + ( u − 1) h ( u ) f ( F − 1 ( u )) d u Bibliography F ( x ) � 1 � 1 1 1 = uh ( u ) f ( F − 1 ( u )) d u − h ( u ) f ( F − 1 ( u )) d u 0 F ( x ) � ∞ � ∞ = F ( y ) h ( F ( y )) d y − h ( F ( y )) d u . −∞ x 9

L-estimators, R-estimators, Examples - location Redescending Estimators M gr. Jakub Petr´ asek Revision Median � L-estimators h = δ 1 / 2 , i.e. h ( x ) d x = 1 R-estimators � G − 1 ( y ) h ( y ) d y = G − 1 (1 / 2) . T ( G ) = B-Robustness and 2 f ( F − 1 (1 / 2)) sgn ( x − F − 1 (1 / 2)) . 1 IF ( x , T , F ) = V-Robustness Redescending Estimators α -trimmed mean Covariance Matrices Estimators h ( x ) = I [ α, 1 − α ] ( x ) � 1 − α Bibliography 1 G − 1 ( y ) d y . T ( G ) = 1 − 2 α α IF ( x , T , F ) see Jureˇ ckov´ a, pages 64 -65. γ ∗ = F − 1 (1 − α ) . 1 − 2 α Breakdown point ǫ ∗ = α 10

L-estimators, R-estimators, Maximal Asymptotic Efficiency Redescending Estimators M gr. Jakub Petr´ asek Remind that Rao-Cram´ er says Revision � 1 L-estimators IF ( x ; T , F θ ∗ ) 2 d F θ ∗ ≥ V ( T , F θ ∗ ) = R-estimators J ( F θ ∗ ) B-Robustness and and equality holds if and only if IF ( x ; T , F θ ∗ ) is proportional to V-Robustness ∂ ∂θ [ln f θ ( x )] θ ∗ or in other words h ( F ( x )) must be proportional Redescending � ∂ � Estimators ∂ to ∂θ [ln f θ ( x )] θ ∗ . Covariance ∂ x Matrices Normal distribution: h ( x ) = 1 / n Estimators Bibliography Logistic distribution: h ( x ) = 6 x (1 − x ) Remark The shape of IF of L-estimators depends on the distribution F whereas for M-estimators IF is proportional to ψ regardless of the distribution. 11

L-estimators, L-estimators - scale R-estimators, Redescending Estimators Definition M gr. Jakub Petr´ asek We again use the formula (2.1) but now the weights are Revision defined by L-estimators � i / n � 1 R-estimators h ( x ) F − 1 ( x ) d x = 1 . a i = h ( x ) d x , B-Robustness and ( i − 1) / n 0 V-Robustness Redescending Function h is usually chosen as skew symmetric Estimators ( h (1 − u ) = − h ( u )). Covariance Matrices Estimators The corresponding functional is given by Bibliography � xh ( G ( x )) d G ( x ) � T ( G ) = xh ( F ( x )) d F ( x ) and � ∞ � ∞ −∞ F ( y ) h ( F ( y )) d y − h ( F ( y )) d y x � IF ( x ; T , F ) = . xh ( F ( x )) d F ( x ) 12

L-estimators, R-estimators, Examples - scale Redescending Estimators M gr. Jakub Petr´ asek Revision L-estimators R-estimators Let h = δ 1 − t − δ t for 0 < t < 1 / 2. Then B-Robustness and T ( G ) = G − 1 (1 − t ) − G − 1 ( t ) V-Robustness Redescending F − 1 (1 − t ) − F − 1 ( t ) Estimators Covariance Matrices For t = 1 / 4 and F = Φ we obtain the same IF as for MAD. Estimators Robustness measures which depend on IF are the same. Bibliography Breakdown point, which is ’only’ ǫ ∗ = 1 / 4. 13