Estimation — Functional linear model Adaptive estimation in functional linear model: a model selection approach Angelina Roche joint work with Élodie Brunel and André Mas I3M-Université Montpellier II 7 èmes Journées de Statistique Fonctionnelle et Opératorielle 28-29 Juin 2012, Montpellier 1 / 34
Estimation — Functional linear model Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 2 / 34
Estimation — Functional linear model Model definition and estimation context Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 3 / 34
Estimation — Functional linear model Model definition and estimation context Functional linear regression model We model the dependence between a functional random predictor X and a scalar response Y by the linear relation: � 1 (1) Y = β ( t ) X ( t ) dt + ε, 0 were β is an unknown function in L 2 ([ 0 , 1 ]) , the slope function; X is a random variable with values in L 2 ([ 0 , 1 ]) , centred, ε is a real random variable, centred, independent of X and with variance σ 2 . The aim is to estimate the function β from the data of a sample { ( X i , Y i ) , i = 1 , ..., n } verifying Equation (1). 4 / 34
Estimation — Functional linear model Model definition and estimation context Some existing works on functional linear model Many estimation procedures with asymptotic convergence results (see e.g. Cardot et al. (1999, 2003), Cai and Hall (2006), Hall and Horowitz (2007), Crambes et al. (2009),...). Optimal theoretical choice of smoothing parameters depends on both unknown regularities of the slope β and the predictors X . Smoothing parameters obtained in practice by cross-validation. Non-asymptotic results providing adaptative data-driven estimators were missing up to the recent papers of Comte and Johannes (2010). 5 / 34
Estimation — Functional linear model Model definition and estimation context Prevision error – Definition Let ( X n + 1 , Y n + 1 ) independent of the sample ( X 1 , Y 1 ) , ..., ( X n , Y n ) and � 1 ˆ ˆ Y n + 1 = β ( s ) X n + 1 ( s ) ds , 0 the value of Y n + 1 predicted from X n + 1 and the estimator ˆ β . Prevision error The prevision error of ˆ β is the quantity: �� � � 2 � β − β, ϕ j > 2 ˆ λ j < ˆ Y n + 1 − E [ Y n + 1 | X n + 1 ] | X 1 , ..., X n = E j ≥ 1 β − β � 2 � ˆ =: Γ , where, for all j , λ j is the eigenvalue of the covariance operator Γ associated to the eigenfunction ϕ j and < · , · > is the usual scalar product of L 2 ([ 0 , 1 ]) . 6 / 34
Estimation — Functional linear model Model definition and estimation context Reformulation of the problem equation Multiplying both sides of the model equation by X ( s ) and taking the expectation we obtain the following formulation of the problem: �� 1 � g ( s ) := E [ YX ( s )] = E β ( t ) X ( t ) dt X ( s ) =: Γ β ( s ) , 0 where Γ is the covariance operator associated to the random function X . The problem of estimating the function β is then clearly related to the inversion of the covariance operator Γ or of its empirical equivalent n � Γ n : f ∈ L 2 ([ 0 , 1 ]) �→ 1 < f , X i > X i . n i = 1 7 / 34
Estimation — Functional linear model Model definition and estimation context Dimension reduction Aim: Find an approximation space S m of dimension D m < + ∞ containing as much information as possible. Final objective is to define an estimator based on functional PCA i.e. take S m as the space spanned by the eigenfunctions associated to the m largest eigenvalues of Γ n . Problem: Difficulty to control an estimator defined on a random space. 8 / 34
Estimation — Functional linear model Model definition and estimation context Hypothesis – Consequence on the covariance operator eigenfunctions Context of “circular data”: assumption on the curve X The curve X is supposed to be 1-periodic ( X ( 0 ) = X ( 1 ) ) and second-order stationary. In this context the Fourier basis ( ϕ j ) j ≥ 1 of L 2 ([ 0 , 1 ]) defined by √ √ ϕ 1 ≡ 1 , ϕ 2 j ( · ) = 2 cos ( 2 π j · ) et ϕ 2 j + 1 ( · ) = 2 sin ( 2 π j · ) , is a basis of eigenfunctions of the covariance operator Γ . 9 / 34
Estimation — Functional linear model Estimation procedure of the slope function Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 10 / 34
Estimation — Functional linear model Estimation procedure of the slope function Estimation– Minimization of the least square contrast Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 11 / 34
Estimation — Functional linear model Estimation procedure of the slope function Estimation– Minimization of the least square contrast Dimension reduction Approximation spaces Let N n ∈ N ∗ and m ∈ { 1 , ..., N n } , we denote by S m := span { ϕ 1 , ..., ϕ 2 m + 1 } , the linear space, called model, spanned by the trigonometric basis. For all m ∈ { 1 , ..., N n } , we define an estimator of β on the space S m . 12 / 34
Estimation — Functional linear model Estimation procedure of the slope function Estimation– Minimization of the least square contrast Estimation on S m minimization of the least square contrast For all m ∈ { 1 , ..., N n } , we compute the least square estimator on S m : ˆ β m := arg min f ∈ S m γ n ( f ) where γ n is the least square contrast defined by: n � γ n : f �→ 1 ( Y i − < f , X i > ) 2 . n i = 1 After this first step, we obtain a family { ˆ β m , m = 1 , ..., N n } of estimators of the slope function β . 13 / 34
Estimation — Functional linear model Estimation procedure of the slope function Penalized contrast model selection Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 14 / 34
Estimation — Functional linear model Estimation procedure of the slope function Penalized contrast model selection Dimension selection – heuristic Define by m ∗ the unknown ideal dimension, called oracle, m ∗ := arg min m = 1 ,..., N n E [ � β − ˆ β m � 2 Γ ] , the idea is to define a data-driven criterion which allows to select a dimension with performance similar to the oracle. Let β m be the orthogonal projection of β into S m , we have the following bias-variance decomposition β m � 2 Γ ] = E [ � β − β m � 2 β m � 2 E [ � β − ˆ Γ ] + E [ � β m − ˆ Γ ] . The idea is to define a criterion with a similar behaviour crit ( m ) := γ n (ˆ β m ) + pen ( m ) . 15 / 34
Estimation — Functional linear model Estimation procedure of the slope function Penalized contrast model selection Dimension selection via penalisation We choose then an element of the family { ˆ β m , m = 1 , ..., N n } minimizing the penalized criterion: crit ( m ) := γ n (ˆ β m ) + pen ( m ) , σ 2 , κ is a numerical constant and σ 2 can be where pen ( m ) := κ 2 m + 1 n replaced by an estimator ˆ σ 2 m (see section simulations). The estimator selected by our penalized criterion is then ˆ m with β ˆ m ∈ arg min m = 1 ,..., N n crit ( m ) . ˆ 16 / 34
Estimation — Functional linear model Upper and lower bound on the risk Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 17 / 34
Estimation — Functional linear model Upper and lower bound on the risk Oracle-inequality Summary Model definition and estimation context 1 Estimation procedure of the slope function 2 Estimation– Minimization of the least square contrast Penalized contrast model selection Upper and lower bound on the risk 3 Oracle-inequality Convergence rate over Sobolev spaces Numerical results 4 Simulation method Results 18 / 34
Recommend
More recommend
