Censored Quantile Regression Redux Roger Koenker University of - PowerPoint PPT Presentation

Censored Quantile Regression Redux Roger Koenker University of Illinois, Urbana-Champaign CEMMAP QR Conference: 1-2 June 2009 Inspired by discussions (and coffee consumption) with Steve Portnoy and Xuming He. Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 1 / 28

Quantile Regression for Survival Models A wide variety of survival analysis models, following Doksum and Gasko (1990), may be written as, h ( T i ) = x ⊤ i β + u i where h is a monotone transformation, and T i is an observed survival time, x i is a vector of covariates, β is an unknown parameter vector { u i } are iid with df F . Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 2 / 28

The Cox Model For the proportional hazard model with log λ ( t | x ) = log λ 0 ( t ) − x ⊤ β the conditional survival function in terms of the integrated baseline hazard � t Λ 0 ( t ) = 0 λ 0 ( s ) ds as, log (− log ( S ( t | x ))) = log Λ 0 ( t ) − x ⊤ β so, evaluating at t = T i , we have the model, log Λ 0 ( T ) = x ⊤ β + u for u i iid with df F 0 ( u ) = 1 − e − e u . Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 3 / 28

The Bennett (Proportional-Odds) Model For the proportional odds model, where the conditional odds of death Γ ( t | x ) = F ( t | x ) / ( 1 − F ( t | x )) are written as, log Γ ( t | x ) = log Γ 0 ( t ) − x ⊤ β , we have, similarly, log Γ 0 ( T ) = x ⊤ β + u for u iid logistic with F 0 ( u ) = ( 1 + e − u ) − 1 . Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 4 / 28

Accelerated Failure Time Model In the accelerated failure time model we have log ( T i ) = x ⊤ i β + u i so P ( e u > te − xβ ) P ( T > t ) = 1 − F 0 ( te − xβ ) = where F 0 ( · ) denotes the df of e u , and thus, λ ( t | x ) = λ 0 ( te − xβ ) e − xβ where λ 0 ( · ) denotes the hazard function corresponding to F 0 . In effect, the covariates act to rescale time in the baseline hazard. Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 5 / 28

Beyond the Transformation Model The common feature of all these models is that after transformation of the observed survival times we have: a pure location-shift, iid-error regression model covariate effects shift the location of the distribution of h ( T ) , but covariates cannot affect scale, or shape of this distribution Warning: in this form unsuitable for time varying covariates! Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 6 / 28

An Application: Longevity of Mediterrean Fruit Flies In the early 1990’s there were a series of experiments designed to study the survival distribution of lower animals. One of the most influential of these was: Carey, J.R., Liedo, P., Orozco, D. and Vaupel, J.W. (1992) Slowing of mortality rates at older ages in large Medfly cohorts, Science, 258 , 457-61. 1,203,646 medflies survival times recorded in days Sex was recorded on day of death Pupae were initially sorted into one of five size classes 167 aluminum mesh cages containing roughly 7200 flies Adults were given a diet of sugar and water ad libitum Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 7 / 28

Major Conclusions of the Medfly Experiment Mortality rates declined at the oldest observed ages. contradicting the traditional view that aging is an inevitable, monotone process of senescence. The right tail of the survival distribution was, at least by human standards, remarkably long. There was strong evidence for a crossover in gender specific mortality rates. Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 8 / 28

Lifetable Hazard Estimates by Gender 0.15 M mortality rate 0.10 F 0.05 0.00 0 20 40 60 80 100 120 days Smoothed mortality rates for males and females. Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 9 / 28

Quantile Regression Model (Geling and K (JASA,2001)) Criticism of the Carey et al paper revolved around whether declining hazard rates were a result of confounding factors of cage density and initial pupal size. Our basic QR model included the following covariates: Q log ( T i ) ( τ | x i ) = β 0 ( τ ) + β 1 ( τ ) SEX + β 2 ( τ ) SIZE + β 3 ( τ ) DENSITY + β 4 ( τ ) %MALE SEX Gender Pupal Size in mm SIZE DENSITY Initial Density of Cage Initial Proportion of Males %MALE Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 10 / 28

Base Model Results with AFT Fit 0.2 ● ● ● ● ● ● ● 3.5 ● ● ● ● Gender Effect 0.1 ● ● ● Size Effect ● ● ● Intercept ● ● ● 0.05 ● ● ● ● ● ● ● ● ● ● 0.0 ● ● ● ● ● ● 2.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● −0.05 ● ● 1.5 ● −0.2 ● ● ● ● ● ● ● ● ● ● 0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8 Quantile Quantile Quantile 1.5 3 Density Effect %Male Effect 1.0 2 ● ● ● ● ● ● ● ● ● ● ● ● ● 1 ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 ● 0.0 −1 0.0 0.4 0.8 0.0 0.4 0.8 Quantile Quantile Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 11 / 28

Base Model Results with Cox PH Fit 0.2 ● ● ● ● ● ● ● 3.5 ● ● ● ● Gender Effect 0.1 ● ● ● Size Effect ● ● ● Intercept ● ● ● 0.05 ● ● ● ● ● ● ● ● ● ● 0.0 ● ● ● ● ● ● 2.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● −0.05 ● ● 1.5 ● −0.2 ● ● ● ● ● ● ● ● ● ● 0.0 0.4 0.8 0.0 0.4 0.8 0.0 0.4 0.8 Quantile Quantile Quantile 1.5 3 Density Effect %Male Effect 1.0 2 ● ● ● ● ● ● ● ● ● ● ● ● ● 1 ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 ● 0.0 −1 0.0 0.4 0.8 0.0 0.4 0.8 Quantile Quantile Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 12 / 28

What About Censoring? There are currently 3 approaches to handling censored survival data within the quantile regression framework: Powell (1986) Fixed Censoring Portnoy (2003) Random Censoring, Kaplan-Meier Analogue Peng/Huang (2008) Random Censoring, Nelson-Aalen Analogue Crucial assumption for random censoring is C ⊥ ⊥ Y | X Available for R in the package quantreg . Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 13 / 28

Powell’s Estimator for Fixed Censoring Rationale Quantiles are equivariant to monotone transformation: Q h ( Y ) ( τ ) = h ( Q Y ( τ )) for h ր Model Y i = T i ∧ C i ≡ min { T i , C i } Q Y i | x i ( τ | x i ) = x ⊤ i β ( τ ) ∧ C i Data Censoring times are known for all observations { Y i , C i , x i : i = 1, · · · , n } Estimator Conditional quantile functions are nonlinear in parameters: � ˆ ρ τ ( Y i − x ⊤ β ( τ ) = argmin i β ∧ C i ) Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 14 / 28

Portnoy’s Approach for Random Censoring I Rationale Efron’s (1967) interpretation of Kaplan-Meier as redistributing mass of censored observations to the right: Algorithm Until we “encounter” a censored observation KM quantiles can be computed by solving, starting at τ = 0 , n � ˆ ξ ( τ ) = argmin ξ ρ τ ( Y i − ξ ) i = 1 Once we “encounter” a censored observation, i.e. when ˆ ξ ( τ i ) = y i for some y i with δ i = 0 , we split y i into two parts: ◮ y ( 1 ) = y i with weight w i = ( τ − τ i ) / ( 1 − τ i ) i ◮ y ( 2 ) = y ∞ = ∞ with weight 1 − w i . i Then denoting the index set of censored observations “encountered” up to τ by K ( τ ) we can solve recursively: � � min ρ τ ( Y i − ξ )+ [ w i ( τ ) ρ τ ( Y i − ξ )+( 1 − w i ( τ )) ρ τ ( y ∞ − ξ )] . i/ ∈ K ( τ ) i ∈ K ( τ ) Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 15 / 28

Portnoy’s Approach for Random Censoring II When we have covariates we can replace ξ by the inner product x ⊤ i β , again splitting the censored points as they are encountered, and solve recursively: � � ρ τ ( Y i − x ⊤ [ w i ( τ ) ρ τ ( Y i − x ⊤ i β )+( 1 − w i ( τ )) ρ τ ( y ∞ − x ⊤ min i β )] . i β )+ i/ ∈ K ( τ ) i ∈ K ( τ ) At each τ this is a simple, weighted linear quantile regression problem. Roger Koenker (U. of Illinois) Censored Quantile Regression Redux CEMMAP: 1.6.2009 16 / 28