Statistical Methods for Infectious Disease Using Validation Sets for - PowerPoint PPT Presentation

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Statistical Methods for Infectious Disease Using Validation Sets for Outcomes in Vaccine Studies M. Elizabeth Halloran Fred Hutchinson Cancer Research Center and University of Washington Seattle, WA, USA March 6, 2009

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Validation Sets General Ideas Final Value Introduction Texas Field Study Analysis Analysis Issues Results Time-to-event The Study Analysis Sensitivity Analysis Other Current and future research

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Validation Sets General Ideas Final Value Introduction Texas Field Study Analysis Analysis Issues Results Time-to-event The Study Analysis Sensitivity Analysis Other Current and future research

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Vaccine efficacy for susceptibility VE S = 1 − RR RR = relative risk in vaccinated compared to unvaccinated ❼ incidence rates, hazard rates, incidence proportion, transmission probability ❼ if all ascertained cases actually disease of interest

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other The problem ❼ In many diseases, influenza, rotavirus, pertussis, and cholera, − → confirmatory diagnosis of suspected case by culture or quick test of specimen ❼ Often difficult or expensive − → use nonspecific case definition − → lower estimates than with specific case definition.

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Example: Influenza vaccine studies ❼ Live attenuated influenza vaccine in children (RCT): with culture confirmed influenza: − → VE S = 0 . 89 (Belshe, et al,1998) ❼ Similar vaccine in adults (RCT): case definition: “upper respiratory tract illness with either fever or cough”: − → VE S = 0 . 25 (Nichol, et al 1999).

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Studies with validation sets for outcomes ❼ In small validation sample, measure both − → good outcome of interest − → correlated auxiliary outcome that is easier or cheaper. ❼ In the large main study, measure − → just easy, cheap correlated auxiliary outcome ❼ Validation sample − → corrects bias ❼ Main study − → improves efficiency

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Validation Sets General Ideas Final Value Introduction Texas Field Study Analysis Analysis Issues Results Time-to-event The Study Analysis Sensitivity Analysis Other Current and future research

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Final value data ❼ Estimating Efficacy of Trivalent, Cold-Adapted, Influenza Virus Vaccine (CAIV-T) Using Surveillance Cultures ❼ Halloran, Longini, Gaglani, Piedra, Chu, Herschler, Glezen ❼ American Journal of Epidemiology, 2003, 158:305–311.

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Field Investigators ❼ W. Paul Glezen, Pedro A. Piedra, Department of Molecular Virology and Microbiology and Pediatrics, Baylor College of Medicine, Houston, Texas ❼ Manjusha J. Gaglani, Gayla B. Herschler, Charles Fewlass, Section of Pediatric Infectious Diseases, Department of Pediatrics, Scott & White Memorial Hospital & Clinic, Scott, Sherwood and Brindley Foundation, Texas A & M College of Medicine, Temple, Texas

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Field Study of Flumist (Medimmune) ❼ Originally for estimating indirect effects in adults of vaccinating children 1.5-18 years old. ❼ Study in three communities, one with vaccination, two without ❼ Here we use only community with vaccination to estimate direct protective effects, VE S

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Field Study of Flumist ❼ Community-based non-randomized, open-label study of experimental vaccine. ❼ Temple-Belton, Texas, August 1998 - June 2001. ❼ All children 1.5–18 years old offered Flumist through Scott & White Clinics. ❼ Beginning in Fall 2003, vaccinated 5 –18 years old after licensure

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other This analysis ❼ Concerned with epidemic year 2000-01 ❼ Children vaccinated either − → 1999 (and possibly 1998, not 2000) − → 2000 (and possibly 1999, 1998) ❼ Members of Scott & White Health Plan

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Non-specific case definition ❼ Medically-attended acute respiratory illness (MAARI) ❼ ICD-9 codes 381-383, 460-487 ❼ upper and lower respiratory track infections, otitis media, sinusitis, and asthma (with other diagnosis) ❼ influenza season defined by surveillance cultures

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Influenza surveillance cultures ❼ Any individual presenting with history of fever and any respiratory illness was eligible. ❼ Throat swab or nasal wash at discretion of health care provider with informed verbal consent ❼ Influenza A and B viruses characterized by CDC

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Viral Strain Information Vaccine Circulating virus wild type 1999-2000 A/Sydney (H3N2) A/Beijing (H1N1) not B/Beijing relevant 2000-01 A/Sydney (H3N2) — A/New Caledonia (H1N1) A/New Caledonia (H1N1) B/Beijing B/Sichuan

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Efficacy using actual influenza cases: ❼ N 1 , N 0 = number of children in vaccinated and unvaccinated groups, respectively. ❼ y 1 , y 0 = number of true influenza cases in vaccinated and unvaccinated groups, respectively. ❼ IP 1 and IP 0 = incidence (binomial) proportion in vaccinated and unvaccinated groups, respectively. = 1 − y 1 / N 1 VE S = 1 − IP 1 � , IP 0 y 0 / N 0

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Using nonspecific case definition: ❼ z 1 , z 0 = number of flu-like cases that are not true flu in vaccinated and unvaccinated groups, respectively. ❼ w ν = z ν + y ν = number of total MAARI cases in group ν , ν = 0 , 1. ❼ Based on total number of influenza-like illnesses, VE S , a = 1 − IP 1 , a = 1 − w 1 / N 1 � , IP 0 , a w 0 / N 0 ❼ a = all influenza-like illness, auxiliary outcome.

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Confidence intervals for VE S , a and VE S ❼ normal approximation of the log of the ratio of two independent binomial random variables

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Estimates using the surveillance samples, VE S , v ❼ Auxiliary outcome and the mean score method by Pepe, Reilly and Fleming (1994) ❼ Estimate score contribution for main study member with only auxiliary outcome data from − → average score contributions of validation sample with same observed covariate and auxiliary outcome values.

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Semiparametric method ❼ Parametric model for good data ❼ Nonparametric estimation or no estimation of relation between good data and surrogate measure ❼ Avoids misspecification of relation and resulting bias

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Auxiliary outcome and mean score method ❼ Pepe, Reilly and Fleming (1994) ❼ Y = outcome of interest (influenza status) ❼ A = auxiliary outcome (MAARI, yes, no) ❼ X = set of covariates (vaccination, age group) ❼ P β ( Y | X ) = probability model ❼ β = parameters to estimate in probability model ❼ S β = score function ❼ V , V = in validation set or not

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Estimation ❼ Estimating equation: � � ˆ S β ( Y i | X i ) + E { S β ( Y | X j ) | A j , X j } = 0 i ∈ V j ∈ V ❼ Unbiased estimator for nonvalidation set person: � S β ( Y i | X i ) / n V ( A j , X j ) ˆ E { S β ( Y | X j ) | A j , X j } = i ∈ V ( A j , X j ) ❼ Inference makes allowance for not having all data.

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Inverse proportional weighting ❼ Basically equivalent to analytic methods that weight observed or sampled case inversely to the probability of being observed. ❼ Horwitz-Thompson (1952) type estimators

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Advantages of this method ❼ Proven consistent ❼ Easy computation of variance estimator ❼ Simple expression for optimal sampling fractions ❼ Simple to explain to epidemiologists

Outline Validation Sets Final Value Time-to-event Sensitivity Analysis Other Potential drawbacks ❼ Restricted to categorical data ❼ Need validation members in every cell ❼ Assumes missing at random (MAR)