Estimating beyond the trial-represented population by incorporating studies with self-selected treatments Eloise E. Kaizar Department of Statistics Ohio State University February 26, 2016 Joint work with: Joel Greenhouse, Kelly Kelleher, Taylor Pressler Vydra, Howard Seltman (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 1 / 18
Does antidepressant use cause suicide among adolescents? No single study could definitively answer Available randomized evidence (Hammad, et al. 2006) 24 studies of various psychological disorders 4582 subjects No completed suicides Instead adjudicated “suicidality events" n=87 (2%) experienced suicidality Goal: Estimate the ’average’ safety of antidepressant use (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 2 / 18
Outline Are Studies Generalizable? 1 Combining RCT and Observational Data 2 Extensions 3 (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 3 / 18
Are Studies Generalizable? Who do RCTs actually study? Studies consistently show that it is usual for RCTs to exclude half or more of the population of interest due to eligibility criteria Asthma, > 57 % (Travers, et al, 2007) Alcohol treatment, 6 − 70 % (Humphreys and Weisner, 2000) Antidepressants, mean=66 % (Zimmerman, et al, 2004) (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 4 / 18
Are Studies Generalizable? Assessing Consistency with Target Populations (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 5 / 18
Are Studies Generalizable? Assessing Consistency with Target Populations, cont. (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 6 / 18
Combining RCT and Observational Data General Methods for Combining RCT and Observational Data Three-level meta-analysis (Prevost, Abrams & Jones, 2000) Response surface (Rubin, 1990) Bias-adjusted model, e.g., Confidence Profile Method (Eddy, Hasselblad & Shachter, 1990) Multiple bias models (Greenland, 2005) Cross Design Synthesis (US GAO, 1992) (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 7 / 18
Combining RCT and Observational Data Cross Design Synthesis a b Nonrandomized Treatment effect Randomized Subjects select Treatment effect treatment Subjects randomized to treatment I n t e y r m t n i d External validity moderators o a i l l a d v s v r e a l o r a t l n a i a d r r t e e o i t d t x r o y E s m (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 8 / 18
Combining RCT and Observational Data Framework Use RCT to estimate effect size in the group it represents Rely on strong internal validity Use observational data to estimate the generalizability bias Rely on strong external validity Put the two estimators together to estimate the population average effect size (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 9 / 18
Combining RCT and Observational Data Simplest Cross Design Synthesis Linear Generalizability Bias (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 10 / 18
Combining RCT and Observational Data Simple Example: Insulin Pump Use Question: Is insulin pump use on average effective in improving metabolic control in the total population of diabetic patients? Outcome: Mean A1C level (lower is better) Control: Insulin injections Goal: Estimate the average treatment effect for use in policy decision making Issue: RCTs exclude the noncompliant ( < 4 checks per day) Doyle, et al (2004), Paris, et al (2009) (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 11 / 18
Combining RCT and Observational Data (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 12 / 18
Combining RCT and Observational Data Assumptions for Unbiasedness No confounding within the levels of inclusion U ⊥ ⊥ T | X or E ( Y | T , U , X ) = g ( T , X ) Socioeconomic status is independent of the choice to use an insulin pump, within monitoring groups, OR Socioeconomic status does not influence metabolic control. Confounding due to inclusion criteria and other variables are separate U ⊥ ⊥ X | T and U ⊥ ⊥ X and E ( Y | T , U , X ) = g 1 ( T , X ) + g 2 ( T , U ) The frequency of glucose monitoring is independent of socioeconomic status and does not moderate its effect on metabolic control. Note that unlike the experimental estimator, the inclusion criterion can moderate treatment, as long as this is separate from the other variables. (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 13 / 18
Combining RCT and Observational Data CDS Properties Bias CDS is better whenever the scaled difference in expected treatment selection error is smaller than the difference in generalizability bias (sample selection error) Variance CDS is never better, although the difference is expected to be small whenever the observational data is large (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 14 / 18
Extensions Improving the Plausibility of Assumption 1 Assumption 1: No confounding within the levels of inclusion Approach: Use existing methods (e.g., propensity scores) to reduce/eliminate confounding within levels of inclusion Simulation Three standard normal covariates, constant correlation One influences treatment assignment Another influences exclusion. (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 15 / 18
Extensions Estimating Generalizability Bias (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 16 / 18
Summary Summary CDS offers a framework for thinking about estimating average treatment effects in populations of interest that are not represented by RCTs. Simple CDS models rely on two rather restrictive (relative to ideal) assumptions. More work is necessary to relax such assumptions and improve its usefulness. CDS often replaces one data limitation with several others: Data availability (including treatment in the observational study) Data harmonization (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 17 / 18
References References Doyle E A, Weinzimer SA, Steffen AT, Ahern JAH, Vincent M, Tamborlane WV. 2004. A randomized, prospective trial comparing the efficacy of continuous subcutaneous insulin infusion with multiple daily injections using insulin glargine. Diabetes Care . 27: 1554-1558. Eddy DM, Hasselblad V, Shachter R. 1990. A Bayesian method for synthesizing evidence: the confidence profile method. International Journal of Technology Assessment in Health Care . 6: 31-56. Greenland S. Multiple-bias modelling for analysis of observational data. 2005. Journal of the Royal Statistical Society: Series A . 168: 267-306. Kaizar EE. 2011. Estimating treatment effect via simple cross design synthesis. Statistics in Medicine . 30(25):2986-3009. Kaizar EE. 2015. Incorporating Both Randomized and Observational Data into a Single Analysis. Annual Review of Statistics and Its Application . 2:49-72 Paris CA, Imperatore G, Klingensmith G, Petitti D, Rodriguez B, Anderson AM, Schwartz ID, Standiford DA, Pihoker C. 2009. Predictors of insulin regimens and impact on outcomes in youth with type 1 diabetes: the SEARCH for Diabetes in Youth study. Journal of Pediatrics . 155: 183-189. Pressler T and Kaizar EE, 2013. The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias. Statistics in Medicine . 32(20):3552-3568. Prevost TC, Abrams, KR, Jones DR. 2000. Hierarchical models in generalized synthesis of evidence: an example based on studies of breast cancer screening. Statistics in Medicine . 19: 3359-3376. Rubin, D. 1990. A new perspective on meta-analysis. In K. M. Wachter & M. L. Straff, eds., The Future of Meta-Analysis . Russel Sage Foundation, 155-165. U.S. General Accounting Office. 1992. Cross-design synthesis: A new strategy for medical effectiveness research (GAO/PEMD-92-18). U.S. General Accounting Office, Washington, DC. http://archive.gao.gov/d31t10/145906.pdf . (Ohio State University) Estimating Beyond the Trial Ross-Royall Symposium 18 / 18
Recommend
More recommend
