TRIPODS Workshop: Models & Machine Learning for Causal Inference - PowerPoint PPT Presentation

Microsimulation Models MSM for lung cancer Calibration Predictive accuracy TRIPODS Workshop: Models & Machine Learning for Causal Inference & Decision Making Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD Department of Biostatistics Brown University School of Public Health Jan 14-18, 2019 Providence, RI, USA TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Calibration Predictive accuracy Outline Microsimulation Models Definition MSM for lung cancer Description The MILC model Calibration Comparative Analysis Results Predictive accuracy Simulation Study Results TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Definition Calibration Predictive accuracy MSMs in medical decision making Micro-simulation models (MSMs): Complex predictive models aimed at simulating individual disease trajectories using Markov Chain Monte Carlo methods. ◮ Incorporate combined information from several sources ◮ Predict outcomes of interest under different medical interventions ◮ Compare findings to decide on the best practice (cost-effectiveness analysis) ◮ Inform health policies TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Description Calibration The MILC model Predictive accuracy The MIcrosimulation Lung Cancer (MILC) Model : an MSM describing the natural history of lung cancer Chrysanthopoulou SA (2017). MILC: A Microsimulation Model of the Natural History of Lung Cancer. International Journal of Microsimulation. 10(3):5-26 TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Description Calibration The MILC model Predictive accuracy Micro-Simulation Model for lung cancer ◮ Streamlined MSM model ◮ Continuous time ◮ Dynamic ◮ Natural history of lung cancer (no screening component) ◮ Covariates ➜ Age ➜ Gender ➜ Smoking: ⊲ Status (current, former, never) ⊲ Start and quit smoking age ⊲ Intensity (cigarettes/day) TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Description Calibration The MILC model Predictive accuracy Markov State Diagram S 0 : Disease-free state S 1 : Local state (onset of the 1st malignant cell) S 2 : Regional state (lumph nodes) S 3 : Distant state (distant metastases) S 4 : Death state Figure: Markov State Diagram of the h ij : the hazard of moving continuous time MSM for lung cancer from state i to j TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Description Calibration The MILC model Predictive accuracy Components • Onset of the 1st malignant cell → TSCE carcinogenesis model ( Moolgavkar 1990 ) • Tumor growth → Gompertz function ( Laird 1964 ) • Disease progression → log-Normal distribution ( Spratt 1964 ) • Survival → competing risks (CIF estimates) → NHIS and SEER data TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Description Calibration The MILC model Predictive accuracy Structure of the MILC model TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Description Calibration The MILC model Predictive accuracy • R package ” MILC ”: http://CRAN.R-project.org/package=MILC TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Calibration methods in MSMs: a comparative analysis TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Calibration vs Estimation in statistical theory Statistical modeling: ”calibration” ≡ ”estimation” ≡ ”model fitting” Here: ”calibration” ≡ ”model tuning” Calibration pertains to the specification of those sets of values for the model parameters that can result in predictions close to the observed, pre-specified target quantities. TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Why calibration? ◮ MSM’s complexity → no closed form of the model’s outputs (e.g., likelihoods, hazard rates, transition probabilities, etc.) ◮ Latent variables + high dimensionality → identifiability problems ◮ Multiple sets of parameter values → underlying correlation structures → parameter uncertainty TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Calibration methods for MSMs (overview) ◮ Directed → optimum set ( Nelder-Mead, simulation annealing, etc) ◮ Undirected → ” acceptable ” sets ( exhaustive or sampling design based grid search) ◮ Bayesian → joint posterior distribution TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Comparative analysis Objective: Comparative analysis of two calibration methods ◮ Bayesian: MCMC Calibration ( Rutter et at, 2009 ) ◮ Empirical : Grid Search using Latin Hypercube Sampling (GSLHS) Implementation: MILC model Comparative analysis: Quantitative and qualitative TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Methods Bayesian Empirical Method ( Rutter et al. 2009 ) ( Grid Search using LHS design ) Goal draw values from the draw sets of ” acceptable ” values for θ joint posterior h( θ | Y ) Specifics ֒ → Gibbs sampler ֒ → Latin Hypercube Sampling → appr. MH algorithm → LR test ֒ ֒ Results Distribution of (multiple draws for) θ TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD

Microsimulation Models MSM for lung cancer Comparative Analysis Calibration Results Predictive accuracy Comparative Analysis Study Design I 1. Parameters to calibrate: ◮ tumor growth ( θ 1 =m) ◮ disease progression ( θ 2 =mdiagn, θ 3 =mreg, θ 4 =mdist) 2. Input data: ◮ sample ( smpl.C N , N=5000 ) from 1980 US population of males, current smokers 3. Calibration targets, Y clbr =[Y < 60 Y 60 − 80 Y > 80 ] T : ◮ lung cancer incidence rates by age group ( 2006 SEER data ) 4. Size (total number of micro-simulations): → M Bayes = 2 · 10 10 (!!!) ( sequential ) → M Emp = 5 · 10 9 (!!!) ( parallel ) TRIPODS Workshop: Models & Machine Learning for Causal Inference Microsimulation Models in Medical Decision Making : Calibration and Predictive Accuracy text Stavroula Chrysanthopoulou, PhD