Missing Data in Randomised Trials Overview and Strategies James R. - PowerPoint PPT Presentation

Randomised Controlled Trials in the Social Sciences Conference Missing Data in Randomised Trials — Overview and Strategies James R. Carpenter London School of Hygiene & Tropical Medicine & MRC CTU at UCL james.carpenter@lshtm.ac.uk www.missingdata.org.uk September 7, 2018 Missing Data in Clinical Trials – 1 / 45

Acknowledgements Overview Harvey Goldstein (LSHTM) • Acknowledgements Rachael Hughes (Bristol) Mike Kenward (LSHTM) • Outline Towards a principled Geert Molenberghs (Limburgs University, Belgium) approach Mike Kenward, James Roger (LSHTM) Critique of common methods Sara Schroter (BMJ, London) Missing At Random Michael Spratt (Bristol) Multiple Imputation under Jonathan Sterne (Bristol) Missing At Random Stijn Vansteelandt (Ghent University, Belgium) MI: example Multiple Imputation under Tim Morris, Ian White (MRC CTU at UCL) Missing Not At Random Discussion Background to this session is in Chs 1 & 2 of ‘Missing data in clinical trials — a practical guide’ (joint with Mike Kenward), commissioned by the NHS, available free on-line at www.missingdata.org.uk. Missing Data in Clinical Trials – 2 / 45

Outline 1. Missing data in trials— the need for a principled approach Overview • Acknowledgements 2. Completers analysis • Outline Towards a principled 3. Imputation of simple mean approach 4. Imputation of regression mean Critique of common methods 5. Last Observation Carried Forward Missing At Random 6. Multiple Imputation Multiple Imputation under Missing At Random • assuming data are Missing At Random MI: example • assuming data are Missing Not At Random Multiple Imputation under Missing Not At Random 7. Discussion Discussion Missing Data in Clinical Trials – 3 / 45

Why is this necessary? Missing data are common. Overview Towards a principled approach However, they are usually inadequately handled in both epidemiological • Why is this necessary? • Further... and experimental research. • The E9 guideline, 1999 • Study validity and sensible analysis For example, [14] reviewed 71 recently published BMJ, JAMA, Lancet • Why there can be no and NEJM papers. universal method: • Example: Trial of training to improve the quality of peer review • 89% had partly missing outcome data. • Key points for analysis • In 37 trials with repeated outcome measures, 46% performed • Towards a systematic approach complete case analysis. • A systematic approach • Only 21% reported sensitivity analysis. Critique of common methods Missing At Random Unfortunately, an update in 2014 [1] showed relatively little had changed, Multiple Imputation under although Multiple Imputation [12] was much more commonly used. Missing At Random MI: example Multiple Imputation under Missing Not At Random Discussion Missing Data in Clinical Trials – 4 / 45

Further... Overview CONSORT guidelines state that the number of patients with missing data Towards a principled should be reported by treatment arm. approach • Why is this necessary? • Further... But [5] estimate that 65% of studies in PubMed journals do not report the • The E9 guideline, 1999 handling of missing data. • Study validity and sensible analysis • Why there can be no [14] identified serious weaknesses in the description of missing data and universal method: • Example: Trial of the methodology adopted. training to improve the quality of peer review • Key points for analysis • Towards a systematic approach • A systematic approach Critique of common methods Missing At Random Multiple Imputation under Missing At Random MI: example Multiple Imputation under Missing Not At Random Discussion Missing Data in Clinical Trials – 5 / 45

The E9 guideline, 1999 • Missing data are a potential source of bias Overview Towards a principled • Avoid if possible (!) approach • Why is this necessary? • With missing data, a trial may still be regarded as valid if the methods • Further... are sensible , and preferably predefined • The E9 guideline, 1999 • Study validity and • There can be no universally applicable method of handling missing sensible analysis • Why there can be no data universal method: • Example: Trial of • The sensitivity of conclusions to methods should thus be investigated, training to improve the quality of peer review particularly if there are a large number of missing observations • Key points for analysis • Towards a systematic approach Guidelines downloadable from www.ich.org • A systematic approach Critique of common The question is, how do we apply these principles in practice? methods Missing At Random Multiple Imputation under Missing At Random MI: example Multiple Imputation under Missing Not At Random Discussion Missing Data in Clinical Trials – 6 / 45

Study validity and sensible analysis Missing data are observations we intended to make but did not. Overview Towards a principled approach The sampling process involves both the selection of the units, and the • Why is this necessary? • Further... process by which observations become missing — the missingness • The E9 guideline, 1999 mechanism . • Study validity and sensible analysis • Why there can be no Thus for sensible inference, we need to take account of the missingness universal method: • Example: Trial of mechanism training to improve the quality of peer review • Key points for analysis By sensible we mean: • Towards a systematic approach • A systematic approach • Frequentist: nominal properties hold. Eg: Critique of common Estimators consistent; confidence intervals attain nominal coverage. methods • Bayesian: Missing At Random Multiple Imputation under Posterior distribution is unbiased, correctly reflects loss of information Missing At Random due to missingness mechanism. MI: example Multiple Imputation under Missing Not At Random Discussion Missing Data in Clinical Trials – 7 / 45

Why there can be no universal method: In contrast with the sampling process, which is usually known, the Overview Towards a principled missingness mechanism is usually unknown. approach • Why is this necessary? • Further... The data alone cannot usually definitively tell us the sampling process. • The E9 guideline, 1999 • Study validity and sensible analysis Likewise, the missingness pattern, and its relationship to the • Why there can be no observations, cannot identify the missingness mechanism. universal method: • Example: Trial of training to improve the quality of peer review With missing data, extra assumptions are thus required for analysis to • Key points for analysis proceed. • Towards a systematic approach • A systematic approach The validity of these assumptions cannot be determined from the data at Critique of common methods hand. Missing At Random Assessing the sensitivity of the conclusions to the assumptions should Multiple Imputation under Missing At Random therefore play a central role. MI: example Multiple Imputation under Missing Not At Random Discussion Missing Data in Clinical Trials – 8 / 45

Example: Trial of training to improve the quality of peer review Overview The graph below shows selected results from a RCT of training to Towards a principled improve the quality of peer review of medical articles [10]. Background approach • Why is this necessary? details are given on your practical sheet. • Further... • The E9 guideline, 1999 • Study validity and no training self−taught package sensible analysis 5 • Why there can be no universal method: • Example: Trial of training to improve the Second paper mean RQI 4 quality of peer review • Key points for analysis • Towards a systematic approach 3 • A systematic approach Critique of common methods Missing At Random 2 Multiple Imputation under Missing At Random MI: example 1 Multiple Imputation under 1 2 3 4 5 1 2 3 4 5 Missing Not At Random First (baseline) paper mean RQI Graphs by Training package Discussion Missing Data in Clinical Trials – 9 / 45

Key points for analysis • the question (i.e. the hypothesis under investigation) — missing data Overview Towards a principled usually does not change this; approach • Why is this necessary? • the information in the observed data, and • Further... • the reason for missing data. • The E9 guideline, 1999 • Study validity and sensible analysis • Why there can be no We will consider the impact of various assumptions about the missing universal method: • Example: Trial of data. training to improve the quality of peer review • Key points for analysis Importantly, the data themselves do not tell us which assumptions are • Towards a systematic approach true. • A systematic approach Critique of common We therefore want to explore whether our conclusions are robust to a methods Missing At Random range of plausible assumptions about the distribution of the missing Multiple Imputation under values. Missing At Random MI: example Note, we can’t get back the missing values themselves! Multiple Imputation under Missing Not At Random Discussion Missing Data in Clinical Trials – 10 / 45