Register of Clinical Prediction Rules, Methodological Quality - PowerPoint PPT Presentation

Royal College of Surgeons in Ireland Coláiste Ríoga na Máinleá in Éirinn Register of Clinical Prediction Rules, Methodological Quality Assessment and Implementation Strategies Emma Wallace, Claire Keogh, Susan Smith, Borislav D. Dimitrov, Tom Fahey Division of Population Health Sciences

Outline • Section 1 - Development of register of Clinical Prediction Rules (CPRs) - Search strategies to identify CPRs relevant to primary care from the MEDLINE database • Section 2 - Methodological quality assessment of CPRs • Section 3 - Implementation strategies Division of Population Health Sciences

SECTION 1 Development of Register of Clinical Prediction Rules (CPRs) Division of Population Health Sciences

Defnitions • Clinical Prediction Rule – Clinical tool that quantifes the contribution of • Patient History • Physical Examination • Diagnostic Tests – Stratify patients diagnosis • Probability of having target disorder. – Outcome can be in terms of diagnosis, prognosis, referral or treatment Division of Population Health Sciences

CPR Examples 1 ) Ottawa ankle rule Division of Population Health Sciences

2 ) The Centor Score

Stages of development of CPR Figure 2. Development of a Clinical Decision Rule Step 1. Derivation Step 2. Validation Step 3. Impact Analysis Identification of factors Evidence of reproducible accuracy Evidence that rule change with predictive power physician behaviour and Narrow Validation Broad Validation improves patient outcomes and/or reduces costs Application of a rule in a Application of rule in Similar clinical setting and multiple clinical settings Population as Step 1 and varying prevalence and outcomes of disease Level of Evidence 4 3 2 1 Division of Population Health Sciences

Ottawa ankle rule: Derivation to Impact Analysis • Derived in 2 large ER departments in Canada • Validated initially in same ERs with a larger cohort of patients (narrow validation) • Validated in many diferent clinical sites with consistent results (broad validation) • Shown to decrease cost by reducing unnecessary radiographs with no increase in missed fractures (impact analysis) Division of Population Health Sciences

Aim • Develop a web-based register of CPRs for use in primary care as part of Cochrane Primary Healthcare Field

CPR register development • Development of an electronic search flter-identify CPRs relevant to primary care • Haynes flter in Clinical Queries • Broad and narrow search flters Division of Population Health Sciences

30 journals included on the register Academic Emergency Medicine Family Medicine American Family Physician Family Practice American Journal of Medicine Journal of American Medical Association Annals of Emergency Medicine Journal of the American Board of Family Medicine Annals of Family Medicine Journal of Clinical Epidemiology Annals of Internal Medicine Journal of Family Practice Annals of Medicine Journal of Internal Medicine Annual Review of Medicine Lancet Archives of Internal Medicine Medical Care BMC Family Practice Medical Decision Making British Medical Journal Medicine British Journal of General Practice New England Journal of Medicine Canadian Family Physician Public Library of Science Medicine Canadian Medical Association Journal Primary Care Cochrane Database Systematic Reviews Scandinavian Journal of Primary Health Care

Methods • Manually searched 30 journals relevant to primary care for the year 2008 (‘reference standard’) • 7 individual electronic searches of the 30 journals (each flter treated as ‘diagnostic tests’) • Test accuracy analysis: Sensitivity and specifcity • Aim: to maximise sensitivity Division of Population Health Sciences

Database Filter name Filter search string PubMed Haynes Broad Filter (predict*[tiab] OR predictive value of tests[mh] OR scor*[tiab] OR observ*[tiab] OR (HBF) observer variation[mh]) PubMed Haynes Narrow Filter (validation[tiab] OR validate[tiab]) (HNF) EBSCO host McGrath/Murphy ((predict* N3 rule* OR predict* N3 model OR predict* N3 models) OR (decision* N3 Broad Filter (MMBF) rule*) OR (TX validat*)) EBSCO host McGrath/Murphy ((predict* N3 rule* OR predict* N3 model OR predict* N3 models) OR (decision* N3 Narrow Filter rule*)) (MMNF) PubMed Teljeur/Murphy "clinical prediction" OR "clinical model*" OR "clinical score*" OR "decision rule*" OR "diagnostic accuracy" OR "diagnostic rule*" OR "diagnostic score*" OR "diagnostic Inclusion Filter 26 item value" OR "predictive outcome*" OR "predictive rule*" OR "predictive score*" OR (TMIF-26) "predictive value" OR "predictive risk*" OR "prediction outcome*" OR "prediction rule*" OR "prediction score*" OR "prediction value*" OR "prediction risk*" OR "risk assessment" OR "risk score*" OR "validation decision*" OR "validation rule*" OR "validation score*" OR (derivation AND validation) OR (sensitivity AND specificity) OR (symptoms AND signs) PubMed Teljeur/Murphy (clinical[tiab] AND predict*[tiab]) OR (clinical[tiab] AND model*[tiab] ) OR (clinical[tiab] AND score*[tiab]) OR (decision [tiab] AND rule*[tiab]) OR (derive*[tiab] Inclusion Filter 22 item AND validat*[tiab]) OR (diagnos*[tiab] AND accura*[tiab]) OR (diagnos*[tiab] AND (TMIF-22) rule*[tiab]) OR (diagnos*[tiab] AND score*[tiab]) OR (diagnos*[tiab] AND value[tiab]) OR (predict*[tiab] AND outcome*[tiab]) OR (predict*[tiab] AND rule*[tiab] OR (predict*[tiab] AND score*[tiab] ) OR (predict*[tiab] AND validat*[tiab]) OR (predict*[tiab] AND value*[tiab]) OR (risk*[tiab] AND assessment*[tiab]) OR (risk[tiab] AND score*[tiab]) OR (sensitivity[tiab] AND specificity[tiab]) OR (symptoms[tiab] AND signs[tiab]) OR (validat*[tiab] AND decision*[tiab]) OR (validat*[tiab] AND rule*[tiab]) OR (validat*[tiab] AND score*[tiab]) OR (predict*[tiab] AND risk*[tiab]) PubMed Teljeur/Murphy (allele OR amino OR animal OR apoptosis OR chromosome OR congenital OR dental OR dna OR endogenous OR endothelial OR epithelial OR mammalian OR mice OR Exclusion Filter molecule OR molecular OR mouse OR mutate OR mutation OR necrosis OR (TMEF) pathogenesis OR phosphorylation OR polymorphism OR receptor OR signal OR species OR tissue OR tumor OR tumour OR tyrosine OR vitro)

Results Manual ‘reference standard’ search retrieved 6344 articles, 41 of which were CPRs Filter name N N CPRs Sensitivity (%) Specificity (%) articles retrieved retrieved Haynes Broad Filter 1251 31 76 81 Haynes Narrow Filter 89 12 29 99 McGrath/Murphy Broad Filter 264 23 56 96 McGrath/Murphy Narrow Filter 63 16 39 99 Teljeur Murphy Inclusion Filter-26 2432 39 95 62 Teljeur/Murphy Inclusion Filter-22 693 34 83 90 Teljeur/Murphy Exclusion Filter 3589 24 59 43

MEDLINE versus the fnal search flter applied to 30 primary care journals ( 1966 – 2008 ) Year of publication

Creating the International Register 2008-1980 resulted in 252 CPRs relevant to primary care Years Average articles retrieved Average CPRs for per year primary care identifed per year* 2008 – 2000 2622 25 1999-1990 2328 3 1989-1980 2243 2 * Still awaiting inter-library loans

Clinical Domains – ICPC 2 Percentage articles retrieved for each clinical domain % %

Clinical Domains – ICPC 2 Respiratory System Respiratory System N=41 Strep throat 12 Pneumonia 8 Asthma 6 Respiratory symptom/complaint other 3 Chronic obstructive pulmonary disease 2 Cough 2 Respiratory infection other 2 Breathing problems , other 1 Infuenza 1 Respiratory disease other 1 Tonsillitis acute 1 Upper respiratory infection acute 1 Sinusitis acute/chronic 1

SECTION 2 Methodological Quality assessment of CPR register Division of Population Health Sciences

Health Research Reporting Division of Population Health Sciences

Health Research Reporting • International initiative ‘Equator’ set up in 2008 – Aim to provide resources & education enabling improved reporting and monitor progress – Develop network of reporting guidelines e.g.  CONSORT- RCTs  STARD- Diagnostic Accuracy studies  STROBE- Observational Studies  PRISMA- Systematic reviews Division of Population Health Sciences

Methodological Quality assessment of CPRs • To date Equator have not developed a reporting guideline for CPR Quality assessment • Wasson et al, 1985 , Laupacis et al, 1997 , Stiell et al 1999, McGinn et al, 2000 , Cook et al, 2010 • Diferent quality markers for derivation and validation CPR studies • Consensus to use McGinn quality checklist due to content and utility Division of Population Health Sciences

Results-preliminary • Quality assessment of a small representative sample of CPR register assessed to date - 1988 - 1998 - 2008 Division of Population Health Sciences

Results • 1988 - Derivation CPRs= 3 - Validation CPRs= 0 Division of Population Health Sciences

Results CPRs (derivation), 1988, n=3

Results • 1998 - Derivation CPRs n= 3 - Validation CPRs n= 7 Division of Population Health Sciences

Results CPRs (derivation), 1998 , n= 3

Results CPRs (validation), 1998 , n= 7

Results • 2008 - Derivation CPRs n= 18 - Validation CPRs n= 20 Division of Population Health Sciences