Brian Haynes McMaster University EBHC Workshop, 2013 The Health - PowerPoint PPT Presentation

Brian Haynes McMaster University EBHC Workshop, 2013

The Health Information Research Unit at McMaster has developed, and produces or supports, several resources that will be discussed (ACP Journal Club, Evidence Updates, DynaMed, MacPLUS FS, ACCESSSS). (IP belongs to McMaster; most services free)

Agenda (negotiate!) 1. Intros – and your most frustrating or rewarding teaching or personal experience in acquiring “current best evidence for clinical practice” 2. Favorite sources of EB info? 3. Sign up: http://plus.mcmaster.ca/ACCESSSS (suggest that you subscribe, but can use ID/PW: guestn/guestn)

To define/personalize the nature of the 1. problems in translating evidence into clinical practice To explore a hierarchy of 2. evidence resources to support clinical care decisions: To demonstrate and practice how you can 3. use/teach the pyramid to keep up to date ( “ push services ” ) and find current best ( “ pull services ” ) evidence

1. Making decisions consistently based on current best evidence ? 2. Keeping up to date with current best evidence ?

Clinicians like you usually have more than 5 questions a day that can be answered by current best evidence Green. Residents' medical information needs in clinic: are they being met? Am J Med 2000

But…  2,000 new articles every day  including 75 trials  and 11 systematic reviews Bastian, Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010

By the year 2020, 90% of clinical decisions will be supported by accurate, timely, and up-to-date clinical information and will reflect the best available evidence. IOM Roundtable on Evidence-Based Medicine This can ’ t happen without excellent connections between best evidence and decisions for and by individual patients.

100% r = -0.54 Choudhry, Fletcher and Soumerai, . .. Ann Intern Med 2005;142:260-73 p<0.001 ... . . .... . ... knowledge 94% of 62 studies found decreasing competence for at 50% ... of current least some tasks, with increasing physician age. .... ... best care .. .... 0% years since graduation

McMaster vs U of Toronto 100% knowledge 50% of current best care 0% years since From Shin et al, graduation CMAJ 1993

The Slippery Slope 100% . .. knowledge ... 50% of best . . .... . ... evidence ... .... ... .. .... 0% You could be the first generation to stay years since on top and avoid the slippery slope… graduation

… 17 to 20 years

To define the nature of the problems in 1. translating evidence into clinical practice To explore a hierarchy of 2. evidence resources to support clinical care decisions: Finding current best evidence To demonstrate how you can use the pyramid 3. to keep up to date ( “ push services ” ) and find is becoming much easier. current best evidence *fast* ( “ pull services ” )

Personalized EBM “ push ” Alerts? eg EvidenceUpdates, ACPJournalWise EBM “ pull ” Resources? eg UTD, DynaMed, Best Practice, ACP PIER EBM Federated Resources? ACCESSSS eg TRIP,

Many evidence-based resources help you answer your questions

But no single resource provides all answers or is sufficiently updated

shows results in a pyramid of resources in a few seconds

showing results in a pyramid of resources in a few seconds

Summaries Pre-appraised research Non-preappraised research

Summaries • are E-B clinical textbooks and E-B guidelines • integrate best evidence for individual care topics But which • provide actionable summary to recommendations choose?

Percentage of 60 ICD-10 codes covered by each summary

Average time of updating of 60 topics Timeliness of updates of 10 online evidence-based texts (randomly selected) as of July 2011 Ranging from 3.5 to 29 months Error bars: 95% Confidence Interval

Rank order of 10 Online Summaries Summary Timeliness Breadth Quality DynaMed 1 3 2 UpToDate 5 1 2 Micromedex 2 8 2 Best Practice 3 4 7 Essential Evidence Plus 7 7 2 First Consult 9 5 2 Medscape Reference 6 2 9 Clinical Evidence 8 10 1 ACP PIER 4 9 7 PEPID N/A 6 10

No answer? or more recent evidence?

Summaries Pre-appraised Research

synopses of systematic reviews systematic reviews synopses of studies

 only a tiny proportion of all research is “ ready for application ”  only a tiny fraction of the “ ready ” research is “ relevant ” for a given clinician  only a tiny proportion of the “ relevant ” research for a given clinician is “ interesting ” in the sense of being something new, important, and actionable.

Critical Appraisal Filters Clinical Relevance Filters ~20 articles + 35,000 articles per clinician screened per year Up to 99.9 % ‘Noise’ Reduction ~3300 articles per year Is ‘ information overload ’ no longer a valid excuse for ignorance? Haynes, B. The Best New Evidence for Patient Care. Ann Intern Med. 2008;148(10):JC3-2

Al-Khatib SM, Thomas L, Wallentin L, et al. Outcomes of apixaban vs. warfarin by type and duration of atrial fibrillation: results from the ARISTOTLE trial. Eur Heart J. 2013 Apr 17. 4 internists 4 neurologists 4 cardiologists 4 hematologists

DISCIPLINE RELEVANCE NEW? Internal Medicine 6/7 6/7 Neurology 6/7 6/7 Cardiology 5/7 6/7 Hematology 5/7 5/7

Internists Neurologists Cardiologists 7/7 6/5 7/6 Hematologists

is published in evidence- based journals and resources

• is continuously sent to you through MacPLUS, ACCESSSS, ACPJW, EvidenceUPDATES alerts http://plus.mcmaster.ca/EvidenceUpdates

Methods • Select 4 leading evidence-based texts • Select 200 topics that are common to all • Identify date of most recent update for each topic in each text • Identify each article in MacPLUS that is more recent than text update • Compare conclusions of MacPLUS studies with conclusions in text

How often does MacPLUS give new and different evidence than Summaries? Best Practice DynaMed PIER UpToDate No. (%) of 200 topics with 119 (60%) 46 (23%) 109 (55%) 104 (52%) potential for updates † *Based on articles in MacPLUS since most recent topic update which have conclusions that differ from topic conclusion(s). † 1 -way ANOVA p<0.01 across texts

Summaries Pre-appraised Research Non-preappraised Research

Why search last? • Studies are not critically appraised • Searches yield large outputs with few relevant studies in the first pages • Current evidence is diluted and more difficult to find, but less so if you use filters

Use Clinical Queries filters for questions of • Therapy • Diagnosis • Etiology • Prognosis • Prediction Guides

E.g. CQ therapy filters Sensitivity/ PubMed Equivalent Specificity ((clinical[Title/Abstract] AND trial[Title/Abstract]) OR clinical trials[MeSH Terms] OR clinical broad trial[Publication Type] OR 99% / 70% filter random*[Title/Abstract] OR random allocation[MeSH Terms] OR therapeutic use[MeSH Subheading]) (randomized controlled trial[Publication Type] narrow OR (randomized[Title/Abstract] AND 93% / 97% filter controlled[Title/Abstract] AND trial[Title/Abstract]))

How many search terms to use? Patients Intervention Comparison Outcomes

Using Keep it simple! Patient + Intervention E.g. COPD mucolytics Works better for Summaries and Pre-appraised research

Combining more terms from your PICO elements COPD mucolytics (exacerbation OR mortality) Works better for unappraised evidence

looks for you in all resources and displays results in a pyramid

PICO question from the audience • …

What is the best current evidence? Did I miss any important evidence with my search? Is there any way I could have retrieved less “junk”?

Alternatives TRIP Database – EB search, guidelines, patient info, fulltext links, videos, news SUMSearch – DARE at highest level STAT!Ref Google Corporate collections – professional organizations; commercial publishers

Additional slides

Search strategies in PubMed Dynamic Initial question Previous knowledge MeSH Thesaurus Trial and Error Search terms Combination Boolean operators = OR , AND Search strategy = PICO query AND filters AND additional strategy P – Patient, Population Search forward from: Methodological filters: I – Intervention, Exposure … Related Articles in Ex: Clinical Queries C – Controls, Comparators Pubmed AND Screen the O – Outomes Content filters (topic,specialty) bibliography of relevant articles AND T – Type of question, design Agoritsas & al. Sensitivity and Predictive Value of 15 PubMed Search Strategies to Answer Clinical Questions Rated Against Full Systematic Reviews. JMIR, 2012.