Using Evidence Maps in SRDR ™ to Efficiently Plan Systematic Reviews Center for Evidence-Based Medicine School of Public Health Brown University Ethan Balk, MD MPH Bryant Smith, MPH CPH SRDR is being developed and maintained by the Brown EPC under contract with the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-2015-00002-I).
Evidence Maps A Brief Overview
Evidence Map Definition (Katz 2003) “A means of systematically organizing the base of evidence pertaining to a broad topic… so that the distribution, breadth, depth, methodology, and overall quality of pertinent evidence is characterized and made readily accessible.” Katz et al. Alternative Therapies. 2003;9(4):22-30. 3
Evidence Map Definition (Hetrick 2010) “ Less exhaustive yet systematic and replicable methodology that allows an understanding of the extent and distribution of evidence in a broad clinical area , highlighting both what is known and where gaps in evidence exist.” Hetrick et al. J Eval Clin Pract. 2010;16(6):1025-30. 4
Evidence Map Definition (Miake-Lye 2016) “A systematic search of a broad field to identify gaps in knowledge and/or future research needs that presents results in a user-friendly format, often a… searchable database. ” Miake-Lye et al. Systematic Reviews 2016; 5:28. 5
“Gap Map” 3ie’s evidence map for education. 6 http://www.scidev.net/global/data/news/gap-maps-holes-development-evidence.html
Features of Evidence Maps • Systematic process (eg, documented search strategy, inclusion criteria) • Review of evidence of (usually) a broad field to – Describe characteristics of existing evidence – Identify research gaps and future research needs • Briefly summarizes included studies • Provides data on (for example) – PICO-D – Sample size – ± Basic study results or conclusions – (Not detailed study results; Usually not study quality) 7
Bubble Plot Evidence Map of Mindfulness 8 Hempel et al. http://www.hsrd.research.va.gov/publications/esp/cam_mindfulness.cfm
Scoping Study (Arksey 2005, O’Brien 2016) • Rapidly map key concepts underpinning a research area, and sources and types of evidence available Extent, range, nature of research activity – • Determine value of undertaking a full systematic review Eg, does literature exist? have SRs been done? – • Summarize and disseminate research findings • Identify research gaps in the existing literature Does not address gaps due to poor quality research – Arksey et al. Int J Soc Res Method 2005;8(1):19-32. 9 O’Brien et al. Health Serv Res 2016;16:305.
Basic Steps • Mostly follow standard systematic review process – Identify research question – Develop protocol – Define eligibility criteria (eg, PICO-D) – Systematically search literature – Screen (select) citations (± full text articles) – Extract pertinent data – Organize, summarize, report, map results 10
Map of Studies Decision Support Tools Trikalinos et al. https://www.effectivehealthcare.ahrq.gov/ehc/products/529/2029/cancer-decision-support-tools-report-141223.pdf 11 (black = included in a systematic review)
How Differs from Systematic Review • Usually covers a broader field / range of questions than a SR • Eligibility criteria may be more expansive (inclusive) than a SR • May be based on citations/abstracts only (not full text) → incomplete extraction of outcome list and analyses • Omits details of study methodology, eligibility criteria, participant and intervention characteristics, etc. • Omits study quality / risk of bias assessment • Usually omits extraction of (detailed) study results or conclusions – May include directionality, statistical significance, etc. • Less time/resource intensive than SR of same topic/scope • Categorizes existing literature more than summarize findings 12
Example Evidence Map Omega-3 fatty acids (n-3 FA) and Cardiovascular disease (CVD) • – Used to describe breadth and types of evidence to scope a systematic review update What is the efficacy or association of n-3 FA (EPA, DHA, EPA+DHA, • DPA, SDA, ALA, or total n-3 FA) exposures in reducing CVD outcomes (incident CVD events, including all-cause death, CVD death, nonfatal CVD events, new diagnosis of CVD, peripheral vascular disease, CHF, major arrhythmias, and hypertension diagnosis) and specific CVD risk factors (BP, key plasma lipids)? – Without known CVD (primary prevention) – At high risk for CVD (primary prevention) – With known CVD (secondary prevention) – Other subpopulations 13 – Interactions with CVD and diabetes drugs
Omega-3 Fatty Acids and CVD • Literature search (and existing SRs) yielded 462 articles that met basic eligibility criteria: – Adults (healthy, at CVD risk, with CVD) – Comparison of n-3 FA intake (as food or supplement) with other n- 3 FA or other level of intake (including “placebo”) – Association of baseline n-3 FA intake or biomarker with future CVD event – CVD events (1 year follow-up), BP/Cholesterol (4 week follow-up) – RCTs (for comparisons) Prospective longitudinal observational studies • Needed to constrain literature review given resource and time constraints, but also to create a searchable database for others to use for future reviews 14
A Priori Approach to Limit Scope • Goal was to capture “best”, “most generalizable” evidence for each specific topic to address most important key questions • Planned to select largest, longest follow-up, most generalizable studies for each specific topic – Wanted to prioritize larger studies with longer follow-up duration or that reported interaction analyses of interest 15
Evidence Map Design • Designed evidence map to capture (from abstract) – Sample size (N, N taking n-3 FA) – Study design – Population type (healthy, at risk, CVD) – Specific n-3 FA(s) (“fish oil” [EPA, DHA, DPA], SDA, ALA, total, unclear) – Biomarker study – Interaction between n-3 FA and drug – Subgroup or predictor analysis – Study duration – Specific outcome(s) • Based on these data, selected “most relevant” studies à 61 RCTs (82 articles) & 37 observational (65 articles) 16
Conducting the Evidence Map in SRDR: Introduction to the Systematic Review Data Repository (SRDR) 17
Systematic Review Data Repository (SRDR) • Online, open access archive of searchable systematic review data – Key questions – Extraction forms – Data extracted from included studies • Collaborative workspace for completing the data extraction phase of systematic reviews – Powerful, web-based tools – Flexible options for data extraction – Platform from which a complete systematic review tool is being developed. 18
System Status • System has been in development for 6+ years • Launched on June 20 th 2012 • Hosted on the Amazon Cloud (via AHRQ link) • Designed and maintained by the SRDR Team lead by Dr. Joseph Lau at Brown University • SRDR is committed to a policy of open access and data sharing. All systematic review data published to the SRDR archive are publically available under the terms of a Creative Commons license 19
Value of SRDR to Systematic Reviewers • Open access – Free registration, access and support • Useful systematic review tools – Data extraction – Data management – Project management • Interaction with the systematic review community – Contribute data to the community – Use data contributed by other review teams • Participate in collaborative development of SRDR 20
Value of SRDR to Guideline Developers • Access to high quality data used in SRs • Minimize need to repeat expensive process, save time, money, and resources – Particularly when updating reviews and guidelines • Collaborate in development of a worldwide resource – Allow developers to benefit from others’ experience and expertise 21
SRDR Features • Free to Use • Searchable repository of published SR data • Community commenting on published data • Collaborative, on-line data extraction • Customizable data extraction forms • Secure database • Export data to MS Excel (.xlsx) and CSV files • Import data from MS Excel (.xlsx) • Supports multiple data abstraction 22
Creating an Evidence Map in SRDR 23
Potential Benefits of Using SRDR to Create an Evidence Map • Customize electronic extraction forms • Work collaboratively collecting data from studies using the extraction forms • Export complete evidence map data-sets to spreadsheet files (to allow analysis) • Share evidence map datasets with other guideline developers by publishing them (no cost) to the publically accessible SRDR published projects page. 24
Steps to Create an Evidence Map Create a project in SRDR 1 Enter general project information and key question(s) 2 Build an SRDR extraction form 3 Add eligible study records to the SRDR project 4 Extract study data into extraction forms for each project 5 Review and publish the project to SRDR’s published projects page 6 25
http://srdr.ahrq.gov 26
Using the SRDR Interface: Creating a New Project Create a project in SRDR 1 Enter general project information and key question(s) 2 Build an SRDR extraction form 3 Add eligible study records to the SRDR project 4 Extract study data into extraction forms for each 5 project Review and publish the project to SRDR’s published projects page 6 27
Recommend
More recommend
