Welcome PCORI Evaluation Group! First Meeting December 13, 2013 - PowerPoint PPT Presentation

Welcome PCORI Evaluation Group! First Meeting December 13, 2013

Objective for Today: Development of Our Evaluation Framework Evaluation Questions Specify PCORI’s Goals Identify and Frame Develop Measures  Usefulness Criteria Prioritize  Uptake and Implementation Develop a set of  Influence evaluation questions for us to pursue in subsequent meetings to determine appropriate methods, approaches, and mechanisms PEG Meeting, December 13, 2013 2

Agenda for Today Welcome and Introductions Overview of PCORI Logic Model Identify Evaluation Questions Discuss Measuring PCORI’s Goals  Usefulness  Use  Influence Prioritize Evaluation Questions Discuss Approaches to Evaluating High Priorities PEG Meeting, December 13, 2013 3

Agenda Item: Welcome and Introductions PEG Meeting, December 13, 2013 4

About This Task Force Advisory Capacity Relation to PCORI Board, MC, Other Panels, etc. Transparency Conflict-of-Interest Non-Disclosure PEG Meeting, December 13, 2013 5

The PCORI Evaluation Group From Our Board of Governors External  Gail Hunt National Alliance for Caregiving  Kimberly Bailey President and CEO Families USA  Bob Jesse Research Director and Director of Health System Reform Department of Veterans Affairs PCORI Patient Engagement Advisory Panel Member Principal Deputy Under Secretary for Health   Claire Brindis Bob Zwolak University of California, San Francisco Dartmouth-Hitchcock Medical Center, Vascular Surgeon Dartmouth Medical School, Professor of Surgery Caldwell B. Esselstyn Chair in Health Policy Director, Philip R. Lee Institute for Health Policy Studies Veterans Administration Medical Center Chief of Surgery and Director of the Non-invasive Vascular Laboratory Professor of Pediatrics and Health Policy From Our Methodology Committee  Jack Fowler University of Massachusetts Boston  Naomi Aronson Senior Research Fellow Blue Cross and Blue Shield Center for Survey Research Executive Director, Association Technology Evaluation Center   Beverly Parsons Mike Lauer InSites National Heart, Lung, and Blood Institute Executive Director Director, Division of Cardiovascular Sciences American Evaluation Association  President-elect Robin Newhouse University of Maryland School of Nursing Chair and Associate Professor, Organizational Systems and Adult Health PCORI Methodology Committee Chair 6

PCORI Staff On the PEG Supporting the PEG  Laura Forsythe (Science)  Olumide Adeleye (Science)  Lori Frank (Science)  Emma Djabali (Science)  Michele Orza  Pavan Jagannathan (Executive Director’s Office) (Executive Director’s Office)  Suzanne Schrandt (Engagement)  Katie Jones (Operations)  Kristen Konopka (Engagement)  Katie Rader (Science)  Victoria Szydlowski (Science) PEG Meeting, December 13, 2013 7

Our Evaluation Activities Our objective is to produce information that is useful to us and others to improve our work and advance the science and practice of Patient- Centered Outcomes Research We plan to conduct this work in a manner consistent with our values and methods – rigorous, focused on and engaging of stakeholders, efficient We are committed to sharing and using this information PEG Meeting, December 13, 2013 8

Evaluation Framework: Our Questions and How We Will Answer Them Evaluation Metrics/Indic Methods Sources Questions ators What do For each What From where PCOR and question, approach will will we get the PCORI what are we we take to data to stakeholders measuring answering answer this want/need to and how will this question? question? know? we measure it? PEG Meeting, December 13, 2013 9

Developing Our Evaluation Framework Guidance from PCORI's Methodology Report PCORI’s Translation Framework for choosing research designs/methods: Keep the question and the methodology separate Focus on clarifying tradeoffs Place individual studies in the context of a program Have the choice of study design take into account state-of-the-art methodology PEG Meeting, December 13, 2013 10

Agenda Item: Overview of PCORI Logic Model PEG Meeting, December 13, 2013 11

Framing A Logic Model: A program planning, implementation, evaluation, and communication tool 2010 – 2013 2014 – 2016 2017 – 2019 2020 – 2022 Cycle Yearly GAO First GAO GAO 8-year Second GAO Congressional 5-year Review Review 5-year Review Financial Audits Oversight and Evaluation Congressional inquiries may occur at any time PCORI Implementing Building Implementing Results Emphasis or Results Implementing Results Impact Theme Primary Inputs Process Outputs Outcomes Evaluation Process Outputs Outcomes Impact Metrics Processes established Overall effectiveness of activities Key Words from Research priorities and projects Use by health care decision-makers GAO Review Objective and credible information Reducing practice variation and disparities Mandate in Transparent process Effect on innovation and health economy Our Legislation Dissemination and training activities Use by public and private payers Data networks PEG Meeting, December 13, 2013 12

Our Strategic Framework – Mission/Vision IMPACT Why We Do It (Mission/Vision) Better Informed Health Decisions Improved Health Outcomes Better Health Care PEG Meeting, December 13, 2013 13

Our Strategic Framework – Goals GOALS IMPACT What We Why We Do It Accomplish (Outcomes) Increase Better Information Informed Health Decisions Improved Health Speed Implementation Outcomes Better Health Care Influence Research PEG Meeting, December 13, 2013 14

Our Strategic Framework – Strategic Imperatives STRATEGIC GOALS IMPACT IMPERATIVES How We Create (Inputs, Processes, Activities) What We Why We Do It Accomplish Engagement Increase Methods Information Better Informed Health Decisions Improved Speed Health Research Implementation Outcomes Better Health Care Influence Research Dissemination Infrastructure PEG Meeting, December 13, 2013 15

Our Five Strategic Imperatives To increase information, speed implementation, and influence research, we: Engage patients, caregivers, and all other stakeholders in our entire research process from topic generation to dissemination and implementation of results. Develop and promote rigorous Patient-Centered Outcomes Research methods , standards, and best practices. Fund a comprehensive agenda of high quality Patient-Centered Outcomes Research and evaluate its impact. Disseminate Patient-Centered Outcomes Research to all stakeholders and support its uptake and implementation. Promote and facilitate the development of a sustainable infrastructure for conducting patient-centered outcomes research. PEG Meeting, December 13, 2013 16

Our Strategic Framework/High-level Logic Model STRATEGIC OUTPUTS GOALS IMPACT IMPERATIVES What We Why We Do It How We Create What We Create Accomplish Skilled Patient-Centered Engagement Outcomes Research Community Methods Patient-Centered Outcomes Research Methods Increase Better Information Informed Health Decisions Improved Portfolio of Patient-Centered Research Health Speed Outcomes Research Studies Implementation Outcomes Better Health Care Communication and Influence Dissemination Activities Dissemination Research Patient-Centered Research Networks Infrastructure PEG Meeting, December 13, 2013 17

2014 RESEARCH EXPECTED OUTPUTS GOALS ACTIVITIES Strategic Priority: Fund High Substantially Impact Research Skilled PCOR Community increase the quantity, quality, • Expanded range of funders • Continue refining topic and timeliness of involved in PCOR generation, prioritization, and useful, selection processes trustworthy • Continue refining merit review information to process support health • Fund multiple cycles of decisions PCOR Methods research via broad and targeted PFAs • Enhanced methods for: • Increase funding for focused Speed the o Research and targeted topics implementation prioritization and use of o Merit review Strategic Priority: Carefully patient-centered Manage Research Portfolio outcomes research • Continue refining funding Portfolio of PCOR Studies evidence application and contracting • Agenda of high priority processes topics • Continue implementing • Portfolio of studies that portfolio planning, are: management, and evaluation Influence clinical o High impact and health care Strategic Priority: Partner o Useful research funded o On-track With Other Funders by others to be more patient- centered • Co-fund and co-sponsor studies with AHRQ, NIH, VA, and others 18