Services Research Helen Burstin, MD, MPH, FACP Chief Scientific - PowerPoint PPT Presentation

The Growing Intersection Between Data Journalism and Health Services Research Helen Burstin, MD, MPH, FACP Chief Scientific Officer, NQF June 26, 2016

NQF Mission The National Quality Forum leads national collaboration to improve health and healthcare quality through measurement.

Definitions of HSR  AHRQ: Examines how people get access to health care, how much care costs, and what happens to patients as a result of this care.  Academy Health: HSR is the science/field of study that asks: ▫ What works? ▫ For whom? ▫ At what cost? ▫ Under what circumstances?  Greater recognition that “research” can take many forms beyond classic academic paradigm 3

Data Journalism  “Real data journalism comes down to a couple of predilections: a tendency to look for what is categorizable, quantifiable and comparable in any news topic and a conviction that technology, properly applied to these aspects, can tell us something about the story that is both worth knowing and unknowable in any other way.“  “Great data projects don't generally begin with great data sets. They begin with great questions and the desire to find the hardest evidence available to answer those questions.“ » http://www.poynter.org/author/troy-thibodeaux/ 4

National Quality Strategy on 5

Progress: Hospital Acquired Infections 6

NQF Evaluation Criteria  Importance to measure and report ▫ What is the level of evidence for the measures? ▫ Is there an opportunity for improvement?  Scientific acceptability of the measurement properties ▫ What is the reliability and validity of the measure?  Usability and Use ▫ Can potential audiences use performance results for both accountability and performance improvement?  Feasibility ▫ Can the measure be implemented without undue burden, capture with electronic data/EHRs?

Importance of Measure Testing Reliability testing  Demonstrates the measure data elements are repeatable, producing the same results a high proportion of the time when assessed in the same population in the same time period and/or that the measure score is precise. Validity testing  Demonstrates that the measure data elements are correct and/or the measure score correctly reflects the quality of care provided, adequately identifying differences in quality. 8

Threats to Validity  Measure focus is not a relevant outcome of healthcare or not strongly linked to a relevant outcome  An unreliable measure cannot be valid  Patients inappropriately excluded from measurement  Differences in patient mix for outcome and resource use measures not adjusted  Measure scores generated with multiple data sources/methods  Systematic missing or “incorrect” data (unintentional or intentional) 9

Rigor & Transparency 10

Measurement: Drive Selection and Change Fung et al. adaptation of Berwick, James and Coye’s Conceptual Framework 11

NQF: Transparency  Transparency is the extent to which performance results about identifiable, accountable entities are disclosed and available outside of the organizations or practices whose performance is measured.  Maximal transparency is achieved with public reporting defined as making comparative performance results about identifiable, accountable entities to the public at large (generally on a public website).  At a minimum, the data on performance results about identifiable, accountable entities are available to the public.  The capability to verify the performance results adds substantially to transparency. 12

Requirements for Transparency of Endorsed Measures  Use in Public Reporting: ▫ Disclosure of performance results to the public at large ▫ Performance results are used in at least one accountability application within three years after initial endorsement and are publicly reported within six years after initial endorsement (or the data on performance results are available). ▫ If not in use at the time of initial endorsement, then a credible plan for implementation within the specified timeframes is provided. 13

14

Consumer Reports, 2011  It's sad but true that it's easier to make an informed choice about cars and refrigerators than it is about health-care providers . To help correct that problem, we’ve teamed with the Society of Thoracic Surgeons (STS) to publish ratings of heart-surgery groups based on their performance data for bypass surgery.  Using this information, consumers can see how surgical groups compare with national benchmarks for overall performance, survival, complications, and other measures.  No organization is better poised to provide this snapshot of surgical outcomes than the STS , a nonprofit organization that represents some 5,400 surgeons worldwide who operate on the thorax, or chest. “ 15

Improvement and Potential for Unintended Consequences  Progress toward achieving the goal of high-quality, efficient healthcare for individuals or populations is demonstrated.  The benefits of the performance measure in facilitating progress toward achieving high-quality, efficient healthcare for individuals or populations outweigh evidence of unintended negative consequences to individuals or populations (if such evidence exists) . 16

Path Forward: Data and Measurement  Push for greater transparency of the best possible data for quality measurement and reporting - status quo is not acceptable  Use standardized measures vetted through multistakeholder process whenever possible to avoid focus on methods over end-goal  Recognize that HSR data and methods must evolve to meet this brave new world of apps and ready access to data 17

Data Use for a Learning Health System  Utilization Measures Onpoint Health Analytics Provider Registry  Expenditure Measures  Unit Costs Clinical  Quality Measures Registry Claims data from APCD  Patient Experience Measures Data extracts BRFSS Data  Comparative Evaluation  Practice Profiles • Process data sets • Check data quality  HSA Profiles CAHPS Data • Address data gaps  PCMH + CHT Evaluation • Link data sets • Analytics  Opioid Program Evaluation • Reporting Corrections data  Associations & Predictive Models  Planning, Coordination, Quality Other?  Performance Payments Craig Jones, Department of Vermont Health Access 18

HSR and Data Journalism  As data are more available, the public and reporters who work on their behalf will use them more  Collaborate and maintain open dialogue to drive toward best possible information for patients and providers  When a reporter calls you for help with data and research: ▫ Be collaborative ▫ Be open to new ideas ▫ Be willing to share methods and ideas 19

Purpose of Measurement: Improve Quality & Safety 20

Helen Burstin, MD, MPH, FACP hburstin@qualityforum.org @HelenBurstin 21