NYU School of Medicine Measuring fidelity in HealthyHearts NYC: A complex intervention using practice facilitation in primary care Presented by Carolyn Berry, PhD December 10, 2018 Funding: Agency for Healthcare Research and Quality (1R18HS023922) The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality
HealthyHearts NYC (HHNYC) is a partnership between NYU School of Medicine, the New York City Department of Health and Mental Hygiene, Primary Care Information Project (PCIP), and the Community Health Care Association of New York State (CHCANYS) 1. Healthy Hearts in the Heartland (Midwest Cooperative) 2. HealthyHearts NYC (New York City Cooperative) 3. Heart Health Now! (North Carolina Cooperative) 4. Healthy Hearts Northwest (Northwest Cooperative) 5. Healthy Hearts for Oklahoma (Oklahoma Cooperative) 6. Evidence Now Southwest (Southwest Cooperative) 7. Heart of Virginia Healthcare (Virginia Cooperative) 8. Evaluating System Change to Advance Learning and Take Evidence to Scale (ESCALATES) (National Evaluator) 2
Study design Randomized controlled stepped wedge enrollment (4 waves) PCIP’ s small independent practices (SIPs) (<10 FTE) (N= 255 sites) Main outcomes: Million Hearts ABCS ( A spirin, B lood pressure, C holesterol and S moking) outcome measures Mixed methods: EHR data, Salesforce data, surveys, qualitative interviews, site observations 3
EvidenceNOW intervention 4
NYC PCIP Intervention protocol 16 Practice facilitators (PF) – Employed by NYC Department of Health and Mental Hygiene PCIP – Worked with 3-23 (average 19) practices over the duration of the project – Most had previous PF experience and experience with these particular SIPs Fairly prescriptive intervention protocol – 13 in-person visits – Complete assessment after every in-person visit – 39 item task list: introductory and ABCS specific tasks – Delivery of 27 Chronic Care Model (CCM) components – Duration of visit and order of tasks were not specified All information (Assessment, Task list, Duration) documented by PFs in Salesforce.com. All entered information/data reviewed by PF manager 5
ABCD Task list Task List Examples check HHNYC quality measures accuracy Introduction (7 tasks) ensure aspirin clinical decision support alert is working Aspirin (5 tasks) discuss the most common causes of inaccurate blood Blood Pressure pressure measurements and ways to improve (12 tasks) accuracy reinforce the use of statin clinical decision aids Cholesterol (7 tasks) adopt a protocol to improve quit rates Smoking Cessation (8 tasks)
Content: CCM practice changes CCM Examples • Order set usage : Train and assist site to use order sets Clinical decision support (9) • Clinical decision support usage : Train and assist site to implement alerts for ABCS focus areas • Dashboard review: Train and assist site to define Clinical information systems (3) benchmarks, run and review dashboards monthly • Patient list/registry usage : Train and assist site to Delivery system redesign (8) generate patient lists/registries monthly to identify high risk patients • Lab order and review: Train and assist site to implement new workflow that supports timely review of labs to preplan visits • Patient self-management goals : Train and assist site to Self-management support (7) set self-management goals and provide links to community based self-management programs
Assessment of fidelity Implementation science literature suggests implementation fidelity moderates intervention outcomes; the more fidelity, the more positive the outcomes - Most studies of complex interventions like practice facilitation use only one measure of fidelity - Few existing tools or models apply to practice facilitation interventions - Two studies of practice facilitation applied the Conceptual Framework for Implementation Fidelity (CFIF) to PF retroactively We applied the CFIF to PF using pre-determined targets
Conceptual framework for implementation fidelity (CFIF) Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science : IS, 2, 40. http://doi.org/10.1186/1748-5908-2-40
Application of the conceptual framework for implementation fidelity to measure four adherence subcategories Adherence Carroll Definition HealthyHearts NYC Definition HealthyHearts NYC Pre-Determined Target Content The skills or knowledge 1) The percentage of sites documented as All sites complete all 39 tasks an intervention seeks to completion of all 39 tasks on the Task List; receive education on all 27 deliver to its recipients 2) The percentage of sites that PFs CCM change strategies documented ‘Educated’ for all 27 CCM strategies. Frequency Number of intervention The HHNYC protocol specified a minimum of 100% related interactions 13 on-site visits during the one-year intervention time period. Duration Length of program or The length of each interaction for intervention No required amount of time, length of each delivery (excluding travel time) and thus no pre-determined intervention interaction target of duration fidelity Coverage Number of recipients that Number sites that received the intervention as 75% sites achieve complete receive the intervention intended: represents combination of the other coverage. as intended three subcategories and is thus the most stringent. Carroll et al, 2007
Results Adherence Pre-Determined Results Target Content 100% PFs completed: • All 39 tasks with 89% of sites • Education of all 27 CCM change strategies in 71% of sites Frequency 100% 94% (240 out of 255 sites) received at least 13 on-site visits (range 10 – 18 visits) PFs spent an average of 26 hours total (range 10 – 52 hours) Duration NA delivering the intervention Coverage 75% Over half (62%) of all sites that completed the HHNYC intervention received it as intended and achieved full coverage (13 visits + completing all content)
Results Few differences between sites with greater and less fidelity: - <13 vs 13+ visits: no differences in task list or CCM completion - CCM completers averaged 27 hours compared to 25 hours among non- completers; no difference for task completion - Task list and CCM completers and non-completers no different on type of EHR, MUA status, PCMH status
Conclusions Virtually all practices met the frequency criteria of at least 13 visits, and facilitators completed all of the tasks on the task list in the vast majority of practices. Compared to the task list items, facilitators educated fewer practices, but still a large majority, on all CCM strategies. A substantial majority of practices received the intervention as intended. HHNYC PFs implemented the intervention with high fidelity, adhering to protocol for documentation and content delivery. This study provides a model for evaluating fidelity adherence in complex, tailored practice facilitation interventions within small urban independent primary care practices
Acknowledgements Donna Shelley, MD, MPH Allison Cuthel, MPH Nina Siman, MA, MSed Charles Cleland, PhD Hang Pham-Singer, PharmD Personnel from PCIP practices
Recommend
More recommend
