Facilitated by Byron Powell! Setting the Stage for Understanding Mechanisms of Implementation David Chambers , DPhil Nate Williams, PhD, Cara C. Lewis, PhD LCSW Deputy Director for Associate Investigator, Implementation Science, Assistant Professor, MacColl Center for Healthcare Division of Cancer Control and Boise State University Innovation Population Sciences School of Social Work Kaiser Permanente Washington Health Research Institute AHRQ Grant Number: 1R13HS025632 March 22, 2019
How we’ll set the stage Part 1 Where have we been & where are we going? Part 2 What are implementation mechanisms & why do they matter? Part 3 What do we know about implementation mechanisms? Part 4 What are we going to do to inform the study of implementation mechanisms? March 22, 2019 2
Part 1: • Progress • Time for the next step… March 22, 2019 3
Evidence-based health practices, programs, and interventions March 22, 2019
Implementation research from 2000- March 22, 2019 5
Theories, frameworks, and models N = 61 http://dissemination-implementation.org/ Exploration Consolidated Adoption/ Interactive Diffusion of Framework for Preparation Systems Innovations Implementation Framework Implementation Research Sustainment Tabak et al., 2012 March 22, 2019 6
Measurement: Toward standardization & harmonization Society for Implementation Research Collaboration • Instrument Review Project • http://www.societyforimplementationresearchcollaboration.org/sirc- projects/sirc-instrument-project/ Grid-Enabled Measures • Developed by the National Cancer Institute • http://cancercontrol.cancer.gov/brp/gem.html Rabin et al. Measurement Resources: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0401-y March 22, 2019 7
Part 2: What are implementation mechanisms & why do they matter? March 22, 2019 8
Implementation mechanisms 2/3 of implementation Nearly 1/2 have no effect efforts fail on outcomes of interest Why do they matter? Need to tailor strategies Cost and complexity to diverse contexts are increasing March 22, 2019 9
Key terms & definitions Implementation Moderator outcomes Determinants of Mediator practice Implementation Mechanism strategies March 22, 2019 10
Proctor et al., 2011 Implementation outcomes The effects of deliberate and purposive actions to implement new treatments, practices, and services. Implementation Service outcomes Client outcomes outcomes • Efficiency • Satisfaction • Acceptability • Safety • Function • Adoption • Effectiveness • Symptomatology • Appropriateness • Equity • Costs • Patient- • Feasibility centeredness • Fidelity • Timeliness • Penetration • Sustainability March 22, 2019 11
Krause et al., 2014 Determinants of practice Factors that obstruct or enable changes in targeted professional behaviors or healthcare delivery processes. March 22, 2019 12
Proctor, Powell & McMillen, 2013; Implementation Powell et al., 2012 strategies Methods or techniques used to enhance the adoption, implementation, and sustainability of clinical program or practice. • Needs assessment • Change systems Plan Restructure strategies strategies • Site visits • Revise roles Quality • Educational meetings • Audit & feedback Educate mgmt. strategies • Make training dynamic • Use advisory boards strategies Policy • Use capitate payment • Change requirements Finance context strategies • Alter incentives • Change liability laws strategies March 22, 2019 13
McKinnon & Luecken, 2008 Moderators Third variable that changes the relationship between two other variables such that the relationship between them differs depending on the level of the moderator variable. Adoption Training of EBP Age March 22, 2019 14
Kazdin, 2007 Mediators An intervening variable that may account (statistically) for the relationship between the independent and dependent variable. Adoption Training of EBP Attitudes March 22, 2019 15
Kazdin, 2007 Mechanisms The reasons why change occurred or how change came about. Knowledge deficit Adoption Training of EBP March 22, 2019 16
Kazdin, 2007 Mechanisms The reasons why change occurred or how change came about. Knowledge Knowledge acquisition deficit Adoption Training of EBP March 22, 2019 17
How it all fits together Determinant Implementation Strategy Mechanism Implementation Outcome Provider Education (provision of Awareness- Feasibility, knowledge information) building, acceptability, deficit knowledge- appropriateness, acquisition adoption Provider skill Training (teaching & practice Skill acquisition, Fidelity to EBP deficit with corrective feedback) refinement, mastery Turnover Train-the-trainer Continuous on- Sustainability site expertise available for consultation Provider Clinical champion-led Organizational Feasibility, engagement implementation team culture, climate acceptability, appropriateness Unstandardized Guidelines Clarity of clinical Fidelity clinical care care options March 22, 2019 18 18
Part 3: What do we know about implementation mechanisms? March 22, 2019 19
Systematic Reviews: Two approaches to “What do we know?” Williams, 2016 Targeted review of implementation RCTs • RCTs that used an implementation strategy to manipulate a potential mechanism and tested the mechanism via mediation analysis • Catalogue, identify, recommend Lewis et al., in prep Broad survey of the field • DI studies that address mechanisms in all areas of health using any methodology • Describe, illustrate, assess March 22, 2019 20
Overview of Williams, 2016 1,309 records screened on title/ abstract 88 RCTs located & full texts reviewed 9 RCTs tested implementation mechanisms via mediation analysis March 22, 2019 21
Two types of mechanisms examined 1. strategy determinant of practice implementation outcome Did the strategy engage the Did the mediator mediator? influence the outcome? 2. strategy implementation outcome clinical outcome March 22, 2019 22
1. strategy determinant implementation Inner setting characteristics tested as mechanisms Inner setting characteristics 4 RCTs • Leader support No inner setting • characteristics Job-related barriers were supported as • Org. climate mechanisms • Agency supportive practices • Resources Implementation Implementation outcome strategy March 22, 2019 23
1. strategy determinant implementation Individual characteristics tested as mechanisms Individual 3 RCTs characteristics No individual characteristics • Attitudes were supported as • Readiness to change mechanisms • Self-efficacy • PBC • Subjective norm Implementation Implementation outcome strategy March 22, 2019 24
1. strategy determinant implementation Features of the implementation process tested as mechanisms Implementation 2 RCTs process Peer opinion leader support was • External change (partially) supported agent contact/ as a mechanism! support ? • Peer opinion leader support Implementation Implementation outcome strategy March 22, 2019 25
2. strategy implementation clinical Implementation outcomes tested as mediators Implementation 4 RCTs outcome No implementation • outcomes were Fidelity • supported as # of sessions • mechanisms # objectives completed • Engagement with ? clients Implementation Clinical outcome strategy March 22, 2019 26
Summary of implementation trials testing mechanisms in mental health Lots of 9 16 1 work to do! Randomized Mechanisms Mechanism trials tested (partially) supported March 22, 2019 27
Overview of Lewis et al. 1979 records screened on title/ abstract 204 full-text records assessed for eligibility 50 studies included for data extraction (6 overlap with Williams, 2016) March 22, 2019 28
Characterizing the landscape of mechanism-focused D&I research Mixed Qualitative Methods 12% 14% Quantitative Randomized 26% Quantitative Non- randomized 48% March 22, 2019 29
What is the quality of studies attempting to identify mechanisms? MMAT scores 70% 60% 50% % of studies 40% 30% 20% 10% 0% 0% 25% 50% 75% 100% Hong, Quan Nha, et al. "The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers." Education for Information Preprint (2018): 1-7. March 22, 2019 30
Illustrating approaches to understanding mechanisms 1. 5. 9. 2. 6. Implementation strategy Determinant of practice 3. 7. Implementation outcome Clinical outcome Mediator 4. 8. Moderator
Criteria for establishing mechanisms • Strong association • Specificity 7 criteria for • Consistency building a • Experimental strong case manipulation for a mechanism • Timeline • Gradient Bradford-Hill, 1965; Kazdin, 2007 • Plausibility or coherence March 22, 2019 32
Recommend
More recommend
