Three Key Analytic Processes That Emerged While Using the Consolidated Framework for Implementation Research (CFIR) to Evaluate Broad-Scale System Change Jennifer N. Hill, MA Implementation Research Scientist Department of Veterans Affairs Center of Innovation for Complex Chronic Healthcare (CINCCH) Combating Antimicrobial Resistance through Rapid Implementation of Available Guidelines and Evidence (CARRIAGE) QUERI 9th Annual Conference on the Science of Dissemination and Implementation December 14, 2016
Disclosures and Funding Information • The views expressed in this presentation are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government. • This article is based on evaluations funded by the Department of Veterans Affairs Office of Patient-Centered Care & Cultural Transformation and the VA Health Services Research & Development Quality Enhancement Research Initiative (PCE 13-001, PI: Bokhour; PCE 13-002, PI: LaVela). VETERANS HEALTH ADMINISTRATION 1
Co-authors • Sara M. Locatelli, PhD - Edward Hines Jr. VA Hospital, Center for Evaluation of Practices and Experiences of Patient-Centered Care (CEPEP), Center of Innovation for Complex Chronic Health Care (CINCCH) • Barbara G. Bokhour, PhD - Center for Healthcare Organization and Implementation Research (CHOIR), ENRM Veterans Affairs Medical Center; Center for Evaluating Patient-Centered Care (EPCC); Boston University School of Public Health • Gemmae M. Fix, PhD - Center for Healthcare Organization and Implementation Research (CHOIR), VA Health Services Research and Development Service, Boston University School of Public Health; Evaluating Patient-Centered Care (EPCC) • Jeffrey Solomon, PhD - Evaluating Patient-Centered Care (EPCC) • Nora Mueller, MS - Center for Healthcare Organization and Implementation Research (CHOIR), ENRM Veterans Affairs Medical Center; Center for Evaluating Patient-Centered Care (EPCC) • Sherri L. LaVela, PhD, MPH, MBA - Edward Hines Jr. VA Hospital, Center for Evaluation of Practices and Experiences of Patient-Centered Care (CEPEP); Center of Innovation for Complex Chronic Health Care (CINCCH) VETERANS HEALTH ADMINISTRATION 2
Background: Theories and Frameworks for Evaluation • Theories and frameworks describe and prescribe aspects of an evaluation, rooted in the needs or requirements of the customer and the purpose of the inquiry. • These include activities or strategies, methods choices, and the responsibilities of and products to be provided by the evaluators [1]. • The Consolidated Framework for Implementation Research (CFIR) – Offers a comprehensive taxonomy of constructs related to the intervention, inner and outer settings, characteristics of individuals, and implementation process [2]. – Primarily has been used to evaluate implementation of single, discrete interventions or programs [3-5] – May also be useful for evaluating broad-scale programs implemented by large, integrated healthcare systems. VETERANS HEALTH ADMINISTRATION 3
Background: The Need for Evaluating a Broad-Scale System Change in VA • The Department of Veterans Affairs (VA) has committed to transforming from a problem-based, disease-focused system of care to one that is patient-centered. • This commitment was solidified with the creation of the Office of Patient-Centered Care and Cultural Transformation (OPCC&CT) in 2010 [6]. The office: – Established four Centers of Innovation (COIs) or “learning laboratories” where PCC innovations are implemented and lessons are gleaned for future large system rollouts. – They sought to understand how these COIs were implementing PCC and, – Engaged health services researchers to evaluate its implementation; given the size and scope of the transformation, two evaluation teams shared this task. • The objective of this presentation is to discuss three key analytic processes that emerged while using CFIR to evaluate a broad-scale system change. VETERANS HEALTH ADMINISTRATION 4
Selection of CFIR and Design of the Evaluation • As part of the application process, each evaluation team had independently selected CFIR to plan and structure their evaluations base on its comprehensiveness as well as the flexibility offered in recommendations of its use [7]. • While finalizing the constructs to be evaluated team used the “menu of constructs” process [only constructs essential to the evaluation were included]. • The selection of these ‘essential constructs’ was informed by the scope of the evaluation and the critical questions and goals of key leadership. • Operational definitions [8] were developed for the selected constructs and the framework was integrated into the evaluation design, data collection, and analysis. VETERANS HEALTH ADMINISTRATION 5
Examples of Selected Constructs and Rationales [Selection of Construct] - Rationale for Selection Construct Domain: Intervention Characteristics [Evaluation Team] - The intervention was ongoing, evaluation team wanted to Intervention Source capture historical information. [Evaluation Team] - Document adaptations, especially given the broad scope. Adaptability [Evaluation Team] - Perceived complexity of the PCC cultural transformation. Complexity Domain: Outer Setting [OPCC&CT and Evaluation Team] - Patient involvement in the transformation. Patient Needs & Resources [OPCC&CT and Evaluation Team]- Receptivity to the transformation. Domain: Inner Setting Climate\Culture Domain: Process [OPCC&CT and Evaluation Team] - How and why staff are engaged Engaging: Staff [OPCC&CT and Evaluation Team] -Site-level processes for tracking progress Reflecting & Evaluating VETERANS HEALTH ADMINISTRATION 6
Methods: Study Participants and Recruitment • Senior management and PCC coordinators at each facility recommended potential participants to the evaluation teams. • To reduce coercion, these individuals were invited to participate via an informational email which offered them the opportunity to contact evaluation team. • Interviewees were told their responses would be used by OPCC&CT to guide future program efforts including addressing program barriers and sharing of best practices. • Interviews were conducted either in-person during site visits or over the telephone if the participant was unavailable during the site visit. VETERANS HEALTH ADMINISTRATION 7
Methods: Qualitative Analysis • A mixed deductive-inductive [9,10] approach to coding was used to analyze data from the interviews. • Deductive coding was: – Guided by CFIR with a structured analytical tool to facilitate rapid qualitative analysis – To investigate implementation processes and successes/failures (deductive) and barriers and facilitators tied to the local context • Inductive coding was used to: – Capture themes not represented in CFIR to ensure coding was reflective of the data VETERANS HEALTH ADMINISTRATION 8
Methods: Process of Synthesis for Operations Partners • The final step of the analysis involved synthesizing these themes into key domains impacting implementation of PCC and cultural transformation. • This process involved: (1) conducting additional analyses on data within key domains to define how and why it was salient (2) developing recommendations that could be utilized by leadership to enhance implementation of the program. • These analytical processes undertaken were tracked by the team as they were considered essential components of utilizing CFIR for this broad scale evaluation. • All key analytic processes identified by the team and the documented aspects of those processes are presented in the results. VETERANS HEALTH ADMINISTRATION 9
Results: Overview • While using CFIR to evaluate these broad-scale programmatic changes, 3 key analytic processes were identified: (1) the creation of adapted definitions for the CFIR constructs to account for its application to the broad-scale evaluation (2) the mixed deductive-inductive coding process demonstrated the flexibility of CFIR for complex evaluation in the emergence of additional CFIR constructs and the several new key themes from the co-occurring inductive thematic coding. (3) the rapid analysis and synthesis of the data into key domains impacting implementation of PCC&CT to develop recommendations to support enhancement of implementation and expansion opportunities for the program . VETERANS HEALTH ADMINISTRATION 10
Results: Adapted Definitions • As a first step, the evaluation team reviewed the CFIR domains and constructs and developed adapted definitions based on the study context, including: (1) the broad scope of the PCC intervention(s) (2) the broad-scale culture change targeted (3) the input and existing knowledge of the operations partner OPCC&CT (4) the goals of the evaluation including assessing what had already occurred and what was currently in progress. VETERANS HEALTH ADMINISTRATION 11
Recommend
More recommend
