1 Criteria for selecting implementation frameworks and theories among implementation researchers and practitioners Sarah A. Birken, PhD The University of North Carolina at Chapel Hill @birkensarah
2 2 @birkensarah
3 How do I choose a theory? 3 @birkensarah
Criteria for selecting theories and frameworks 4 • Snowball approach beginning with seminal articles 1-3 in implementation science regarding framework and theory selection • Iterative refinement based on discussion among authors 1. Holmström, J, Truex D: What does it mean to be an informed IS researcher? Some criteria for the selection and use of social theories in IS research. Information Systems Research Seminar in Scandinavia (IRIS) 2001, 313 – 326. 2. The Improved Clinical Effectiveness through Behavioral Research Group (ICEBeRG): Designing theoretically informed implementation interventions. Implementation Science 2006, 1:4. 3. Wacker JG: A definition of theory: research guidelines for different theory-building research methods in operations management. Journal of Operations Management 1998, 16: 361-385. 4 @birkensarah
Criteria for selecting theories and frameworks 5 1. Process guidance: Provision of a step-by-step approach for application 2. Inclusion of change strategies/techniques: Provision of specific method(s) for promoting change in implementation-related processes and/or outcomes • Falsifiability : Verifiable; ability to be supported with empirical data 3. Associated research method (e.g., informs qualitative interviews; associated with a validated questionnaire or methodology for constructing one): Recommended or implied method to be used in an empirical study that uses the framework or theory • Simplicity/parsimony: Relatively few assumptions are used to 4. Uniqueness: Ability to be distinguished from other theories or frameworks 5. Falsifiability: Verifiable; ability to be supported with empirical data explain effects 6. Inclusion of a diagrammatic representation: Elaboration in a clear and useful figure representing the concepts within and their inter-relations (if applicable) • Analytic level (e.g., individual, organization, system): Level of the 7. Application to a specific setting (e.g., hospitals, schools)/population (e.g., cancer): Intentional or historical use in studies related to particular groups and/or conditions 8. Specificity of causal relationships among constructs: Summary, explanation, organization, and description of relationships among constructs socioecological system at which constructs in the framework or 9. Empirical support: Use in empirical studies with results relevant to the framework or theory, contributing to cumulative theory-building 10. Disciplinary origins: Philosophical foundations theory lie 11. Simplicity/parsimony: Relatively few assumptions are used to explain effects 12. Fecundity: Offers a rich source for generating hypotheses • Generalizability : Applicability to various disciplines, settings, and 13. Analytic level (e.g., individual, organization, system): Level of the socioecological system at which constructs in the framework or theory lie 14. Generalizability: Applicability to various disciplines, settings, and populations populations 15. Logical consistency/plausibility: Inclusion of meaningful, face-valid explanations of proposed relationships 16. Explanatory power/testability: Ability to provide explanations around variables and effects; generates hypotheses that can be empirically tested 17. Outcome of interest: Conceptual centrality of the variable to which included constructs are thought to be related 18. Disciplinary approval: Frequency of use, popularity, acceptability, and perceptions of influence among a given group of scholars or reviewers, country, funding agencies, etc; endorsement or recommendation by credible authorities in the field 19. Description of a change process: Provides an explanation of how changes in process factors lead to changes in implementation-related outcomes 5 @birkensarah
6 6 @birkensarah
7 7 @birkensarah
Background 8 • Implementation frameworks and theories guide evidence translation, identify implementation determinants, and evaluate implementation. • The lack of guidance for selecting frameworks and theories may limit their (appropriate) use. 8 @birkensarah
Background 9 • Evidence exists of superficial use of frameworks and theories (e.g., Kirk et al., 2016; Eckstein & El Zarrad , 2012)… • This may be evidence of lack of clarity around selecting appropriate frameworks and theories. 9 @birkensarah
Background 10 • There is a need to promote appropriate selection and use of frameworks and theories. • Diminishing silos across disciplines, countries, degree types, and institution types in the following may promote appropriate selection and use of frameworks and theories. 10 @birkensarah
Objective 11 • Understand implementation researchers’ and practitioners’ criteria for selecting frameworks and theories 11 @birkensarah
12 Variables 12 @birkensarah
Respondent demographics 13 • Highest degree earned • Seniority (years of experience, external funding, articles published, etc.) • Institution type (e.g., academic, CRO) • Location of institution (country) 13 @birkensarah
Research vs. practice 14 Which of the following best describes the nature of your work? Please select one. I conduct or collaborate on implementation research studies proceed I implement programs and/or engage in quality improvement initiatives proceed I do some of both proceed None of the above do NOT proceed 14 @birkensarah
Criteria for selecting theories and frameworks 15 Thinking of your research in the last two years, which of the following characteristics did you consider when selecting a framework or theory for your research? Please select all that apply. Process guidance : Provision of a step-by-step approach for application Inclusion of change strategies/techniques : Provision of specific method(s) for promoting change in implementation-related processes and/or outcomes Associated research method (e.g., informs qualitative interviews; associated with a validated questionnaire or methodology for constructing one): Recommended or implied method to be used in an empirical study that uses the framework or theory Uniqueness: Ability to be distinguished from other theories or frameworks Falsifiability: Verifiable; ability to be supported with empirical data Inclusion of a diagrammatic representation: Elaboration in a clear and useful figure representing the concepts within and their inter-relations (if applicable) Application to a specific setting (e.g., hospitals, schools)/population (e.g., cancer): Intentional or historical use in studies related to particular groups and/or conditions Specificity of causal relationships among constructs: Summary, explanation, organization, and description of relationships among constructs Empirical support: Use in empirical studies with results relevant to the framework or theory, contributing to cumulative theory-building Disciplinary origins: Philosophical foundations Simplicity/parsimony: Relatively few assumptions are used to explain effects Fecundity: Offers a rich source for generating hypotheses Analytic level (e.g., individual, organization, system): Level of the socioecological system at which constructs in the framework or theory lie Generalizability: Applicability to various disciplines, settings, and populations Logical consistency/plausibility: Inclusion of meaningful, face-valid explanations of proposed relationships Explanatory power/testability: Ability to provide explanations around variables and effects; generates hypotheses that can be empirically tested Outcome of interest: Conceptual centrality of the variable to which included constructs are thought to be related Disciplinary approval: Frequency of use, popularity, acceptability, and perceptions of influence among a given group of scholars or reviewers, country, funding agencies, etc; endorsement or recommendation by credible authorities in the field Description of a change process: Provides an explanation of how changes in process factors lead to changes in implementation-related outcomes None of the above 15
Additional criteria 16 In the space provided, please list any characteristics NOT included in the list above that you considered when selecting a framework or theory for your research or practice in the past two years. 16 @birkensarah
Criteria ranking 17 In the space provided, please list the three most important criteria in order of importance Most important Second most important Third most important 17 @birkensarah
Analysis 18 • Descriptive statistics 18 @birkensarah
19 Results 19 @birkensarah
20 Study sample (n = 224) 20 @birkensarah
Highest degree earned 21 N=186 21 @birkensarah
Seniority 22 Item N Mean SD Minimum Maximum Range 184 13.8 8.9 1 54 53 Years of research Years of implementation research 184 7.4 7.1 0.5 44 43.5 Number of implementation research 179 6.0 7.8 0 50 50 projects Number of articles published 180 36.6 61.4 0 425 425 Number of articles in implementation 180 10.2 18.7 0 150 150 science published 184 No = 35.9% Yes = 63.0% PI of externally funded study? 22 @birkensarah
Recommend
More recommend
