Evaluating the Impact of Your Intervention: From Research Question - PowerPoint PPT Presentation

Evaluating the Impact of Your Intervention: From Research Question to Dissemination Jeffrey T. Olimpo, Ph.D. BUILDing SCHOLARS e-Conference

Disclaimer This e-Conference, meeting materials, and related content are solely the responsibility of UTEP and do not necessarily represent the official views of the National Institutes of Health.

Session Overview • What is discipline-based education research (DBER)? • What is the process for conducting DBER work? • Where and in what ways can I share my findings?

DBER Defined • DBER studies “investigate learning and teaching in a discipline from a perspective that reflects the discipline’s priorities, worldviews, knowledge, and practices.” (NRC, 2012) • In a more practical sense, DBER studies examine how students (and/or other key stakeholders) come to understand and apply the knowledge, concepts, and practices of a particular discipline.

Research Process Observation in the Classroom Formulation of a Problem Identification of a Research Question Incorporation of Theory Study Design & IRB Review Data Collection & Analysis Synthesis of Results Dissemination

Getting Started: Questions to Consider • What am I interested in exploring? In what context? • Who will be involved in the study, and who will I be collecting data from throughout the study? • What data will I collect? How and when will data collection take place? • How do I plan to disseminate the results of my study?

Identifying a Focus for Your Research • Multiple outcomes can be explored, including the impact of your intervention on participants’: • Content knowledge and/or conceptual understanding • Attitudes, motivations, beliefs, and/or perceptions • Professional skills development • Level of engagement within a specific learning environment • Depending upon the central focus of your research, multiple study populations (e.g., students; faculty; administrators; external stakeholders) could be included as part of your work.

Data Collection Methods • Should be aligned to your research question(s) and take into account the number of individuals you anticipate enrolling in your study (i.e., sample size) • Use of previously-validated measures is preferable. • Quantitative Approaches: ** Pay Attention to Number of Constructs/Factors Included ** Concept Inventories (e.g., IMCA) Likert-item Questionnaires/Surveys Scenario-based Prompts (e.g., E-EDAT) “Visualization” Tasks (e.g., SPFA)

Data Collection Methods (Cont’d) • Qualitative Approaches: • Coding of participant responses to open-ended prompts • Interviews (semi-structured/focus group) • Participant observations or evaluation of student artifacts • Mixed methods research = quantitative + qualitative

Pulling It All Together Research Question Instrument(s) Methods of Study Timing of Data Analysis Population(s) Collection

Pulling It All Together Research Question Instrument(s) Methods of Study Timing of Data Analysis Population(s) Collection What impact does Scientific Process Paired t-tests with All student Pre-/post-activity the XYZ activity have Flowchart Bonferroni participants on students’ Assessment correction, as development of (SPFA) needed scientific process skills? What aspect of the Open-ended Content analysis of All student Post-activity XYZ activity did prompt students’ participants students find most responses to the beneficial, and why? prompt

The IRB Process • Your Institutional Review Board (IRB) oversees all research involving human subjects. • The IRB review process is important for… • Adhering to ethical research practices • Obtaining and renewing grants • Presenting and/or publishing your work • Protection of investigators and institutions Exempt Review Expedited Review Full Review

STEM Education Conferences/Journals • DBER/SoTL Conferences: • National Association for Research in Science Teaching (NARST) • Society for the Advancement of Biol. Educ. Research (SABER) • AAC&U Transforming STEM Higher Education Conference • Council for Undergraduate Research (CUR) • DBER/SoTL Journals:

Contact Information Jeffrey T. Olimpo, Ph.D. Dept. of Biological Sciences University of Texas at El Paso jtolimpo@utep.edu