Assessing course design and student outcomes in an inquiry-based - PowerPoint PPT Presentation

Assessing course design and student outcomes in an inquiry-based lab course BPK 420: “Cell Physiology Lab” Megan Barker, Ciara Morgan-Feir, Damon Poburko, Nadine Wicks, Tom Claydon megan.barker@sfu.ca | twitter/wordpress: @meganbarkerase

BPK 420: Cell Physiology Laboratory Developed jointly by the depts. of Biological Sciences and Biomedical Physiology & Kinesiology Four units: 1) Introduction to cells Timeline: 2) Inducing and measuring Week 4 – discuss possible projects gene expression in Week 7 – submit top 2 ideas/picks mammalian cells Week 9 – submit project proposal 3) Earthworm physiology Week 10 – in-lab consultation, 4) Independent projects submit protocol, “order” reagents Weeks 11/12 – do experiment(s) Week 13 – poster session Week 14 – submit project report

How/what to assess? 1)Develop/refine (non-content) learning outcomes 2)Find or build assessments (mixed methods) 3)Prioritize! 4)Deploy 5)Analyze 6)Do it again!

Workflow of developing LOs & tools… à 4 quantitative tools chosen/built

Quantitative Assessment Attitudes about biology Content knowledge Tool: Diagnostic test Tool: CLASS-Bio (Semsar, Knight, Birol, Smith 2011) Likert-scale questions to measure novice-to-expert-like perceptions about biology. Multiple choice test, compiled from concept inventories. Deployed pre- &post-course.

Quantitative Assessment Experimental Design Skills Confidence in lab skills Tool: Self-assessment of Tool: EDAT confidence in lab skills (Experimental Design Ability Test, Sirum & Humberg 2011) Likert-scale responses to “rate your confidence in each lab activity” Students given prompt, design a related experiment, their designs scored on a rubric. Deployed “pre” (before independent projects), and on last day of term

Qualitative assessment Weekly feedback At end of term • What were the • Prior to developing your most challenging, and project ideas, what the most interesting parts aspect(s) of our course of your work this week? helped you understand how to design a (good) experiment? • From doing your project, what have you learned about experimental design so far?

Preliminary data… Content knowledge Experimental design ched pairs t-test ** * 1.2 1.0 1.0 Content knowledge 0.8 EDAT Score 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 Week 7 Week 13 1 3 1 k e k e e W e W P = 0.0179; Two-tailed; non-parametric; P = 0.0083; Two-tailed; non-parametric; Paired, Wilcoxon matched pairs t-test Paired, Wilcoxon matched pairs t-test

Comments from students… I learned that designing your own protocol actually gives you a much deeper understanding about how Designing my own experiment was the procedure affects your final really great because obtaining results from an experiment that results. you made is a lot more satisfying than following someone else’s instructions. I have a greater appreciation for people who do research and the amount of time, dedication, planning, lit reviews, and commitment that goes into working in science. I think that no matter how frustrating an experiment can seem at the time, one can feel accomplished and satisfied with results even if they aren’t what were expected.

• Level 2—Open inquiry: The prob • Level 3—Authentic inquiry: The • Level 0—Confirmation: An activ • Level ½—Structured inquiry: The Authentic work: Characterizing the level of inquiry • Level 1—Guided inquiry: The labo based on what is provided to students From Buck et al, JCST, 2013 TABLE 2 A rubric to characterize inquiry in the undergraduate laboratory. BPK 420: Level 0: Level ½ : Level 1: Level 2: Level 3: Confirmation Structured Guided inquiry Open inquiry Level ??? Authentic inquiry inquiry Characteristic Not provided Problem/Question Provided Provided Provided Provided Not provided Somewhat provided Theory/Background Provided Provided Provided Provided Not provided Somewhat provided Procedures/Design Provided Provided Provided Not provided Not provided Somewhat provided Results analysis Provided Provided Not provided Not provided Not provided Provided Results communication Provided Not provided Not provided Not provided Not provided Not provided Conclusions Provided Not provided Not provided Not provided Not provided More structure Less structure More structured Less structured

Anecdotally, what we learned… Getting them prepared is key • Saw much stronger understanding with pre-class quizzes versus pre-lab quizzes Build in-class/lab time for theory & especially analysis • Especially important in a non-cookbook lab Scale down the scope of the experiments • Pushing them to focus improves their conclusions Backwards design -> more straightforward assessment • Lots of data, so having clear questions was helpful Connect with student undergraduate research journal • Pride and ownership in work – authentic experience