Research Tools and Methods: Investigating Student Success in - PowerPoint PPT Presentation

Research Tools and Methods: Investigating Student Success in Calculus-based Physics I KATHY KOENIG AND ALEX MARIES, PHYSICS DEPT UNIVERSITY OF CINCINNATI

In 2011, it launched its Undergraduate STEM Initiative to encourage systemic reforms to science education to improve teaching and learning, especially in first- and second-year courses. Higher education is “now reaching a major tipping point,” Coleman added. “We cannot condone poor teaching of introductory STEM courses because we are trying to weed out the weaker students in the class or simply because a professor, department and/or institution fails to recognize and accept that there are, in fact, more effective ways to teach.” AAU Five-year Status Report 10-2-17

Failing to adopt evidence-based teaching practices “must be viewed as irresponsible, an abrogation of fulfilling our collective mission to ensure that all students …..are provided with the maximum opportunity to succeed”. Department-wide innovations are becoming institutional priorities, teaching and learning centers are being redesigned, and data and analytics are being used to monitor and improve student learning.

Investigating Student Success in Calculus-based Physics I Setting:  Spring Semester 2017  7 sections of Calculus-based physics (~130 students each)  6 faculty taught across these sections (v arious teaching styles)

Data Collection:  Common exam scores and final course grades (DFW rates)  Force Concept Inventory (FCI) – pre-test (and post-test)  ACT Math Scores  Student engagement survey  Conversations with faculty to better understand teaching style

Teaching Styles 2 Flipped Classrooms  “Flipped” and “Flipped (New)”  videos assigned outside of class (selected from internet)  essentially no lecture  in-class problem solving (Powerpoint slides)  frequent use of clickers

Teaching Styles (cont’d) “Active” for Active Learning  assigned textbook reading (no videos assigned)  essentially ~30%-40% lecture  in-class problem solving (similar problems to Flipped)  frequent use of clickers

Teaching Styles (cont’d) “Lec-to-Active”  initial instructor went on medical leave after 1 st Exam (~week 6)  assigned textbook reading  used clickers and group problems solving (similar to others) “Lec-Based 1”, “Lec-Based 2”  Lec-Based 1 assigned textbook reading  essentially ~75% lecture (reported by students)  some in-class problem solving (similar problems to others)  one section included use of clickers

DFW Rates Spring 2017 25% 20% DFW Rate 15% 10% 5% 0% Active Flipped 1 Flipped Lec-to-Active Lec-Based 1 Lec-Based 2 (New)

What goe t goes i into t o the Cal alc-Ba Base sed P Physi sics g grade? 50% common exams 20% quizzes and other points up to the instructor 15% recitation attendance and homework 15% WebAssign Online Homework Red font represents common elements across courses.

Mean ACT Math Scores 30.0 Mean ACT Math (36 max) 29.5 29.0 28.5 28.0 27.5 27.0 26.5 Active Flipped Flipped Lec-to-Active Lec-based 1 Lec-based 2 (New)

Force Concept Inventory Pre-test 30 25 Pre-FCI Mean (of 30) 65% is mastery 20 15 10 5 0 Active Flipped Flipped Lec-to-Active Lec-based 1 Lec-based 2 (New)

All Exams (Avg of 3 Exams Plus Final) 85 80 Exam Mean (%) 75 70 65 60 55 50 Active Flipped Flipped Lec-to-Active Lec-based 1 Lec-based 2 (New)

Mean Exam Score by Quartiles 95 90 Mean of All Exams (%) 85 80 75 Active 70 Flipped 65 60 Flipped (New) 55 Lec-to-Active 50 Lec-based 1 45 1 2 3 4 Lec-based 2 Quartile

Mean Exam Score by Quartiles 95 90 Mean of All Exams (%) 85 7% spread 80 75 Active 10% spread 70 Flipped 65 12% spread Flipped (New) 60 55 Lec-to-Active 50 13% spread Lec-based 1 45 1 2 3 4 Lec-based 2 Quartile

How might these patterns be explained?

Measuring Classroom Climate The 4 Properties of Powerful Teachers  Personality - approachable, humor, good-natured  Presence – “own” the pedagogy, connect with content and students  Preparation - the obvious plus reassess b/n each class  Passion – Rob Jenkins, Chronicle of Higher Education, March 16, 2015

Student Engagement Survey  Survey given to all students last two weeks of the term  Survey was anonymous but tagged to instructor  Extra credit tied to survey completion (91% completed it!)  Objectives:  student background info not available through Institutional Research  how students engaged in course components  how useful students thought those components were  how each faculty conducted class and set up classroom climate

How often did you read Flipped Lec-based the text? Active (both*) (all 3**) Read text >75% of the time 6% 25% 2%-8% Read text ~50% of the time 8% 14% 1%-9% Read text ~25% of the time 36% 33% 24%-40% Read none of the text 51% 28% 33%-66% *responses similar **Lec-based 2 did not assign textbook reading according to students Of those who read the text: ~10% indicated text extremely useful all the time 18-33% indicated text never useful

How often did you do the Flipped Lec-based online homework? Active (both) (all 3) Most of the time 86% 78%-86% 79%-86% 75% of the time 12% 11%-12% 8%-17% 50% of the time 0% 1%-8% 2%-4% 25% or less of the time 2% 1%-3% 1%-3% Was the HW useful? Active FL New Lec-based Very useful 38% 24% 19% 18%-23% Somewhat useful 58% 62% 55% 52%-62% Not at all useful 4% 14% 26% 17%-24% Instructor of Active class assigned HW for all sections to complete.

Attendance (self-reported)  80% attended every class (another 10% attended 3/4 of classes) What motivated to attend? Flip Lec- Lec Lec Choose ALL. Active Flip (New) Active 1 2 Instructor took attendance. 4% 72% 37% 39% 65% 90% Clickers-participation grade. 7% 80% 83% 73% 36% 2% Lecture was useful. 70% 73% 34% 8% 65% 30% Instructor worked problems. 63% 74% 63% 9% 55% 32% Students worked problems. 32% 56% 58% 11% 37% 23% None of above motivated me. 13% 0% 1% 9% 6% 5%

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ALL) Choice (select ALL that apply) Response Textbook Khan Academy or online lecture source Other students SI Leader Learning Assistance Center (tutoring, etc) Office Hours with instructor Office hours with LA or graduate TA Google/internet I didn’t seek out help Other

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ALL) Choice (select ALL that apply) Response Google/internet 87% Other students 77% Khan Academy or online lecture source 51% Textbook 45% SI Leader 17% Other 10% Learning Assistance Center (tutoring, etc) 7% Office Hours with instructor 5% Ranged from 0%-11% Office hours with LA or graduate TA 3% I didn't seek out help 0%

When didn’t understand something or couldn’t do a HW problem, where did you generally go for help? (check ONE) Choice (select ALL that apply) Response Google/internet 51% 27% 38% Flip (New) Other students 23% Khan Academy or online lecture source 10% Textbook 9% SI Leader 1% Other 1% Learning Assistance Center (tutoring, etc) 1% Office Hours with instructor 1% Office hours with LA or graduate TA 1% I didn't seek out help 0%

Of the time you spent on physics each week, how much time was spent working with other students? a. I never worked with other students outside of class.** b. I occasionally worked with other students outside of class. c. I often worked with other students but it was informal without regular meeting times or necessarily the same students. d. I often worked with the same set of students but we never set specific meeting times. e. I often worked with the same set of students and we set regular study group meeting times.

Measuring Classroom Climate My physics instructor made me feel comfortable in asking questions either during or after class. Flip Lec- Lec Lec Active Flip (New) Active 1 2 Strongly Disagree/Disagree 8% 7% 18% 34% 9% 17% Disagree Little/Agree Little 37% 17% 41% 52% 41% 47% Agree/Strongly Agree 55% 76% 41% 14% 50% 36% Attended Office Hours: 11% 4% 0% 6% 2% 3%

Measuring Classroom Climate My physics instructor encouraged students to seek help if needed (i.e. office hours, Physics Learning Center, MASS, SI, etc.). Flip Lec- Lec Lec Active Flip (New) Active 1 2 Strongly Disagree/Disagree 7% 3% 5% 24% 2% 10% Disagree Little/Agree Little 34% 12% 17% 38% 27% 41% Agree/Strongly Agree 59% 85% 77% 40% 71% 50%

Measuring Classroom Climate My physics instructor made me feel like I could succeed in this course. Flip Lec- Lec Lec Active Flip (New) Active 1 2 Strongly Disagree/Disagree 10% 3% 10% 38% 7% 15% Disagree Little/Agree Little 37% 9% 29% 49% 50% 53% Agree/Strongly Agree 53% 89% 59% 14% 42% 22%