Using R to Assess Mathematical Sense-Making in Introductory Physics - PowerPoint PPT Presentation

Using R to Assess Mathematical Sense-Making in Introductory Physics Courses by Brian Danielak, Eric Kuo, Michael M. Hull, and Andrew Elby 2010 UseR! Conference - July 21, 2010 1

I work behind here 2

Students in conventional courses often treat formulas as algorithms. Sherin, B. L. (2001). How Students Understand Physics Equations. Cognition and Instruction, 19(4), p. 479 3

if I know The uniform and the the initial acceleration... time... velocity... v = v 0 + at Then I can calculate the final velocity! 4

Sense-making is about seeking coherence 5

v = v 0 + at

v = v 0 + at

v = v 0 + at if I know The uniform and the the initial acceleration... time... velocity... t 1 = + v = v 0 + at v = v 0 + at Then I can calculate the final velocity! 8

v = v 0 + at if I know The uniform and the the initial acceleration... time... velocity... t 1 = + v = v 0 + at v = v 0 + at t 2 = + Then I can calculate the final velocity! 9

v = v 0 + at if I know The uniform and the the initial acceleration... time... velocity... t 1 = + v = v 0 + at v = v 0 + at t 2 = + t 3 = + Then I can calculate the final velocity! 10

The same mathematical structure models lots of other situations 11

money = P + rt t 1 = + t 2 = + t 3 = + 12

ˆ Y = β 0 + β 1 X 1 x 1 = + x 2 = + x 3 = + 13

So here’s the big Question 14

v = v 0 + at if I know The uniform and the the initial acceleration... time... velocity... How can we test for t 1 = + v = v 0 + at v = v 0 + at thinking like this t 2 = + t 3 = + Then I can calculate the final velocity! 15 if I know The uniform and the the initial acceleration... time... velocity... When we usually test for v = v 0 + at thinking like this ? Then I can calculate the final velocity! 10 15

A Tale of 3 Instructors 1 2 3 New Veteran Veteran Sense-making Traditional Approach 16

And one Final Exam Multiple Choice Free Response 17

And one Final Exam Multiple Choice Free Response 18

1 2 3 4 426 Students Sense-making 5 items 3 Classes 6 7 chosen a priori 8 19

Class 1 Class 2 Class 3 2 3 4 e-score 20

Looking graphically at the data 21

Across all 8 MC items Raw Score Distributions by Instructor 35 30 25 Sense-making 20 1 (new) 15 10 5 0 35 30 25 Sense-making 20 count 2 (vet) 15 10 5 0 35 30 25 Traditional 20 3 (vet) 15 10 5 0 0 2 4 6 8 rawscore 22

Just the 5 Sense-making Items Escore Distributions by Instructor 40 30 Sense-making 1 (new) 20 10 0 40 30 Sense-making count 2 (vet) 20 10 0 40 30 Traditional 3 (vet) 20 10 0 − 1 0 1 2 3 4 5 6 escore 23

We ran a one-way ANOVA, then MCPs 24

Result 1 Across all 8 Multiple Choice items, Sense-making > traditional approach 25

95% family − wise confidence level 2 − 1 3 − 1 3 − 2 − 0.15 − 0.10 − 0.05 0.00 0.05 0.10 Differences in mean levels of instructor Rawratio 26

Across all 8 MC items 95% Family-wise Confidence Level - - - 3 2 1 1 3 2 TukeyHSD 0.061 -0.019 -0.080 p-adj 0.053 0.751 0.007 Cohen’s d 0.277 -0.0839 -0.365 27

Result 2 Across the 5 specific sense-making items, Sense-making >> traditional approach 28

95% family − wise confidence level 2 − 1 3 − 1 3 − 2 − 0.20 − 0.15 − 0.10 − 0.05 0.00 0.05 0.10 Differences in mean levels of instructor Eratio 29

On Sense-making MCs 95% Family-wise Confidence Level - - - 3 2 1 1 3 2 TukeyHSD 0.0450 -0.0871 -0.132 p-adj 0.296 0.0130 < 0.001 Cohen’s d 0.182 -0.331 -0.510 30

Exploring Free-Response Data 31

What happens to mechanical energy after the explosion? ? 32

What Happens to the Mechanical Energy of the Two Blocks? 140 120 100 q09d stays the same 80 increases count blank decreases 60 missing value uncodeable 40 20 0 1 2 3 instructor 33

What Happens to the Mechanical Energy of the Two Blocks? 140 120 100 q09d stays the same 80 increases count blank decreases 60 missing value uncodeable 40 20 0 1 2 3 instructor 34

What Happens to the Mechanical Energy of the Two Blocks? 140 120 100 q09d stays the same 80 increases count blank decreases 60 missing value uncodeable 72 40 43 43 20 0 1 2 3 instructor 35

What Happens to the Mechanical Energy of the Two Blocks? 140 120 Mechanical Energy 100 Stays the Same q09d stays the same 80 increases count blank decreases 60 missing value uncodeable 72 40 43 43 20 0 1 2 3 instructor 36

Mechanical Energy Stays the Same 72 43 43 = Sense-making Sense-making Traditional (new) (vet) (vet) 37

Mechanical Energy Stays the Same 72 43 43 = Sense-making Sense-making Traditional (new) (vet) (vet) 38

Result 3 72 When considering energy, more traditional students 43 43 asserted something counter-intuitive Sense-making Sense-making Traditional (new) (vet) (vet) 39

What Happens to the Mechanical Energy of the Two Blocks? 140 120 100 q09d stays the same 80 increases count blank decreases 60 missing value uncodeable 40 20 0 1 2 3 instructor 40

Mechanical Energy Increases 33 75 59 41

Mechanical Energy Increases 33 75 59 42

Mechanical Energy Increases 75 59 33 43

Mechanical Energy Increases 75 59 33 > Sense-making Sense-making Traditional (new) (vet) (vet) 44

Mechanical Energy Increases 75 59 33 > Sense-making Sense-making Traditional (new) (vet) (vet) 45

Result 4 When considering energy, 75 more sense-making 59 students were consistent 33 with simple intuition. Sense-making Sense-making Traditional (new) (vet) (vet) 46

Ahead, t ar b e ... DRAGONS! http://www.flickr.com/photos/aztlek/2421800480/in/ set-72157605762980035/ 47

This stuff’s in beta. http://www.flickr.com/photos/aztlek/2421800480/in/ set-72157605762980035/ 48

Students Who Quoted ‘‘Conservation of Energy'' as their #§* Explanation for q09d 30 25 20 variable count 5 15 5x 10 5 0 1 2 3 instructor 49

Mechanical Energy Stays the Same... 32 11 8 50

Mechanical Energy Stays the Same... Because Energy is Conserved 32 11 8 Sense-making Sense-making Traditional (new) (vet) (vet) 51

Result 5 32 When justifying their answer, more traditional students just quoted a rule , ignoring that it was tautological and inaccurate. 11 8 Sense-making Sense-making Traditional (new) (vet) (vet) 52

Summary 53

95% family − wise confidence level Result 3 2 − 1 72 When considering energy, 3 − 1 more traditional students 43 43 asserted something counter-intuitive Sense-making Sense-making Traditional (new) (vet) (vet) 3 − 2 − 0.20 − 0.15 − 0.10 − 0.05 0.00 0.05 0.10 Differences in mean levels of instructor Eratio 29 39 Result 4 Result 5 32 When justifying their answer, When considering energy, 75 more traditional students just more sense-making 59 quoted a rule , ignoring students were consistent 33 that it was tautological with simple intuition. and inaccurate. 11 8 Sense-making Sense-making Traditional (new) (vet) (vet) Sense-making Sense-making Traditional (new) (vet) (vet) 46 52 54

http://github.com/briandk/user2010public 55

Thanks! Andy Ayush Project Directors Elby Gupta Graduate Eric Mike Kuo Hull Assistants 56

Many Thanks NSF-DRL grant # 0733613 NSF-EEC grant # 0835880 57

95% family − wise confidence level Result 3 2 − 1 72 When considering energy, 3 − 1 more traditional students 43 43 asserted something counter-intuitive Sense-making Sense-making Traditional (new) (vet) (vet) 3 − 2 − 0.20 − 0.15 − 0.10 − 0.05 0.00 0.05 0.10 Differences in mean levels of instructor Eratio 29 39 Result 4 Result 5 32 When justifying their answer, When considering energy, 75 more traditional students just more sense-making 59 quoted a rule , ignoring students were consistent 33 that it was tautological with simple intuition. and inaccurate. 11 8 Sense-making Sense-making Traditional (new) (vet) (vet) Sense-making Sense-making Traditional (new) (vet) (vet) 46 52 58