High stakes automatic assessments: developing an online linear algebra examination Chris Sangwin School of Mathematics University of Edinburgh August 2018 Chris Sangwin (University of Edinburgh) Exams August 2018 1 / 28
Introduction Can we automatically mark current paper examinations? 1 To what extent is this equivalent to a paper exam? 2 Do we want to keep the current systems, and what are the 3 alternatives? Chris Sangwin (University of Edinburgh) Exams August 2018 2 / 28
Introduction Can we automatically mark current paper examinations? 1 To what extent is this equivalent to a paper exam? 2 Do we want to keep the current systems, and what are the 3 alternatives? Chris Sangwin (University of Edinburgh) Exams August 2018 2 / 28
Introduction Can we automatically mark current paper examinations? 1 To what extent is this equivalent to a paper exam? 2 Do we want to keep the current systems, and what are the 3 alternatives? Chris Sangwin (University of Edinburgh) Exams August 2018 2 / 28
The problem Large and growing university mathematics courses. Demands for more assessment & feedback. Chris Sangwin (University of Edinburgh) Exams August 2018 3 / 28
The problem Large and growing university mathematics courses. Demands for more assessment & feedback. Chris Sangwin (University of Edinburgh) Exams August 2018 3 / 28
Solution: online assessment Chris Sangwin (University of Edinburgh) Exams August 2018 4 / 28
STACK Demo Chris Sangwin (University of Edinburgh) Exams August 2018 5 / 28
Chris Sangwin (University of Edinburgh) Exams August 2018 6 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
STACK Computer algebra at the core. Separate validity from correctness. ◮ Students know what is required. ◮ Students not penalised on a technicality. ◮ Increases robustness of marking. Include CAS calculations within feedback. Formative feedback. Answer tests: Test if this is in factored form. Support for scientific units. Flexible multi-part questions. Question blocks. Reasoning by equivalence. Unit testing of questions. Chris Sangwin (University of Edinburgh) Exams August 2018 7 / 28
Integration with other systems LTI API + YAML. Chris Sangwin (University of Edinburgh) Exams August 2018 8 / 28
Integration with other systems LTI API + YAML. Chris Sangwin (University of Edinburgh) Exams August 2018 8 / 28
Community Over 800 Moodle sites. Every university in Finland uses STACK. Abacus materials consortium: https://abacus.aalto.fi/ ILIAS community (Germany) Chris Sangwin (University of Edinburgh) Exams August 2018 9 / 28
Community Over 800 Moodle sites. Every university in Finland uses STACK. Abacus materials consortium: https://abacus.aalto.fi/ ILIAS community (Germany) Chris Sangwin (University of Edinburgh) Exams August 2018 9 / 28
Community Over 800 Moodle sites. Every university in Finland uses STACK. Abacus materials consortium: https://abacus.aalto.fi/ ILIAS community (Germany) Chris Sangwin (University of Edinburgh) Exams August 2018 9 / 28
Research questions Can we automatically mark current paper examinations? 1 To what extent is this equivalent to a paper exam? 2 Chris Sangwin (University of Edinburgh) Exams August 2018 10 / 28
Research questions Can we automatically mark current paper examinations? 1 To what extent is this equivalent to a paper exam? 2 Chris Sangwin (University of Edinburgh) Exams August 2018 10 / 28
Method: background Add online practice exam to a university course. Introduction to Linear Algebra (ILA). Year 1, semester 1. 20/120 credits. > 600 students, (578 took the written exam). Grade: 80 % exam, 20 % coursework including STACK quizzes. Students requested a practice exam. Only one week between end of teaching and exam. Paper exam takes 35 person-days to mark. Chris Sangwin (University of Edinburgh) Exams August 2018 11 / 28
Method: background Add online practice exam to a university course. Introduction to Linear Algebra (ILA). Year 1, semester 1. 20/120 credits. > 600 students, (578 took the written exam). Grade: 80 % exam, 20 % coursework including STACK quizzes. Students requested a practice exam. Only one week between end of teaching and exam. Paper exam takes 35 person-days to mark. Chris Sangwin (University of Edinburgh) Exams August 2018 11 / 28
Method: background Add online practice exam to a university course. Introduction to Linear Algebra (ILA). Year 1, semester 1. 20/120 credits. > 600 students, (578 took the written exam). Grade: 80 % exam, 20 % coursework including STACK quizzes. Students requested a practice exam. Only one week between end of teaching and exam. Paper exam takes 35 person-days to mark. Chris Sangwin (University of Edinburgh) Exams August 2018 11 / 28
Method Conditions: Practice exam likely to be taken seriously. No contribution to grade = no incentive to cheat. Written exam “open book" Materials: Took oldest 2 past exams. Implemented as many questions as possible exactly . Justification vs answer? Chris Sangwin (University of Edinburgh) Exams August 2018 12 / 28
Method Conditions: Practice exam likely to be taken seriously. No contribution to grade = no incentive to cheat. Written exam “open book" Materials: Took oldest 2 past exams. Implemented as many questions as possible exactly . Justification vs answer? Chris Sangwin (University of Edinburgh) Exams August 2018 12 / 28
Method Conditions: Practice exam likely to be taken seriously. No contribution to grade = no incentive to cheat. Written exam “open book" Materials: Took oldest 2 past exams. Implemented as many questions as possible exactly . Justification vs answer? Chris Sangwin (University of Edinburgh) Exams August 2018 12 / 28
Recommend
More recommend
