GAUGING MOOC LEARNERS ADHERENCE TO THE DESIGNED LEARNING PATH DAN - PowerPoint PPT Presentation

GAUGING MOOC LEARNERS’ ADHERENCE TO THE DESIGNED LEARNING PATH DAN DAVIS , GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN TU DELFT WEB INFORMATION SYSTEMS GROUP

WHAT IS A LEARNING PATH ? WHAT IS A LEARNING PATH ?

WHAT IS A LEARNING PATH ? THE SEQUENCE OF EVENTS A STUDENT TAKES TOWARDS A LEARNING OBJECTIVE

RQ to what extent do learners adhere to the designed learning path?

VIDEO QUIZ PROGRESS FORUM WATCH START VIEW START SUBMIT SUBMIT END END

DESIGNED LEARNING PATHS QUIZ QUIZ QUIZ VIDEO START SUBMIT END Data Analysis Data Analysis v v v v v v v Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q QUIZ QUIZ QUIZ VIDEO SUBMIT END START Functional Programming Functional Programming v v Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q FORUM VIDEO START SUBMIT END Framing Framing v v v F F F v F F F v F F F v F F F v v F F F v F F F v F F F QUIZ FORUM QUIZ QUIZ VIDEO START SUBMIT END SUBMIT START END Responsible Innovation Responsible Innovation v v v v v v v F F F Q Q Q Q Q Q Q Q Q Q Q Q Q

RELATED WORK RELATED WORK Demographic differences in how students navigate through MOOCs. In L @ S ‘14 PHILIP GUO & KATHARINA REINECKE Identifying latent study habits by mining learner behavior patterns in massive open online courses. In CIKM ’14 MIAOMIAO WEN & CAROLYN ROSÉ

* MOOC ENROLLED PASS CHAINS QUIZ TRIES VIDEOS RATE PASS/NON QUESTIONS ACCESSED FP101x 37k 5.3% 1.06M/807K 288 1 67.5% RI101x 4.3% 9k 66k/30k 75 1—3 49.7% Frame101x 34k 2.4% 95k/141k 26 2 51% EX101x 33k 6.5% 1.02M/855k 136 2 45% * Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%

* MOOC ENROLLED PASS CHAINS QUIZ TRIES VIDEOS RATE PASS/NON QUESTIONS ACCESSED FP101x 37k 5.3% 1.06M/807K 288 1 67.5% RI101x 4.3% 9k 66k/30k 75 1—3 49.7% Frame101x 34k 2.4% 95k/141k 26 2 51% EX101x 33k 6.5% 1.02M/855k 136 2 45% * Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%

FORUM FORUM START START VIDEO VIDEO 4 1 3 1 QUIZ QUIZ START START FORUM FORUM SUBMIT SUBMIT 6 1 3 QUIZ QUIZ SUBMIT SUBMIT QUIZ QUIZ FORUM FORUM END END 3 12 END END 6 1 EX101x FP101x PROGRESS PROGRESS FORUM FORUM START START 7 1 7 1 VIDEO VIDEO 1 3 5 QUIZ QUIZ START START FORUM FORUM SUBMIT SUBMIT 1 1 7 QUIZ QUIZ SUBMIT SUBMIT QUIZ QUIZ FORUM FORUM END END 10 END END 1 1 6 Frame101x RI101x PROGRESS PROGRESS

FORUM FORUM START START VIDEO VIDEO 4 1 3 1 QUIZ QUIZ START START FORUM FORUM SUBMIT SUBMIT 6 1 3 QUIZ EACH COURSE IS UNIQUE QUIZ SUBMIT SUBMIT QUIZ QUIZ FORUM FORUM END END 3 12 END END 6 1 EX101x FP101x PROGRESS PROGRESS FORUM FORUM START START 7 1 7 1 VIDEO VIDEO 1 3 5 QUIZ QUIZ START START FORUM FORUM SUBMIT SUBMIT 1 1 7 QUIZ QUIZ SUBMIT SUBMIT QUIZ QUIZ FORUM FORUM END END 10 END END 1 1 6 Frame101x RI101x PROGRESS PROGRESS

FORUM FORUM START START VIDEO VIDEO 4 1 3 1 QUIZ QUIZ START START FORUM FORUM SUBMIT SUBMIT 6 1 3 QUIZ EACH COURSE IS UNIQUE QUIZ SUBMIT SUBMIT QUIZ QUIZ FORUM FORUM END END 3 12 END END 6 1 EX101x FP101x PROGRESS PROGRESS FORUM FORUM START START 7 1 7 1 ANALYZE ACCORDINGLY VIDEO VIDEO 1 3 5 QUIZ QUIZ START START FORUM FORUM SUBMIT SUBMIT 1 1 7 QUIZ QUIZ SUBMIT SUBMIT QUIZ QUIZ FORUM FORUM END END 10 END END 1 1 6 Frame101x RI101x PROGRESS PROGRESS

APPROACH APPROACH 1. VIDEO INTERACTIONS 2. BEHAVIOR PATTERN CHAINS 3. EVENT TYPE TRANSITIONS

FINDINGS FINDINGS

1. VIDEO INTERACTIONS DO LEARNERS WATCH VIDEO LECTURES IN THE PRESCRIBED ORDER?

VIDEO INTERACTIONS Executed Paths Passing Week 3 Week 2 Week 1 Non-Passing FP101x Designed Lecture Order

VIDEO INTERACTIONS Passing Passing Week 3 Week 3 Week 2 Week 2 Week 1 Week 1 Non-Passing Non-Passing FP101x Frame101x Passing Passing Week 3 Week 3 Week 2 Week 2 Week 1 Week 1 Non-Passing Non-Passing RI101x EX101x

2. BEHAVIOR PATTERN CHAINS WHAT ARE THE MOST COMMON SEQUENCES IN THE EXECUTED LEARNING PATHS?

EXAMPLE EIGHT-STEP CHAIN FORUM END LECTURE WATCH QUIZ START QUIZ SUBMIT QUIZ SUBMIT QUIZ END PROGRESS LECTURE WATCH

* MOOC ENROLLED PASS CHAINS QUIZ TRIES VIDEOS RATE PASS/NON QUESTIONS ACCESSED FP101x 37k 5.3% 1.06M/807K 288 1 67.5% RI101x 4.3% 9k 66k/30k 75 1—3 49.7% Frame101x 34k 2.4% 95k/141k 26 2 51% EX101x 33k 6.5% 1.02M/855k 136 2 45% * Guo, P. & Reinecke, K. Demographic differences in how students navigate through MOOCs (2014) : 78%

OPEN CARD SORTING OPEN CARD SORTING

MOTIF A RECURRING OR REPEATED ELEMENT

FP101x MOTIF FREQ FREQ FREQ TOTAL PASS FAIL QUIZ COMPLETE 1. 552,363 328,995 223,368 (29.4%) (30.8%) (27.7%) X QUIZ events only with at least 1 X = SUBMIT BINGE WATCHING 2. 149,784 59,498 90,286 (8%) (5.6%) (11.2%) WATCH events only LECTURE -> QUIZ COMPLETE 3. 100,179 50,415 49,764 (5.3%) (4.7%) (6.2%) WATCH event(s) followed by X QUIZ events w/ >1 X = SUBMIT QUIZ COMPLETE -> FORUM 4. 99,828 67,722 32,106 (5.3%) (6.3%) (4%) X QUIZ events w/ >1 X = SUBMIT followed by X FORUM events

3. EVENT TYPE TRANSITIONS HOW DO STUDENTS NAVIGATE BETWEEN CERTAIN EVENT TYPES?

EVENT TYPE TRANSITIONS (EXECUTED) Frame101x Non-Passing Markov Model State Visualization

EVENT TYPE TRANSITIONS (EXECUTED) Frame101x Non-Passing Frame101x Passing

EVENT TYPE TRANSITIONS (EXECUTED) FORUM START FORUM START QUIZ START QUIZ VIDEO FORUM START VIDEO FORUM SUBMIT SUBMIT QUIZ QUIZ 44% FORUM QUIZ END END 35% FORUM END SUBMIT QUIZ END SUBMIT 21% 28% Frame101x Non-Passing Frame101x Passing PROGRESS 32% PROGRESS 44%

EXAMPLE FEEDBACK DAVIS, D., CHEN, G., JIVET, I., HAUFF, C., & HOUBEN, G. J. (2016). ENCOURAGING METACOGNITION & SELF-REGULATION IN MOOCS THROUGH INCREASED LEARNER FEEDBACK. LAK 2016 LAL WORKSHOP

Frame101x Non-Passing Frame101x Passing EX101x Non-Passing EX101x Passing RI101x Non-Passing RI101x Passing FP101x Non-Passing FP101x Passing

APPLICATIONS APPLICATIONS

1. BEYOND PASS-FAIL SEGMENTATION 2. PATTERN MINING & CONFORMANCE CHECKING 3. POSITIVE DEVIANCE

BIT.LY/WIS-LEARNING-ANALYTICS THANK YOU DAN DAVIS , GUANLIANG CHEN, CLAUDIA HAUFF, GEERT-JAN HOUBEN TU DELFT WEB INFORMATION SYSTEMS GROUP