Adaptive Learning Meets Crowdsourcing Towards Development of - PowerPoint PPT Presentation

Adaptive Learning Meets Crowdsourcing Towards Development of Cost-Effective Adaptive Educational Systems Dr Hassan Khosravi The University of Queensland Brisbane, QLD, Australia h.khosravi@uq.edu.au Adaptive Learning Meets Crowdsourcing

Overview Adaptive Educational systems provide a potential solution. Challenges in delivering learner-centered learning at scale But they are expensive to purchase or develop. Reflections and intellectual challenges RiPPLE: A discipline-agnostic cost-effective crowdsourced adaptive educational system Adaptive Learning Meets Crowdsourcing 2

Adaptive Educational Systems (AESs) Developing Cost-Effective AESs The RiPPLE Platform Reflections and Intellectual Challenges Conclusion Adaptive Learning Meets Crowdsourcing Development and Adoption of an Adaptive Learning System 3

On Overview of Adaptive Educational Systems Adaptive Learning Meets Crowdsourcing 4

Adaptive Educational Systems Adaptive Educational Systems make use of data about students, learning processes, and learning products to provide an efficient, effective and customised learning experience for students by dynamically adapting learning content to suit their individual abilities or preferences. Domain Content Model Repository Learner Adaptation Engine Model Adaptive Learning Meets Crowdsourcing 5

How and When to Adapt? • Design-loop adaptivity : data-driven decisions made by course designers before and between iterations of system design • Task-loop adaptivity : data-driven decisions the system makes to select instructional tasks for an individual learner • Step-loop adaptivity : data-driven decisions the system makes in response to individual actions a student takes within an instructional task Adaptive Learning Meets Crowdsourcing 6

What to adapt to? • Knowledge state : This includes prior knowledge and knowledge growth • Students’ path through a problem : This includes solution strategy, specific errors, requests for help • Affect, motivation : This includes mind-wondering, emotion and cognitive load. • Metacognition: This includes self-regulation strategies • Learning Styles Adaptive Learning Meets Crowdsourcing 7

The Adaptivity Grid Design Loop Task Loop Step Loop Knowledge Using erroneous examples to Personalised adaptive task selection in The effect of positive feedback in a constraint improve mathematics learning air traffic control: Effects on training based intelligent tutoring system (Mitrovicet with a web-based tutoring efficiency and transfer (Saldenet al., al., 2013) system (Adam etal., 2014) 2010) Students’ path Example problems that improve The invention lab: Using a hybrid of Does supporting multiple student strategies student learning in algebra: model tracing and constraint- based lead to greater learning and motivation? Differentiating between correct modeling to offer intelligent support in Investigating a source of complexity in the and incorrect examples (Booth inquiry environments (Roll et al., 2010) architecture of intelligent tutoring systems et al., 2013) (Waalkenset al., 2013) Affect, motivation Confusion can be beneficial for Using adaptive learning technologies to Gaze tutor: A gaze-reactive intelligent learning ( D’Mello et al., 2014) personalize instruction: The impact of tutoring system (D’Mello et al., 2014) interest-based scenarios on performance in algebra. (Walkington& Sherman, 2012) Metacognition Limitations of student control: Supporting students’ self -regulated Motivation matters: Interactions between Do students know when they learning with an open learner model in achievement goals and agent scaffolding for need help? (Aleven& Koedinger, a linear equation tutor (Long & Aleven, self-regulated learning within an intelligent 2000) 2013) tutoring system(Duffy & Azevedo, 2015) Adaptive Learning Meets Crowdsourcing 8

Effectiveness of Adaptive Educational Systems A meta-analysis of 107 studies on ITSs involving 14,321 participants This review found that the effect size of human tutoring is d = 0.79 found that: ITS were associated with higher achievement relative to and the effect size of intelligent tutoring systems was 0.76, so they teacher-led large-group instruction, non-ITS computer based- are nearly as effective as human tutoring instruction, and texbooks or workbooks. This paper conducted a study on 3422 students from 198 offerings The meta-analysis indicated that Intelligent Tutoring Systems that have used ALEKS reporting significantly higher pass rates produced a large effect size on reading comprehension when amongst students using ALEKS. Yilmaz compared to traditional instruction (0.86) Adaptive Learning Meets Crowdsourcing 9

Developing Adaptive Educational Systems Publisher Model: designed with pre-existing Platform Model: provides a content-agnostic content. system infrastructure that enables instructors to • Examples : Pearsons MyLabs , McGraw-Hills develop content. • LearnSmart and ALEKS Examples: Smart Sparrow , Desire2Learn • Successful in K12 where content is standardized. and edX incorporate adaptive • Expensive to use functionalities • Introduce significant overhead for instructors • 25 hours of an expert's time for each hour of adaptive instruction (Aleven et al., 2006). • Both types are very expensive to develop /purchase and challenging to scale across different domains. Adaptive Learning Meets Crowdsourcing 10

Adaptive Educational Systems (AESs) Developing Cost-Effective AESs The RiPPLE Platform Reflections and Intellectual Challenges Conclusion Adaptive Learning Meets Crowdsourcing Development and Adoption of an Adaptive Learning System 7

Developing Cost-Effective Adaptive Learning Systems Adaptive Learning Meets Crowdsourcing 12

Successful Crowdsourcing Stories Outside of Education Crowdsourcing information Crowdsourcing answers Crowdsourcing Knowledge Crowdsourcing Service Crowdsourcing micro tasks Crowdfunding Adaptive Learning Meets Crowdsourcing 13

Successful Crowdsourcing Stories in Education Piazza: crowdsources answers PeerWise: Crowdsourcing learning resources in discussion forums Peer grading and evaluation system Crowdy: Interactive, Collaborative, Crowd- powered Video Learning Adaptive Learning Meets Crowdsourcing 14

Developing Cost-Effective Adaptive Learning Systems • Would Students Benefit from Creating/Evaluating Resources? • Can Students Create High-Quality Resources? • Can Students Accurately Evaluate the Quality of Resources? Adaptive Learning Meets Crowdsourcing 15

Would Students Benefit from Creating/Evaluating Resources? Question writing frequency correlated most strongly with summative performance (Spearman's rank: 0.24, p=<0.001). Only two questions of the 300 'most-answered' questions analysed had an unacceptable discriminatory value (<0.2) Adaptive Learning Meets Crowdsourcing 16

Can Students Create High-Quality Resources? “People with greater expertise tend to make assumptions about “60% of all explanations classified as being of high or outstanding student learning that turn out to be in conflict with students’ actual quality. Overall, 75% of questions met combined quality criteria” performance and developmental propensities.” “Crowdsourcing can efficiently yield high -quality assessment items that meet rigorous judgmental and statistical criteria. ” Adaptive Learning Meets Crowdsourcing 17

Can Students Accurately Evaluate the Quality of Resources? Under review by the Journal of IEEE Transaction in Educational Technology Students’ subjective rating of the quality of 1. 1. Aggregated subjective ratings are highly learning resources strongly correlates with that (and stat. sig.) predictive of the resources’ of domain experts. average learning gains, with Pearson correlation of 0.78. 2. Using a hybrid human-machine intelligence crowdsourcing consensus approach can increase the accuracy of the results Adaptive Learning Meets Crowdsourcing 18

Integrating Human/Machine Intelligence for Development of AESs • For this vision to be successful, educational technologies need to devise effective mechanisms to: Harness the creativity and Enable knowledgeable and Utilize AI algorithms to suggest evaluation power of students time-poor academics to spot-checking, dynamically adapt to create high-quality learning facilitate students in content instruction, learning content and resources. creation and moderation while activities to suit students’ providing minimal oversight. individual abilities or preferences. Adaptive Learning Meets Crowdsourcing 19

Adaptive Learning Systems (ALSs) Developing Cost-Effective ALSs The RiPPLE Platform Reflections and Intellectual Challenges Conclusion Adaptive Learning Meets Crowdsourcing Development and Adoption of an Adaptive Learning System 13

The RiPPLE Platform Adaptive practice Content creation Content moderation See ripplelearning.org Peer study recommendations Clicker-based in-class activity Adaptive Learning Meets Crowdsourcing 21

Content Creation MCQs Worked Examples Notes Adaptive Learning Meets Crowdsourcing 22

Content Moderation Adaptive Learning Meets Crowdsourcing 23

Content Moderation Feedback Adaptive Learning Meets Crowdsourcing 24

Spot Checking Developing algorithms that can utilize limited efforts in spot-checking to increase the accuracy of the moderation results Adaptive Learning Meets Crowdsourcing 25