Understanding how AI is applied in training: Case Studies ROBBY ROBSON EDUWORKS (CEO AND CO-FOUNDER) IEEE STANDARDS ASSOCIATION STANDARDS BOARD (MEMBER) WWW.CASSPROJECT.ORG (PRINCIPAL INVESTIGATOR) 1 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Outline Motivation Types of AI (Rules versus Machine Learning) Uses of AI (Decide versus Classify) Input Data Proposed Analysis Framework Use Cases Learning Navigator GIFT & PSTAAT Human Instruction ALEKS ElectronixTutor Summary 2 UNDERSTANDING AI IN TRAINING 15 - May - 2019
Motivation 3 15 - May - 2019 UNDERSTANDING AI IN TRAINING
US Department of Education “What Works Clearinghouse” Report on Carnegie Learning’s Cognitive Tutor Mixed effects No discernable effects Potentially negative effects The Cognitive Tutor™: Successful Application of Cognitive Science Dr. Stephen Blessing, Cognitive Scientist Carnegie Learning 4 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Producers don’t engage in studies Researchers are isolated from producers Consequences Consumers don’t know what to believe Purchasers don’t know what to buy Beneficial technology stays on the shelf 5 15 - May - 2019 UNDERSTANDING AI IN TRAINING
IEEE STANDARDS ACTIVITY What does AI mean in adaptive instructional systems (AIS)? How can we clarify the use of AI to improve adoption? 6 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Definition of AI Definitions of Artificial Intelligence (AI) Oxford : The theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages Barr : The part of computer science concerned with designing intelligent computer systems, that is, systems that exhibit the characteristics we associate with intelligence in human behavior – understanding language, learning, reasoning, solving problems, and so on. IBM : Anything that makes machines act more intelligently, including basic and applied research in machine learning, deep question answering, search and planning, knowledge representation, and cognitive architectures. 7 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Types of AI Rules and Formulas Any sufficiently advanced machine Expert Systems behaviour is indistinguishable from AI . Event-Condition-Action tables (apologies to Arthur C. Clarke) Hard-coded branching decisions Natural Language Processing Machine Learning Computational linguistics Naïve Bayes Dialog agents Neural Networks Text analysis Genetic Algorithms Machine translation Clustering Algorithms Speech recognition Ensemble Learning (Stacking) Ontological methods Supervised and Unsupervised 8 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Uses of AI Decisions What action to take? What topic is next? What content to display? Classifications What does the learner know? What topic does this content address? How difficult is this task? How engaged is the learner? 9 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Input Data Activity Steams and Test Results Sensor Data and Biometrics Competency Frameworks, Topic Maps, Knowledge Spaces Models and Data from Simulations Learner Input (text, voice, other) 10 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Issues to Consider Transparency Bias Regulations 11 UNDERSTANDING AI IN TRAINING 15 - May - 2019
Identify major components used for adaptivity and personalization Identify where AI is used or might be used For each such component identify: • the input data used; • whether the component uses rules or ML (and any known techniques or algorithms used); The Framework • whether the component decides or classifies; and • how data are fed forward among the components. Map this out visually Add text description and analysis, ideally: • High level description of system • High level description of classifiers and decision making • Transparency and potential biases 12 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Visual Representation Decide Rules DATA ML DATA Classify 13 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Gooru Navigator 14 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Gooru Navigator Decide Locator Recommender Competency Uses Event-Condition- Locates the learner Action Table (Rules) Frameworks using ML Activity Rules Learning ML Stream Goals Data Activity properties Activity properties Standard computed using formulas computed using ML Metadata Catalog Catalog Classify 15 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Generalized Intelligent Framework for Tutoring (GIFT) Instructional management has taken a leap forward with the development of the engine for managing adaptive pedagogy (EMAP) which examines learner domain competency, motivation, goal-orientation, and grit to aid in recommending courses and course paths for the learner, based upon research evidence (Goldberg et al., 2012). Domain modelling remains a complicated and challenging area for standardisation, but progress is being made in branching tutors from simple desktop tools for cognitive domains to more complex and dynamic tutors for psychomotor tasks. Brawner, Keith W., Anne M. Sinatra, and Robert A. Sottilare. "Motivation and research in architectural intelligent tutoring." IJSPM 12, no. 3/4 (2017): 300-312. 16 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Generalized Intelligent Framework for Tutoring (GIFT) Decide Domain Module Pedagogical Module Hard-coded eMAP or customized References Domain Content Knowledge File (Rules) pedagogical rules Course File outlining Metadata Rules ML topics about the content Sensor Module Learner Module Sensor Data Translates sensor inputs Learner levels, Surveys into state data preferences, etc. Classify 17 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Generalized Intelligent Framework for Tutoring (GIFT) (As used in Psychomotor Skills Training Agent-based Authoring Tool) Decide Domain Module References Domain Pedagogical Module Knowledge File (Rules) Hard-coded eMAP or customized Content pedagogical rules Course File Learner State outlining Metadata Rules ML topics about the content Compute Condition Expert Model Learner Module Sensor Data Classifies performance Learner levels, Surveys Sensor Module level in real time preferences, etc. Translates sensor inputs Expert & Novice Classify into state data Performance 18 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Human Instruction 19 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Human Instruction Decide Learner test Syllabus, Brain results, body regulations, Instructor uses brain to language, requirements, determine actions. reputations, classroom observed rules, time Rules behaviors, HL* constraints, facial situational Syllabus, regulations, Instructor’s interpretation expressions, awareness, requirements, class of learner state based on etc. past teaching management rules, etc. data and beliefs experience, Curriculum Assessment *HL = Human biases, etc. Classify Learning 20 15 - May - 2019 UNDERSTANDING AI IN TRAINING
In ALEKS, the basic element of the graph is not an individual concept or topic, but a “knowledge state”, that is, the combination of topics that might constitute an actual state of student knowledge in a subject. We use “big data” to build knowledge spaces, which map the relations among the knowledge states, or feasible states of student knowledge. These knowledge spaces enable ALEKS to accurately determine which individual topics the student has already mastered, and which ones she is ready to learn. - Smart ALEKS INTERVIEW | by Victor Rivero, Ed Tech Digest, April 10, 2013 21 15 - May - 2019 UNDERSTANDING AI IN TRAINING
ALEKS Decide Recommender Knowledge Space Student Recommends topics assessment based on student’s ZPD Associated results (entire Content student Rules ML population) Individual Identification of Student’s Adaptive assessment uses knowledge state from machine-learned student’s Curated assessment results algorithm assessment assessments Knowledge Space Assessment results Classify 22 15 - May - 2019 UNDERSTANDING AI IN TRAINING
AutoTutor (Expectations / Misconceptions Version) 23 15 - May - 2019 UNDERSTANDING AI IN TRAINING
AutoTutor (Expectations / Misconceptions Version) Decide Dialog scripts Semantic space classified as for analyzing Dialog Selection pumps, hints, responses Selects dialog based on prompts or classification of response feedback Text representing Rules ML List of topics correct and text responses and Uses semantic analysis to related to misconceptions compare student each topic response to sample text Off-the-shelf Assessment Student dialog agent Classify Response (i.e. an avatar) 24 15 - May - 2019 UNDERSTANDING AI IN TRAINING
Recommend
More recommend
