ET-805 Modeling Learners Emotions Ramkumar.Rajendran@iitb.ac.in - PowerPoint PPT Presentation

ET-805 Modeling Learner’s Emotions Ramkumar.Rajendran@iitb.ac.in

Activity - TPS An ITS adapts the learning content/feedback based on learners’ performance and preferences. Think individually and write down two most important preferences we should consider for adaptation (2 mins) Share (5 mins) 2

Activity - Class Response - Motivation - Interest - Skill on a concept or topic - mental state of the student - Based on student level of response - Language - Formal or informal - Affective State - mood - 3

Activity - TPS Think How humans teachers detect emotions in a traditional classroom scenario write one such approach (3 mins) Pair select one approach that the teachers used to predict the students' emotion and discuss how to use that approach in ITS (i.e., automate the approach) (5 mins) Share (3 mins) 4

Activity - Class Response - Self reporting - Emotion panel, Ask students directly - Body Posture and gesture - Camera - Engagement - asking student a simple reporting 5

Affective States Basic Emotions: Sad, Fear, Happy, Anger, Surprise, Disgust and Contempt Learner-centered emotion: frustration, boredom, confusion, curiosity, delight, engagement, surprise, and anxiety are more applicable to computer learning environments Rafael A. Calvo, and Sidney D’Mello, Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications, 6 2010

Voice and Text - Paralinguistic Features of Speech (Voice) - Text: Natural Language Processing - Low cost - Non-intrusive - Scalable But ? - Very Few ITS developed for Voice and Text input 7

Body Language and Posture ● Non-Intrusive ● Body Pressure Measurement System ● Machine learning methods by analyzing data has shown up-to 83% detection accuracy 8

Physiology Sensors Electroencephalography (EEG) Electromyogram (EMG) 9

Physiology Sensors Electrodermal Activity (EDA) 10

Activity - Discussion List the drawbacks of detection emotions using sensors - Cost - non-scaleable - Sensitive to small moves - Lab studies not for real classroom - Computationally costly - Privacy issues 11

Emotion Detection using Facial Expressions - Manual Observation - Trained coders can observe student’s emotion like in the traditional classroom - Can be done in real-time - Cost ineffective, not scalable - Coarse Grain data 12

Activity - Discussion List the drawbacks of detecting emotion using human observers - Can’t be used to personalize - Observers might have bias - Non scalable 13

Facial Expressions Automatic emotion detection using facial expressions - Learner’s facial expression is captured using web camera - 24 frames of pictures/Second video - Machine learning and deep learning networks are used to identify the units (AU) in each frame - Dataset labeled using human observers are used for training the machine learning classifiers 14

What is Action Units (AUs)! • FACS – Facial Action Coding System AU Number FACS Name 0 Neutral face 1 Inner Brow Raiser 2 Outer Brow Raiser 4 Brow Lowerer 5 Upper Lid Raiser 6 Cheek Raiser 15

Research Studies using Facial Expression 16

Activity - Discussion List the drawbacks of automatic facial emotion detection software - Too much of data - finer grained - Basic emotions - Posture and gesture in not considered - Can’t detect student facial expression all the time 17

Modeling Emotions using Log data - Mining log data and developing classifiers using labelled data - Labelling done by human observers - Features in data mining approaches are created by experts - Example features: response to questions, response to last 3,5,…, n questions, time taken to answer the question, etc - Machine Learning classifiers are used to detect emotions - Detection Accuracy > 80% 18

Activity - Discussion List the drawbacks of data mining approaches to detect emotions 19

Modeling Emotions using Log data - Learner’s emotions are detected by applying theory - Features for the classifiers are constructed by applying theoretical definition of emotions - Captures both when and why - Accuracy is less compared to data mining approaches 20

Theory-Driven Approach to Detect Frustration 21

Definition of Frustration The following factors of frustration are considered to model the student's frustration. - Frustration is the blocking of a behavior directed towards a goal - The distance to the goal is a factor that influences frustration - Frustration is cumulative in nature - Time spent to achieve the goal is a factor that influences frustration 22

Performance of Theory-Driven Approach Using Linear Regression 23

Performance of Theory-Driven Approach Using Different Classifiers 24

Activity - Discussion List the drawbacks of theory-driven approach to detect emotions 25

Last Activity - Muddy Points List down - two important and - two least clear (muddy) points from today’s class - https://tinyurl.com/et8 05mp 26