IIS - PROJECT 3 Group 8 Qian Sun Louis Janse van Rensburg - PowerPoint PPT Presentation

IIS - PROJECT 3 Group 8 Qian Sun Louis Janse van Rensburg Abdulghani Zubeir

Previous Literature Virtual vs Physical Embodiment Costa et al. (2016) found Users are more attentive to physically embodied ● emotional storytelling (Robot with Facial Expression Capability) Nao Limitation: no facial motors, dependence on postural expression and use of non-facial expression capacities (Sound/Voice, LED Colors)

Previous Literature (cont.) Conveying Emotion without Facial Expression Erden (2013) used postural expression derived from behavioural studies ● (Coulson, 2004) to express Happiness, Sadness & Fear in a Nao robot Hyde et al. (2014) found users rate emotions more intensely when agents ● have ‘high’ auditory emotion level Paiva et al. (2014) highlight use of color (LEDs) in robot emotion ● expression

Research Questions When Combining LED, Sound & Postural Capacities of Nao Robot: 1. Does rate of correct identification of the emotion expressed (Happiness, Sadness, Fear, Surprise, Anger, Disgust) differ between Virtual and Physical Embodiment Conditions? 2. Do ratings of naturalness and intensity differ between virtual and physical embodiment conditions: Across all emotions ○ For each individual emotion ○

Methodology 1. Emotion Postures for the 6 emotions derived from Coulson (2004) For each emotion they determined the most identifiable posture from 32 possible ○ candidate-postures 2. Transformation functions of joint angles of the happiness posture used to map postures to to the Nao robot in et al. We extrapolated this transformation function to determine motor-joint angles in Nao ○ postures for all emotions 3. LED Colors determined by Psychological Color-Emotion Association (Parker, 2012) 4. Emotion vocalization sounds selected from public soundbite sites

Deriving Postures Physical/Virtual ○ Posture Joint Angles for 6 Basic Emotions (Coulson, 2004) Nao Motor Angles Implemented Using Choregraphe Transformation Functions (Erden, 2013)

Experimental Design Within Subjects & Between Subjects ● All subjects experienced Virtual and Physical Embodiment conditions ○ Interleaved Order of Presentation of Physical/Virtual Conditions ○ Randomized order of emotion presentation ● For each trial in each physical/virtual condition ○ Questionnaire ● Classification of Emotion ○ Measure of Emotion Intensity (10 Point Likert Scale) ○ Measure of Emotion Naturalness (10 Point Likert Scale) ○

Within and Between Subjects Mixed Ordering of Condition Presentation 7 Participants: Physical First Virtual Second 3 Participants: Virtual First, Physical Second

Questionnaire Following Each emotion expression participants filled out the following:

Demonstration (video)

Results Sample: 10 Subjects ● 7 viewed Physical first, 3 viewed Virtual first ○ Demographics: ● 3 Females ○ 7 Males ○ All master’s students Swedish & International ○

Results: Emotion Classification (Between Subjects) For the first viewing (Between Subjects Analysis) 41/42 (97.6%) Correct Classifications of Emotions in the Physical-First ● Condition (7 Subjects) 15/18 (83.3%) Correct Classifications of Emotions in Virtual-First Condition ● (3 Subjects) Chi Square Analysis ● Chi2 value of 4.132, p = 0.0421 ○ Participants in the Physical-first condition were significantly better at classifying emotions ○ than those in the Virtual-first condition.

Results: Emotion Classification (Within Subjects) Combining first and second viewings, McNemar Test was used Note: Chi Square cannot be used for within subjects analysis due to violation ● of independent observations assumption 59/60 (98.3%) Correct Classifications of all Physical Emotion Expressions ● 56/60 (93.33%) Correct Classifications of all Virtual Emotion Expressions ● McNemar Test ● No significant difference in subjects ability to classify emotions between Physical and ○ Virtual conditions (p = .125) when combining first and second viewings.

5/120 Misclassifications Anger misclassified once as Disgust ● Disgust misclassified 3 times as Fear (to be discussed, no pun intended) ● Surprise misclassified once as Happiness ●

Results: Paired Sample T-Tests Virtual vs Physical Intensity and Naturalness across all Emotions: Within Subjects (combined first and second viewing)

Physical vs Virtual: Intensity by Emotion (within subjects) Paired Samples T-Tests No sig. differences ●

Physical vs Virtual: Naturalness by Emotion (within subjects) Paired Samples T-Tests Sig. difference for Fear only ●

Conclusions and Discussion 1. Upon first viewing, subjects were able to better classify emotions expressed by the physical robot compared to the virtual robot 2. The difference in ability to classify emotions between these conditions dissappeared when combining first and second viewing data (improvement bias). 3. Overall (across all emotions) the Physical robot was rated as more intense. However, when looking at individual emotions, no individual emotion displayed any significant difference between physical/virtual conditions. 4. Overall, there was no significant difference in ratings of naturalness between the two conditions, however, Fear was significantly rated as more natural in the Physical robot.

Limitations Small Sample Size ● Unbalanced samples between subjects first viewing (7 vs 3 subjects in ● physical and virtual conditions respectively) Within subjects design prohibits use of ANOVA (and between subjects ● sample set is too small & unbalanced to conduct meaningful ANOVA). Transformation Functions restricted by Range of Nao Robot Movements ● Motor Angles for Disgust resulted in unbalanced centre of weight of ● robot, one arm had to be moved inward possibly causing worse classification of Disgust Questionnaire framing: Classification limited to a set of 6 possible ● answers. Would be better if subjects generated an answer themselves.

Ethical Concerns Virtue ethics ● Virtuous emotions ○ Moral concerns ○ Emotions moral variations ○ Consequentialism ● Goodness ○ Alternative emotions ○ Deontologist: ● Emotions on act ○ Interactive emotions ○ General Concern: Emotion expression in robots could be used to ● manipulate people.

References Costa, S. Et al., 2016. Emotional storytelling using virtual and robotic agents . Retrieved May 2017 from: https://arxiv.org/pdf/1607.05327.pdf Coulson, M., 2004. Attributing emotion to static body postures: Recognition accuracy, confusions, and view point dependence. Journal of Nonverbal Behaviour: 28(2): 1-23. Erden, M. S., 2014. Emotional postures for humanoid nao robot. Int J Soc Robot 5: 441-446. Hyde, J. Et al., 2014. Assessing naturalness and emotional intensity: a perceptual study of animated facial motion. Proceedings of the ACM Symposium on Applied Perception. 1: 15-22. Paiva, A. Leite, I., Ribiero, T., 2014. Emotion Modelling for Social Robots. Retrieved May 2017 from: http://people.ict.usc.edu/~gratch/CSCI534/Readings/ACII-Handbook-Robots.pdf. Parker, R., 2012. The meaning of colours. Retrieved May 2017 from: https://resources.oncourse.iu.edu/access/content/user/rreagan/Filemanager_Public_Files/meaningofcolors.htm.