Perception and Animation of Body and Hand Motions for Realistic Virtual Characters Sophie J¨ org Postdoctoral Researcher Carnegie Mellon University Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Virtual Characters Are Increasingly Common Final Fantasy (2001) Museumguides [Swartout10] NBA2K11 (2011) The Sims 3 (2009) Beowulf (2007) Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
People Are Very Good At Recognizing Human Motions Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
More Detail Is Not Always Better Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Not All Virtual Characters Are Engaging The Polar Express (2004) Mars Needs Moms (2011) The Polar Express: . . . cold and distanced animation. . . (EmanualLevy.Com) . . . More like Night of the Living Dead . . . (Bangor Daily News) Mars Needs Moms: . . . waxy, inexpressive faces. . . (New York Times) . . . the dehumanized child characters . . . (ColeSmithey.com) . . . affectless, mechanical cartoons, . . . (Film.com) Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
The Uncanny Valley Hypothesized Uncanny Valley graph, redrawn from [Mori 70] Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
The Uncanny Valley Hypothesized Uncanny Valley graph, redrawn from [Mori 70] Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
The Uncanny Valley Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Research Goals • Understand how people perceive and interpret human-like motions. • Develop better algorithms to create virtual characters. Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Hand and Finger Movements • integral and crucial part of language and communication [Kendon04, McNeill92] • very hard to capture and often animated separately [Majkowska et al. 04] or only a few degrees of freedom are captured [Kitagawa and Windsor 08] GreenDot Project, Williams et al. 10 Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
How to Animate Fingers • Motion capture • Keyframe animation • Algorithms Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
How to Animate Fingers • Motion capture → not always accurate [Kahlesz et al. 2004] , needs post-processing [Kitagawa and Windsor 2008] • Keyframe animation • Algorithms CyberGlovesII Capturing A Christmas Carol Markerset for hands Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
How to Animate Fingers • Motion capture • Keyframe animation → time-consuming • Algorithms Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
How to Animate Fingers • Motion capture • Keyframe animation • Algorithms → rule-based [ElKoura03] , physics-based [Pollard05, Liu08, Liu09] , data-driven [Majkowska06] Liu08 Liu09 Pollard05 Majkowska06 ElKoura03 Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Finger Perception: Threshold Experiment Are errors in finger motion noticeable? We investigate synchronization errors between finger and body motion, which are common when hand and body are captured separately. published in The Perception of Finger Motions , APGV 2010 Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
“Is this motion MODIFIED or NOT MODIFIED?” Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Design Four actions: • Count, Drink, Snap, Point Three versions of each action Four levels of synchronization error: • 0.1s, 0.2s, 0.3s, 0.4s Total of 60 different clips Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Result: Errors As Small As 0.1s Can Be Recognized proportion of motions rated as unmodified level of error Count Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Result: Error Perception Depends on the Motion proportion of motions rated as unmodified level of error Point Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Theshold Experiment Summary Participants recognize errors as little as 0.1 seconds → Correct timing of hand motions is very important ֒ Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Finger Perception: Interpretation Experiment Does Correct Finger Motion Matter? Investigate the impact of incorrect finger motion on the interpretation of a scene published in The Perception of Finger Motions , APGV 2010 Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
New Approach: Vignettes Animated movies that tell a story and convey emotional content. Previous studies: • simple scenarios • little or no affective content • existing videos Our approach: • tell a story • elicit an emotional response • ask higher level questions, e.g., interpretation of a scene Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Why Vignettes? Closer to real experience. Investigate higher-level questions. Find effects that are impossible to measure with short snippets of motion. Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Stimuli: Original Motion Captured Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Stimuli: Finger Joints Delayed by 0.5 Seconds Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Design Between subjects design: each group only saw one of the versions • original: 47 participants • delayed: 69 participants Questionnaire: • empathy with the character • factual questions • describe the scene • quality Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Data Analysis Five-point scale response data: between groups ANOVA significant differences? no empathy questions no factual questions no quality Descriptions in free form text: original delayed 21% 1% character using ctrl-alt-del 40% 15% computer freezing Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Interpretation Experiment Summary Finger motion can alter the interpretation of a scene, even without altering its perceived quality. Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Finger Perception: Animation Method Experiment What animation method should we recommend? original: motion captured no hands: no motion, stiff hands in a natural pose keyframe: imitating motion capture with limited number of keyframes random: small idle motion captured movements published in The Perception of Finger Motions , APGV 2010 Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
“In which clip was the motion of better quality?” Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Stimuli and Design Four types of finger animations: • original, no hands, keyframe, random Four scenes out of a narration Two types of viewpoint: • long shot, close-up (hip to neck) Pairwise comparisons of animation types 48 pairs of clips in total Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Result: Motion Capture Rated Best proportion of motions rated as having the better quality animation type Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Animation Method Experiment Summary The motion captured movements are considered to have the best quality Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Finger Perception Summary Subtle errors in finger motions can be perceived. They can change the interpretation of a movie. Motion captured movements are considered to have the best quality. Finger motions are hard to capture. Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Facial Animation and Sound Investigate the effect of facial animation and sound on the emotional content and interpretation of a scene Used three vignettes, about 30 seconds each Published in The Saliency of Anomalies in Animated Human Characters , TAP 2010 Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Create Stimuli Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Stimuli: Milk – Original Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Results Sound and motion interacted with content in complex ways. Results could not have been found with short stimuli. Even unrelated choices of stimuli (e.g., sound/no sound) can influence results. Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Questions? Recognizing Human Motions The Uncanny Valley Finger Perception: Threshold Experiment Finger Perception: Interpretation Experiment Finger Perception: Animation Method Experiment Finger Motion Synthesis Facial Animation and Sound Future Research Sophie J¨ org Perception and Animation of Virtual Characters Feb 16, 2012
Recommend
More recommend
