in Virtual Environments Representing People Representing People - PDF document

Will Steptoe 30 th November 2010 in Virtual Environments Representing People Representing People

What’s in this lecture? • Part 1 : Virtual Characters – History, Agency, Control in Immersive and Non- History Agency Control in Immersive and Non Immersive systems, Copresence and measures, Fidelity Uncanny Valley Fidelity, Uncanny Valley. • Part 2 : 3D Studio Max Demo • Part 3 : Technical Aspects of Virtual Characters – Motion Capture, Skeletal Subspace Deformation, Forward Kinematics, Inverse Kinematics, Blend Shapes

Early Virtual Characters • “Mechanical Turk” chess-playing machine, 1770. • Instead of just a machine a human figure is presented Instead of just a machine, a human figure is presented. • Makes experience more compelling, provides a focus for visual attention relates to theory of social agency for visual attention, relates to theory of social agency.

Social Agency - General • “Individuals mindlessly apply social rules and expectations “I di id l i dl l l i l l d t ti to computers”, Nass and Moon. • People generally require minimal encouragement to view • People generally require minimal encouragement to view computer systems and applications as social agents, reading far more understanding than is warranted from reading far more understanding than is warranted from symbols and graphical displays. ELIZA, Weizenbaum, 1996

S Social Agency - Eliza i l A Eli • • First documented example is ELIZA: a computer program for the First documented example is ELIZA: a computer program for the study of natural language communication between man and machine (Weizenbaum, 1966). • ELIZA used text-processing to rephrase input statements from users into questions. People often became emotionally engaged when “communicating” with ELIZA, and some even asked to be h “ i ti ” ith ELIZA d k d t b left alone with the system. Often termed the “ELIZA effect”. • Due to tendency for humans to unconsciously equate Due to tendency for humans to unconsciously equate programmed computer behaviour as analogous to conscious human behaviour despite conscious knowledge to the contrary. • May be considered a precursor to many observations of immersion and presence reported in the VE literature.

A Agency – Virtual Characters Vi t l Ch t • In the specific context of software-based virtual humanoids • In the specific context of software-based virtual humanoids, agency describes their method of control or interaction, with avatars and agents occupying either end of the agency- g py g g y spectrum. • Agency is the extent to which a virtual human is perceived by to be a representation of an individual in the ‘real’ world. • Avatar/agent hybrids are common.

Agency – An Issue of Control • For agents the behaviour is completely programmed. • For avatars the behaviour is ideally completely determined by the behaviour of the real tracked human human. • In practice the human cannot be fully tracked – typically in VR only head and one hand typically in VR only head and one hand movements are tracked!

Control Methods Control Methods • Typed Text, Emoticons, Traditional GUI, Speech, yp , , , p , Full body tracking

Minimal Tracking for IK in VR • Badler et al showed a minimal configuration for IK representing fi ti f IK ti the movements of a human in VR – www.cis.upenn.edu/ – www cis upenn edu/ ~hollick/presence/presence.html • It was shown that 4 sensors are sufficient to reasonably reconstruct the approximate body configuration in real-time. fi ti i l ti

E Embodiment in Collaborative Virtual b di t i C ll b ti Vi t l Environments (CVEs) ( ) • In shared VEs, users’ avatar embodiments act as the fundamental mediators of the visual component of an p interaction. • Avatars function both to identify users and to communicate nonverbal behaviour including position, identification, focus of attention, gesture and action. • Avatars generally exhibit generic humanoid form, which A t ll hibit i h id f hi h reflects their status as a representation of a human user, and critically enables a natural mapping between a user’s and critically, enables a natural mapping between a user s bodily movement and the corresponding virtual behaviour. • Avatars that exhibit humanoid form and behaviour have Avatars that exhibit humanoid form and behaviour have been shown to evoke a richer sense of copresence in observers.

Immersive Collaborative Virtual Immersive Collaborative Virtual Environments (ICVEs / CVEs)

Controlling Avatars in Non-Immersive CVEs: Spark (Morley D and Myers K Spark (Morley, D. and Myers, K., 2004) 2004) • Text Chat based environment environment • Parse users text input for interactional input for interactional information • Use this information to generate behaviour

Controlling Avatars in Non-Immersive CVEs: Spark Spark

Problems with Controlling Avatars in Non- Immersive Systems Immersive Systems • Two modes of control: at any moment the user must y choose between either selecting a gesture from a menu or typing in a piece of text for the character to say. This means the subtle connections and synchronisations means the subtle connections and synchronisations between speech and gestures are lost. • Explicit control of behaviour: the user must consciously p y choose which gesture to perform at a given moment. As much of our expressive behaviour is subconscious the user will simply not know what the appropriate behaviour user will simply not know what the appropriate behaviour to perform at a give time is [BodyChat, Vilhjalmsson, H. and Cassell, J., 1998]

Problems with Controlling Avatars in Non- Immersive Systems Immersive Systems • Emotional displays: current systems mostly concentrate on displays of emotion whereas Thórisson and Cassell (1998) have shown that envelope displays – subtle gestures and actions that regulate the flow of a dialog and establish mutual focus and attention mutual focus and attention – are more important in are more important in conversation. • User tracking: direct tracking of a user’s face or body does not help as the user resides in a different space from that of the avatar and so features such as direction of gaze will not map over appropriately not map over appropriately. [BodyChat, Vilhjalmsson, H. and Cassell, J., 1998]

Solutions for Controlling Avatars in Non- Immersive Systems Immersive Systems • Always ensure that any control is done through a single interface (e g through text chat) single interface (e.g. through text chat) • BUT…. • The body language of an avatar should be largely autonomous, and indirectly controlled by users • Minimize the level of control needed [BodyChat, Vilhjalmsson, H. and Cassell, J., 1998]

Copresence • Referred to as Copresence in the CVE literature, referred to as Social Presence in general telecommunications. g • Theory of social presence in telecommunication systems is the degree of salience of another person taking part in the interaction, with a particular emphasis on how the transmission of nonverbal cues is supported by the medium (Short 1976) (Short 1976). • In multi-user VEs, it is the sense of being in the company of another person during the course of mediated interaction another person during the course of mediated interaction. • The term is parallel to the established usage of ‘presence’ that entails the sense of being present in a VE. that entails the sense of being present in a VE.

Measuring Copresence • Success of a VE is measured in terms of the extent to S f VE i d i t f th t t t which sensory data projected within a virtual environment replaces the sensory data from the physical world – quantified by rating the individuals’ sense of presence during the experience • For virtual characters: success is taken as the extent to which participants act and respond to the agents as if they were real – Subjective: Questionnaires, Interviews Subjective: Questionnaires, Interviews – Objective: Physiological, Behavioural

Subjective means • Traditional methods: Questionnaires and • Traditional methods: Questionnaires and interviews – Various questionnaires exist Various questionnaires exist – http://www.presence-research.org • Criticised due to its various dependencies Criticised due to its various dependencies – the individual’s accurate post-hoc recall, – processing and rationalisations of their experience in processing and rationalisations of their experience in the VE and – Varying interpretations of the word ‘presence’