Supernumerary Hand Illusions in Multimodal Realities Nina Rosa - PowerPoint PPT Presentation

Self & Body Augmented Reality Further Research Supernumerary Hand Illusions in Multimodal Realities Nina Rosa Feb. 27, 2019

Self & Body Augmented Reality Further Research P ROGRAM Self & Body Augmented Reality Further Research

Self & Body Augmented Reality Further Research R UBBER H AND I LLUSION Botvinick, M., & Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature , 391(6669), 756. Image from: IJsselsteijn, W. A., de Kort, Y. A. W., & Haans, A. (2006). Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence: Teleoper- ators and Virtual Environments , 15(4), 455-464.

Self & Body Augmented Reality Further Research S ENSE OF E MBODIMENT * Body ownership is just one feeling connecting self and body. ◮ Ownership ◮ Self-location ◮ Agency Separate concepts, not independent *Kilteni, K., Groten, R., & Slater, M. (2012). The sense of embodiment in virtual reality. Presence: Teleoperators and Virtual Environments , 21(4), 373-387.

Self & Body Augmented Reality Further Research F ACTORS Recall these factors in the illusion: ◮ Anthropomorphism ◮ Movement/sensory synchronity ◮ Visual discontinuity ◮ Relative orientation

Self & Body Augmented Reality Further Research E XAMPLE FACTOR : P ERSPECTIVE Marc Owens, Avatar Machine, Talk to Me @ MoMA, 2011

Self & Body Augmented Reality Further Research R EPLACEMENT Typical illusion setups look at replacement scenarios IJsselsteijn, W. A., de Kort, Y. A. W., & Haans, A. (2006). Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence: Teleoperators and Virtual Environments , 15(4), 455-464. Normand, J. M., Giannopoulos, E., Spanlang, B., & Slater, M. (2011). Multisensory stimulation can induce an illusion of larger belly size in immersive virtual reality. PloS one , 6(1), e16128.

Self & Body Augmented Reality Further Research S UPERNUMERARY H AND I LLUSION Guterstam, A., Petkova, V. I., & Ehrsson, H. H. (2011). The illusion of owning a third arm. PloS one , 6(2), e17208.

Self & Body Augmented Reality Further Research I LLUSIONS IN VR What about AR? Won, A. S., Bailenson, J., Lee, J., & Lanier, J. (2015). Homuncular flexibility in virtual reality. Journal of Computer-Mediated Communication, 20(3), 241-259. Egeberg, M., Lind, S. L., Serubugo, S., Skantarova, D., & Kraus, M. (2016, March). Extending the human body in virtual reality: effect of sensory feedback on agency and ownership of virtual wings. In Proceedings of the 2016 Virtual Reality International Conference (p. 30). ACM.

Self & Body Augmented Reality Further Research S UPERNUMERARY H AND I LLUSION IN AR

Self & Body Augmented Reality Further Research T HIRD ARM - F IRST A TTEMPT ◮ How does MM feedback affect the SoE in an SHI in AR? ◮ hands, VT, VM ◮ 3 minutes ◮ 21-point Q ◮ data of 27 participants Gilbers, C. (2017) The Sense of Embodiment in Augmented Reality: A Third Hand Illusion. Master thesis, Utrecht University.

Self & Body Augmented Reality Further Research R ESULTS ◮ Overall low ownership scores, possibly due to bad virtual hand appearance emphasized by real environment ◮ Agency differences by active VM ◮ Number of hands did not play a role in agency ◮ No shift when 3 hands ◮ Possible extra location ◮ Shift when active VM

Self & Body Augmented Reality Further Research T HIRD ARM - S ECOND A TTEMPT ◮ New hand ◮ Same indep.vars. but not all combis ◮ 2 minutes + threat ◮ 9-point Q + SCR

Self & Body Augmented Reality Further Research N UMBER OF H ANDS C1 - 3 hands, synch VT, synch VM C2 - 2 hands, synch VT, synch VM

Self & Body Augmented Reality Further Research V ISUAL - TACTILE S YNCHRONY C3 - 3 hands, synch VT, no VM C4 - 3 hands, asynch VT, no VM

Self & Body Augmented Reality Further Research V ISUAL - MOTOR S YNCHRONY C5 - 3 hands, no VT, synch VM C6 - 3 hands, no VT, asynch VM

Self & Body Augmented Reality Further Research T HREAT

Self & Body Augmented Reality Further Research R ESULTS ◮ Generally, ownership and agency Q-responses were highest for C1 and C2, and lowest for C6. ◮ Location Q-responses were less positive, but found differences in shift/more locations depending on #hands. ◮ No differences in SCR data!? Not even after habituation correction! ◮ Combining Q-data and SCR-data found 2 groups: 1. low Q means, low Q sds, low-med SCR means 2. high Q means, high Q sds, low-high SCR means ◮ Hand realism

Self & Body Augmented Reality Further Research SCR 1.0 1.0 28 28 0.8 0.8 11 11 11 28 18 5 28 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 -0.2 -0.2 C1_logSCR_10b_10s C2_logSCR_10b_10s C3_logSCR_10b_10s C4_logSCR_10b_10s C5_logSCR_10b_10s C6_logSCR_10b_10s first_logSCR_10b_10s second_logSCR_10b_10s third_logSCR_10b_10s fourth_logSCR_10b_10s fifth_logSCR_10b_10s sixth_logSCR_10b_10s

Self & Body Augmented Reality Further Research F OLLOW - UP Q UESTION Does personality influence the extent of ownership in rubber hand illusions? Does ownership over a supernumerary virtual arm positively influence performance in a task using that arm? (For thesis starting approximately April 2019) Pilot: Test ownership factors in this setting (For small project starting approximately April 2019)

Self & Body Augmented Reality Further Research R ESEARCH C ONTEXT ◮ Player-Avatar Relation in Multimodal AR Games ◮ n.e.rosa@uu.nl Rosa, N. (2016, October). Player/Avatar body relations in multimodal augmented reality games. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (pp. 550-553). ACM.

Self & Body Augmented Reality Further Research S UMMARY 1. The rubber hand illusion paradigm allows us to examine the sense of embodiment: ownership, agency, self-location. 2. Our supernumerary hand illusions in AR suggest that: ◮ it is possible to own a third arm ◮ more synchronous cues may lead to greater ownership ◮ virtual hand appearance plays a role, possibly because the fakeness is emphasized by the real environment