Mobile Technologies context and task theory interaction techniques - PowerPoint PPT Presentation

Mobile Technologies context and task theory interaction techniques in/output technologies 1 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Sensors context and task • gyroscope theory • accelerometer interaction • magnetometer techniques in/output technologies • always available mobile input sensors – SixthSense project – EMG (Skinput) HMD fabrication teaching shape-changing mobile projection 2 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile SixthSense Project context and task • vision-based approach for always available theory input interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing https://www.youtube.com/watch?v=JnxGUPzP_U0 mobile projection Literature: Mistry, P . : Wear Ur World - A Wearable Gestural Interface, CHI 2009 3 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Biosensors context and task • signals traditionally used for diagnostic theory medicine interaction – emotional state: heart rate, skin resistance techniques – brain sensing technology cognitive and emotional state (BCI - brain computer interaction): in/output technologies • electrocephalography (EEG) • functional near-infrared spectroscopy (fNIR) sensors – signals generated by muscle activation HMD • electromyography (EMG) fabrication • bone conduction – sound frequencies propagate well through bone teaching shape-changing mobile projection 4 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Brain-Computer Interfaces context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing mobile contact Mariam.Hassib@ifi.lmu.de projection 5 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile The *annoyingly* talking window context and task theory interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing https://www.youtube.com/watch?v=azwL5eoE5aI mobile projection 6 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Skinput context and task • Bio-sensing approach theory – leverages natural acoustic conduction properties of the human body interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing https://www.youtube.com/watch?v=g3XPUdW9Ryg mobile projection 7 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Head-Mounted Display context and task • Principles theory – closed HMD interaction – VR with HMD techniques – AR with see-through in/output • video technologies • optical sensors – google glasses • Head-up Displays with head tracking HMD fabrication • oculus rift teaching shape-changing mobile projection 8 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Google Glasses Google glasses and augmented reality context and task • c theory http://www.tomsguide.com/us/ google-glass,news-17711.html interaction techniques in/output technologies sensors HMD fabrication teaching shape-changing Ivan Sutherland, 1968 mobile projection http://louislau91.files.wordpress.com/2013/06/sword_of_damocles.jpeg 9 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Principle: closed (video only) HMD context and task • Monitor is mounted very close to the eye theory • Additional lens makes it appear distant interaction • è all images appear at the same distance techniques – Usually at infinity or slightly less in/output technologies sensors HMD fabrication teaching shape-changing mobile projection 10 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Creating VR with a HMD context and task theory interaction techniques in/output technologies Head Rendering sensors tracker HMD fabrication 3D scene teaching shape-changing mobile projection 11 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Challenges with HMDs in VR context and task • Lag and jitter between head motion and theory motion of the 3D scene interaction – Due to tracking à predictive tracking techniques – Due to rendering à nowadays mostly irrelevant in/output • Leads to different motion cues from technologies – Eye (delayed) and sensors – Vestibular system (not delayed) HMD • Result: cyber sickness fabrication teaching shape-changing mobile projection 12 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Mobile Oculus Rift context and task • costom-built motion and orientation sensor theory unit interaction – gyroscope techniques – accelerometer in/output – magnetometer technologies • head tracking with 3 degrees of freedom sensors HMD fabrication teaching shape-changing mobile projection http://static3.businessinsider.com/image/ 51d1949aecad04ad3e000000/heres-what-happened-when-we- strapped-a-bunch-of-people-into-the-oculus-rift-virtual-reality- headset.jpg 13 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Creating AR with video see-through Mobile HMDs context and task theory interaction techniques Video Mixing in/output technologies Head Rendering sensors tracker HMD Parallax error fabrication 3D scene teaching shape-changing mobile projection 14 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Advantages of video-based see- Mobile through context and task • Lag between physical and virtual image theory can be compensated interaction • Camera can be used for tracking as well techniques –Physical image = raw tracking data in/output –Perfect registration possible technologies • Video mixer can add or subtract light sensors –Virtual objects can be drawn in black HMD –Physical objects can be substituted fabrication –Virtual objects can be behind physical teaching objects • Just one image with a given focus shape-changing distance mobile projection 15 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Challenges of video-based see- Mobile through context and task theory • Lag between physical and virtual image can be compensated interaction techniques – … by delaying the physical image – Leads back to the cyber sickness problem in/output technologies • Parallax error can not be corrected electronically sensors – Wrong stereo cues when used for stereo HMD • Richness of the world is lost fabrication – Video image just 0.5 megapixels – Resolution of human vision is much higher (>10x) teaching shape-changing mobile projection 16 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14

Creating AR with optical see-through Mobile HMDs context and task theory interaction techniques in/output technologies Head Rendering sensors tracker HMD fabrication 3D scene teaching shape-changing mobile projection 17 LMU München — Medieninformatik — Andreas Butz, Julie Wagner — � Mensch-Maschine-Interaktion II — WS2014/15 Slide Thursday 27 November 14