Scientific domain Human-Computer Interaction Interaction Computer - PDF document

1 Usable Digital World Human-Computer Interaction Laurence NIGAY Laurence.Nigay@univ-grenoble-alpes.fr EHCI team - Engineering for Human-Computer Interaction 2 Scientific domain • Human-Computer Interaction Interaction Computer science Supported by contribution Social science • Designing, developing and evaluating interaction techniques • Development of conceptual and technical tools based on HCI principles: Utility, Usability, Context Laurence.Nigay@univ-grenoble-alpes.fr 1

3 Usable Digital World: Context • HCI in the context of Digital and Human Ecosystems – a seamless environment of computing The Computer for the 21 st Century 1991 – M. Weiser 4 Usable Digital World: Context Invisible technology Technology available at any place Symbiosis of the real and digital worlds Laurence.Nigay@univ-grenoble-alpes.fr 2

From WIMP - Windows Icons Menus Pointer Laurence.Nigay@univ-grenoble-alpes.fr 3

… to Post-WIMP Laurence.Nigay@univ-grenoble-alpes.fr 4

9 Usable Digital World : Context A seamless environment of computing The Xerox Star has Computers are everywhere reached its limits HCI is « out of the box » http://www.digibarn.com/ 10 Research themes Graphical User Interfaces System User Interface Multimodal Interaction Widget UI Adapatation Interaction technique Mixed Reality Interaction Device and sensory-motor phenomenon Mobile Interaction Laurence.Nigay@univ-grenoble-alpes.fr 5

11 Ubiquitous environments: Distant pointing 12 Distant pointing • Physical targets Laurence.Nigay@univ-grenoble-alpes.fr 6

13 Distant pointing • Digital targets 14 Distant pointing: digital targets Laurence.Nigay@univ-grenoble-alpes.fr 7

15 Pointing technologies 16 Pointing gestures • Fitts' Law: ID M ovement T ime, A mplitude, Target W idth, I ndex of D ifficulty W A Laurence.Nigay@univ-grenoble-alpes.fr 8

17 Pointing gestures • Initial Impulse Model [Meyers, 1988] : – Ballistic movement + Corrective sub-movements VELOCITY TIME (ms) 18 Pointing gestures • Initial Impulse Model [Meyers, 1988] : – Ballistic movement + Corrective sub-movements VELOCITY TIME (ms) Laurence.Nigay@univ-grenoble-alpes.fr 9

19 Pointing gestures • Initial Impulse Model [Meyers, 1988] : – Ballistic movement + Corrective sub-movements VELOCITY TIME (ms) 20 VTE (Voronoi Target Expansion) • Target expansion – Static global target augmentation • « Closest target » principle à Voronoi tessellation Laurence.Nigay@univ-grenoble-alpes.fr 10

21 VTE Rationale • Novice users – 10 pointings / surgery • Cognitive simplicity – Simple forms – Simple cursor – Visual stability 22 VTE (Voronoi Target Expansion) Laurence.Nigay@univ-grenoble-alpes.fr 11

23 Target expansion 24 Distant pointing: physical targets Laurence.Nigay@univ-grenoble-alpes.fr 12

25 Distant pointing: physical targets Direct physical interaction Handheld AR 26 Physical Object Selection: Disambiguating ? Laurence.Nigay@univ-grenoble-alpes.fr 13

27 Pointing Task in the Physical World Ray-Casting is Volume-based pointing difficult to improve in & disambiguation step the physical world Alternative 28 Balancing Focus And Performance User’s Focus Digital Physical Representation World è Design of focus-maintaining ? Disambiguation Techniques [Ailisto, 2006] [Välkkynen, 2006] Physical Pointing Roll ( P2Roll ) Physical Pointing Slide ( P2Slide ) Laurence.Nigay@univ-grenoble-alpes.fr 14

29 P2Roll 2" 1" 2" 1" Rolling" Current" Light"on" Rolling" Angle" 3" 3" Range" " 1 2" 3" 4" 4" Light"off" A) Poin0ng:" B)"Touching:"" selec0on"volume"" selec4on"volume"locked" 2" 1" 2" 1" 3" 3" " " 1 1 2" 2" 3" 3" 4" 4" C)"Rolling"Le.:"the"element"on" D)"Rolling"Right:"the"element"on" the"le."becomes"the"current"one" the"right"becomes"the"current"one" 30 P2Roll Laurence.Nigay@univ-grenoble-alpes.fr 15

31 P2Slide 32 Baseline: List Laurence.Nigay@univ-grenoble-alpes.fr 16

33 Design Rationale: User’s Focus User’s Focus Physical Digital 1" Representation World P2Roll List element"on" [Ailisto, 2006] [Välkkynen, 2006] P2Slide ement"on"the" 34 Experiment: Targets Densities: 2, 4, 8 and 16 targets Laurence.Nigay@univ-grenoble-alpes.fr 17

35 Results: Completion Time 36 Distant pointing: physical targets Direct physical interaction Handheld AR Laurence.Nigay@univ-grenoble-alpes.fr 18

37 AR: Pointing at physical targets • Specific to AR: – ‘Real’ AND ‘Virtual’ – Spatiotemporal relationship between the physical world and digital content • How to relax the spatial constraint while keeping physical/digital colocation? Framework • 4 entities Representation Touch Visual Physical of the physical surface augmentation world world Control space Display space Handheld device Laurence.Nigay@univ-grenoble-alpes.fr 19

39 Framework • 4 entities linked by spatial relationships Representation Touch Visual Physical of the physical surface augmentation world world Handheld device Registration jitter Hand tremor Motion induced by touches 40 Spatial mapping between the physical world and its representation • Adapt TapTap [Roudaut 08] to AR • Explicit and transient freeze rather than sustained • 2 views: one with freeze, the other with live video Represen Touc Visual tation of Physi- h augmen- the cal surfa- tation physical world ce world Handheld device Laurence.Nigay@univ-grenoble-alpes.fr 20

41 Spatial mapping between the physical world and its representation Represen Touc Visual tation of Physi- h augmen- the cal surfa- tation physical world ce world Handheld device 42 Spatial mapping between the physical world and its representation • Adapt Shift [Vogel 2007] with freeze-frame • Shift’s callout and cursor overcome the ‘fat finger’ problem • Freeze-frame avoids viewpoint instability • On-demand precise quasi-mode Represen Touc Visual tation of Physi- h augmen- the cal surfa- tation physical world ce world Handheld device Laurence.Nigay@univ-grenoble-alpes.fr 21

43 Spatial mapping between the physical world and its representation Represen Touc Visual tation of Physi- h augmen- the cal surfa- tation physical world ce world Handheld device 44 Framework • Frames of reference for pointing Frame of reference of the screen: instability Representation Touch Visual Physical of the physical surface augmentation world world Control space Display space Handheld device Laurence.Nigay@univ-grenoble-alpes.fr 22

45 Framework • Frames of reference for pointing Frame of reference of the physical object Representation Touch Visual Physical of the physical surface augmentation world world Control space Display space Handheld device 46 Frame of reference of the physical object Instrument Cursor ! Crosshair Relative Pointing No instrument Direct Touch On the object Frames of reference Screen Physical object Laurence.Nigay@univ-grenoble-alpes.fr 23

47 Frame of reference of the physical object Instrument Cursor ! Crosshair Relative Pointing No instrument Unstable finger Direct Touch On the object Frames of reference Screen Physical object Handheld AR : 48 Pointing at physical targets • 4 entities linked by spatial relationships • 2 frames of reference for pointing Representation Touch Visual Physical of the physical surface augmentation world world Handheld device Laurence.Nigay@univ-grenoble-alpes.fr 24

Distant pointing 49 in ubiquitous environment • Distant pointing: precision • Digital targets • Physical targets 50 Conclusion: new research axis • Deformable / Shape-changing User Interfaces Laurence.Nigay@univ-grenoble-alpes.fr 25

http://iihm.imag.fr/en/ Thank you for your attention Questions? Comments? Laurence.Nigay@univ-grenoble-alpes.fr 26