Eric Lecolinet LTCI Tlcom ParisTech - PowerPoint PPT Presentation

HABILITATION A DIRIGER DES RECHERCHES Eric Lecolinet LTCI – Télécom-ParisTech 10 December 2018 Eric Lecolinet – LTCI Télécom ParisTech 1

Preamble Research domains § Cursive script recognition § Graphical toolkits and software architecture § Information visualization § Tactile feedback § Target acquisition on small devices § Augmented reality § Wall-sized displays § Gestural interaction Eric Lecolinet – LTCI Télécom ParisTech 2

Why using gestures? Eric Lecolinet – LTCI Télécom ParisTech 3

Analogical user interfaces Analogy § A thing which is comparable to something else in significant respects ‘works of art were seen as an analogy for works of nature’ (oxforddictionaries.com) Eric Lecolinet – LTCI Télécom ParisTech 4

Analogical user interfaces Idea § Metaphors make UIs 'intuitive' § "Intuitive = familiar" [Raskin94] § Tremendously successful! Eric Lecolinet – LTCI Télécom ParisTech 5

Analogical user interfaces Limitation § Efficient for novice users § Not so for expert users § Everyone is an expert of something! Eric Lecolinet – LTCI Télécom ParisTech 6

Analogical user interfaces Reasons § Don't leverage user knowledge § Based on recognition : (search) / point / recognize / click § Transient objects § Limited syntactic capabilities Eric Lecolinet – LTCI Télécom ParisTech 7

Analogical user interfaces Small devices § Occlusion, accuracy § Mobility conditions § Eyes-free interaction Large devices § Large gestures § Users must constantly walk Solution? Eric Lecolinet – LTCI Télécom ParisTech 8

Hotkeys Limitations § Only on the PC § Explicit learning (different modality) § Limited expressivity (name collisions) § Hard to customize Consequence § Largely underused [Lane05] Eric Lecolinet – LTCI Télécom ParisTech 9

Users are not rational! Paradox of the active user [Carroll87] § Users tend to favor short-term solutions • Even if they know more efficient method s exist [Krisler08] • True in economics [Goodwin18] ... and HCI ! Changing modalities § Performance drop [Scarr11] Solution? Eric Lecolinet – LTCI Télécom ParisTech 10

Recall vs. recognition Speech interaction § Rely on recall of prior knowledge § Direct access § Rich syntax Eric Lecolinet – LTCI Télécom ParisTech 11

Recall vs. recognition Limitations § Slow § Privacy concerns § So 'natural'? Solution? Eric Lecolinet – LTCI Télécom ParisTech 12

Gestural interaction! Advantages § Recall => direct access § Fast § Possibly eyes-free § Limited conflicts Problem § "Most gestures are neither natural nor easy to learn or remember " [Norman10], Solution? Eric Lecolinet – LTCI Télécom ParisTech 13

'Natural ' gestures User-defined gestures § Elicitation studies § "Ask users to perform the gestures causing given effects" [Wobbrock09] Limitations § Few commands elicit agreement § Especially abstract commands Eric Lecolinet – LTCI Télécom ParisTech 14

'Natural ' gestures Nature dimension [Wobbrock09] § Physical gestures § Metaphorical gestures § Symbolic gestures § Abstract gestures Eric Lecolinet – LTCI Télécom ParisTech 15

'Natural ' gestures Nature dimension [Wobbrock09] § Physical gestures § Metaphorical gestures § Symbolic gestures § Abstract gestures physical or metaphorical? abstract, symbolic, metaphorical? symbolic or abstract? Eric Lecolinet – LTCI Télécom ParisTech 16

'Natural ' gestures Analogue-abstract spectrum [Zhai12] § "The more analogous gestures are to the user’s prior experience , the easier they are to learn" Eric Lecolinet – LTCI Télécom ParisTech 17

'Natural ' gestures Analogue-abstract spectrum [Zhai12] § "The more analogous gestures are to the user’s prior experience , the easier they are to learn" It's all about familiarity! § "Intuitive = familiar" [Raskin94] § 'Natural ' have nothing to do with 'nature' ! § Relative/evolving notion : users have a brain, they do learn ! Eric Lecolinet – LTCI Télécom ParisTech 18

Useful gestures Useless § Whatever can be done easily § Avoid solving problems that do not exist! Useful § Small and large devices § User or application specific tasks : • Find and Select Previous • Open the NYT webpage • Open XCode on this file • Show CNN on my TV, etc. Eric Lecolinet – LTCI Télécom ParisTech 19

Useful gestures must be learnt! Research questions § How to to increase the expressivity of gestures? § How to improve the learning and retention of gestures? (and of gesture/command associations) § How to design efficient + user-friendly interaction techniques? Eric Lecolinet – LTCI Télécom ParisTech 20

Outline § Expressivity of gestures § Memorization of gestures & gesture/command associations § Conclusion § Perspectives Eric Lecolinet – LTCI Télécom ParisTech 21

First part: Expressivity of gestures Eric Lecolinet – LTCI Télécom ParisTech 22

Gestures = a large resource deposit § Dimensionality (2D, 3D) § Frame of reference: § Shape • Movements on the device • Movements around the device § Kinematics • Movements of the device § With or without friction § Pression § Multitouch § etc. A few examples... Eric Lecolinet – LTCI Télécom ParisTech 23

Marking menus Directional radial gestures § Simple and fast Learning by repeating § Implicit learning Eric Lecolinet – LTCI Télécom ParisTech 24

Marking menus Limitations § Limited breadth § No inner groupings Eric Lecolinet – LTCI Télécom ParisTech 25

Shape > curvature Flower menus § Gilles Bailly, Laurence Nigay Three dimensions § Direction § Curvature § Curvature direction Advantages § Time efficient § Large cardinality (56 commands) § Semantic groupings Eric Lecolinet – LTCI Télécom ParisTech 26