PRE-TOUCH SENSING for MOBILE INTERACTION Ken Hinckley Seongkook - PowerPoint PPT Presentation

PRE-TOUCH SENSING for MOBILE INTERACTION Ken Hinckley Seongkook Heo (KAIST & MSR) Michel Pahud Christian Holz Hrvoje Benko Abigail Sellen Gavin Smyth Richard Banks Kenton O’Hara Bill Buxton

PRE-TOUCH SENSING for MOBILE INTERACTION Seongkook Heo (KAIST & MSR) Ken Hinckley Michel Pahud Christian Holz Hrvoje Benko Abigail Sellen Gavin Smyth Richard Banks Kenton O’Hara Bill Buxton

IF NATURAL HUMAN TOUCH IS ANALOG and CONTINUOUS…

(x,y) Then why do we boil the entire vocabulary of mobile touch down to flatland?

Much of what characterizes ‘touch’ starts before contact and originates beyond the confines of the screen.

FROM TOUCH AS A POINT, AN INSTANT IN TIME…

TO A WHOLE WORLD THAT SURROUNDS IT…

This treasury of contextual detail — the combination of hover and grip that we collectively refer to as pre-touch — has been lost to mobile interaction.

SENSOR fogale-sensation.com /technology – Self-capacitance is well-known. Not our contrib.

You’ve probably heard about hover, or seen examples of control gestures in-air.

You’ve probably heard about hover, or seen examples of control gestures in-air. The focus of pre- touch isn’t on such overt, ‘foreground’ gestures.

You’ve probably heard about hover, or seen examples of control gestures in-air. The focus of pre- touch isn’t on such overt, ‘foreground’ gestures. Instead, pre-touch is primarily about the ‘background’ of the interaction—

You’ve probably heard about hover, or seen examples of control gestures in-air. The focus of pre- touch isn’t on such overt, ‘foreground’ gestures. Instead, pre-touch is primarily about the ‘background’ of the interaction— what happens before, the framing, the context.

This is a design space paper.

This is a design space paper. We explore an emerging modality, breadth-first across a number of possibilities.

This is a design space paper. We explore an emerging modality, breadth-first across a number of possibilities. But we do not go depth-first into any one technique in exhaustive detail.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch, simultaneous with hover

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out ( 1.2s) but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway… take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway, take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: ad-lib interface : key ideas – when fingers away, no UI. Only content. – approach suggests intent to engage – so we fade in (0.2s) … later fade out (1.2s) but… – since we’re fading in anyway, take liberty – to fade in context-appropriate controls.

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: multiple grip + hover contexts

even adaptation of control gestures (“dialing”)

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS ad-lib interface calm revelation approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS ad-lib interface calm revelation approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS ad-lib interface calm revelation approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, single-finger distance above touchscreen WE EXPLORED: 30mm temporal trajectory 20mm for a large target interface contingent on grip and hover temporal trajectory 10mm 2) RETROACTIVE INTERPRETATIONS for a small target approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES -1s -800ms -600ms -400ms -200ms 0ms (touch down) on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch + above-screen aspects

TO ILLUSTRATE THIS DESIGN SPACE, WE EXPLORED: 1) ANTICIPATORY REACTIONS interface contingent on grip and hover 2) RETROACTIVE INTERPRETATIONS approach trajectory informs touch semantics 3) HYBRID TOUCH-HOVER GESTURES on-screen touch simultaneous with hover