Interaction beyond Computation Michel Beaudouin-Lafon in|situ| Université Paris-Sud & INRIA Stanford University 1
Why this talk? 2
Why this talk? Interaction (not just human-computer interaction) is becoming a key factor of most computer systems Streaming algorithms, Anytime algorithms, Interactive grids, Cloud computing, Interactive proofs, Service-oriented architectures, etc. Human-computer interaction has stopped “thinking big” Lack of long-term visions Many incremental point designs, Few full-scale explorations, Integrative research 3
Overview Interactive Computation Interaction as phenomenon: multi-scale pointing and navigation Interaction as first class object: Instrumental interaction Interaction in the large: the WILD room 4
Interactive computation Non-algorithmic computational problem Dynamic interleaving of input & output streams Dependency on the environment Parallel computation of human and computer Non-computability of the environment 5
Non-algorithmic computational problem “Models of interaction capture the notion of performing a task or providing a service, rather than algorithmically producing outputs from inputs” Sometimes we rely on the computer being unable to solve the problem: MillionVis (Fekete, infovis02) 6
Dynamic interleaving of input and output streams “Interactions may consist of interleaved inputs and outputs modeled by dynamic streams; future input values can depend on past output values.” Feedback loops at multiple levels of scale Octopocus (Bau & Mackay, uist08) 7
Dynamic interleaving of input and output streams StreamLiner (Yuan, Tabard & Mackay, kam08) 8
Dependency on the environment “In models of interaction, the world or environment of the computation is part of the model and plays an active part in the computation by dynamically supplying the computational system, or agent, with inputs, and consuming the output values the system produces.” ABook (Mackay et al., uist01) 9
Parallel computation of user and computer “In models of interaction, computation may be concurrent; a computing agent can compute in parallel with its environment and with other agents.” Pick and drop (Rekimoto, uist97) 10
Non-computability of the environment “The environment cannot be assumed to be static or even effectively computable; for example, it may include humans or other real-world elements.” 11
What does this tell us? We need models, Environment frameworks and tools that account for interaction Interaction as phenomenon Interaction as Users Users Artifacts Artifacts Computers first-class object Integrative research 12
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law MT = k.ID, ID = log 2 (A/W + 1) Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot (Fitts, 1954) Sigma Lenses Wall pointing 13
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits MT = k.ID Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses (Guiard & Beaudouin-Lafon, hci01) Wall pointing 14
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size MT = k.ID / V Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses (Guiard, Beaudouin-Lafon, Wall pointing Bastin, Pasveer & Zhai, avi04) 15
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot (Appert & Fekete, chi06) Sigma Lenses Wall pointing 16
Interaction as phenomenon: multi-scale pointing and navigation 17
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Visual space Object pointing Dynaspot motor space Sigma Lenses (Blanch, Guiard & Beaudouin-Lafon, chi04) Wall pointing 18
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot (Guiard, Blanch & Beaudouin-Lafon, gi04) Sigma Lenses Wall pointing 19
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot (Chapuis, Labrune & Pietriga, chi09) Sigma Lenses Wall pointing 20
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses (Appert & Pietriga, chi08) Wall pointing 21
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses Wall pointing 22
Interaction as phenomenon: multi-scale pointing and navigation Fitts’ law beyond 10 bits Effect of view size Orthozoom Semantic pointing Object pointing Dynaspot Sigma Lenses Wall navigation (Nancel et al., chi’11) 23
Multi-scale pointing and navigation: summary Information transfer with a very tight feedback loop Interaction is directly observable Assisted pointing: what information can the system provide to improve pointing performance? 24
Interaction as first-class object: instrumental interaction (D’alembert & Diderot, L’Encyclopédie, 1751) 25
26
27
Instrumental interaction Two levels of interaction: mediation Instrument as extension of one’s body (Beaudouin-Lafon, chi00) 28
Instrumental interaction Covers many interaction styles: Traditional GUI Novel techniques Tangible interaction 29
Proof-of-concept: CPN2000 Bi-manual interaction, Marking menus, Toolglasses 40,000+ downloads 30
Instrumental interaction: Design principles Reification extends the notion of what constitutes an object Polymorphism extends the power of these commands with respect to these objects Reuse provides a wa of capturing and reusing interaction patterns (Beaudouin-Lafon & Mackay, avi00) 31
Reification: aligning objects Align command Align now and forget it Alignment instrument Align, and keep aligned 32
POLYMORPHISM AND REUSE Polymorphism Output Reuse Open-Close Copy-paste Cut-Copy-Paste Duplicate Undo-Redo Input Reuse Color picker Redo Macros 33
Instrumental interaction: Design principles Reification Reification and Reuse Reuse polymorphism: Interface Objects Objects more objects but fewer commands Reification facilitates output reuse Commands Polymorphism facilitates input reuse Polymorphism 34
Instrumental interaction: VIGO Ubiquitous instrumental interaction Separating instruments from objects Instruments that span multiple surfaces (Klokmose & Beaudouin-Lafon, chi09) 35
Interactions as first class objects: VIGO architecture Objects = state Object Governor Governors = rules Instrument Instruments = interaction View Manipulates Views = rendering Observes Decoupling and integration (Klokmose & Beaudouin-Lafon, chi09) 36
Integrative Research Interaction In-the-large 37
Integrative Research in HCI Putting together complete systems Testing validity in the field Working with extreme users Creating software tools (Xerox Star, 1981) 38
Integrative research: Metisse Window system to test windowing techniques with real applications (Chapuis & Roussel, uist05 + uist06, chi07, chi09) 39
Metisse: Façades 40
Integrative research: WILD 41
Our extreme users: Scientists 42
43
Exploring Multisurface Interaction 44
Interacting with complex data Navigate Compare Aggregate Communicate 45
Interacting with complex data Navigate Compare Aggregate Communicate 46
Substance : infrastructure for multisurface interaction The Wall B D B B B/D D B D B D B B/D D D D D B B B B/D D B/D B B D D B D D Behaviour Centric Behavior Oriented Data Oriented Data Centric Machine 1 Machine 16 Control Flow Information Flow PaperWall facet Data-oriented model Visualiser facet Replicated scene graph Sharable graph Multi-touch table Wall front-end/VICON controller Instruments Distributed Instruments PaperWall Master Table facet Collection Application facet of papers Application teleportation Scene graph Replicated scene graph (Scotty) Input Papers Devices 47
WILD: multisurface interaction 48
One step beyond: DIGISCOPE 9 interactive rooms interconnected by high-end videoconferencing Applications to scientific discovery, product life-cycle management, decision making, crisis management, training and education 49
Summary A paradigm shift: From algorithmic to interactive computation An interaction model: Instrumental interaction - interaction as first-class object Studying interaction at all scales: From low-level phenomena to integrative environments An overarching goal: Generative theories to create the next generation of interactive systems 50
Recommend
More recommend
