ubiquitous computing and augmented realities virtual and augmented - PDF document

ubiquitous computing and augmented realities chapter 20 • ubiquitous computing – filling the real world with com puters ubiquitous computing and augmented realities • virtual and augmented reality – m aking the real world in a computer! Challenging HCI Assumptions Ubiquitous Computing • What do we imagine when we think of a • Any computing technology that permits computer? human interaction away from a single workstation “The most profound technologies are those that disappear.” • Implications for Weiser – Technology defining the interactive experience • 1990’s: this was not our imagined – Applications or uses computer! – Underlying theories of interaction Scales of devices Device scales • Inch • Weiser proposed – PDAs – Inch – PARCTAB – Foot – Voice Recorders – Yard – sm art phones • Implications for device size as well as • Individuals own many relationship to people of them and they can all communicate with each other and environment. 1

Device scales Device scales • Foot • Yard – notebooks – electronic whiteboards – tablets – plasm a displays – digital paper – smart bulletin boards • Individual owns • Buildings or several but not institutions own them assumed to be and lots of people always with them. share them. Defining the Interaction Different Inputs Experience • Implicit input – Sensor-based input – Extends traditional explicit input (e.g., keyboard and mouse) – Towards “awareness” – Use of recognition technologies – Introduces ambiguity because recognizers Sensors on a PDA Capacitive sensing on a table are not perfect Multi-scale and distributed The output experience output • Screens of many sizes • More than eye-grabbing raster displays – (very) small – Ambient: use features of the physical environment to signal information – Peripheral: designed to be in the – (very) large background • Examples: – The Dangling String – The Water Lamp (shown) • Distributed in space, but coordinated 2

Merging Physical and Digital Application Themes Worlds • How can we remove • Context-aware computing A “digital” desk the barrier? – Sensed phenom ena facilitate easier – Actions on physical interaction objects have • Automated capture and access meaning electronically, and – Live experiences stored for future access vice versa • Toward continuous interaction – Output from – Everyday activities have no clear begin-end electronic world conditions superimposed on physical world An augmented calendar New Opportunities for Theory Evaluation Challenges • Knowledge in the world • How can we adapt other HCI techiques – Ubicom p places m ore em phasis on the physical world to apply to ubicomp settings? • Activity theory – Ubicom p activities not so task-centric – Goals and actions fluidly adjust to physical state of world – Technologies are so new, it is often hard to • Situated action and distributed cognition get long-term authentic sum m ative evaluation – Em phasizes im provisational/ opportunistic behavior versus planned actions – Metric of success could be very different • Ethnography (playfulness, non-distraction versus – Deep descriptive understanding of activities in efficiency) context ambient wood City - shared experience • real wood! … filled with electronics • visitors to Mackintosh Interpretation Centre – som e on web, som e use VR, som e really there • light and m oisture m eters • interacting – recorded with GPRS location – drawn on m ap later – talk via m icrophones – ‘see’ each other virtually • ‘periscope’ – shows invisible things – uses RFI D • different places • triggered sound • different m odalities • shared experience 3

virtual and augmented reality VR - technology & experience web, desktop and simulators AR – mixing virtual and real virtual reality technology VR headsets • headsets allow user to “see” the virtual world • small TV screen for each eye • gesture recognition achieved with DataGlove • slightly different angles (lycra glove with optical sensors that measure hand and finger • 3D effect positions) • eyegaze allows users to indicate direction with eyes alone • whole body position sensed, walking etc. immersion on the desktop • VR • headset VR – com puter sim ulation of the real world – expensive, uncom fortbale • mainly visual, but sound, haptic, gesture too • desktop VR – experience life-like situations – use ordinary m onitor and PC • too dangerous, too expensive • cheap and convenient – see unseen things: • in gam es … • too small, too large, hidden, invisible • and on the web – e.g. manipulating molecules – VRML – virtual reality m arkup language • the experience – aim is im m ersion, engagem ent, interaction 4

VRML … VR on the web command and control #VRML V1.0 ascii • scenes projected on walls Separator { • realistic environment Separator { # for sphere Material { • hydraulic ram s! emmissiveColor 0 0 1 # blue • real controls } Sphere { radius 1 } • other people } Transform { translation 4 2 0 } Separator { # for cone Texture2 { • for: filename "big_alan.jpg" } – flight sim ulators Cone { – ships radius 1 # N.B. width=2*radius height 3 – m ilitary } } } augmented reality (AR) applications of AR • images projected over the real world maintenance – aircraft head-up display – overlay instructions – sem i-transparent goggles – display schematics – projecting onto a desktop • types of information examples – unrelated – e.g. reading email with wearable – related – e.g. virtual objects interacting with world – photocopier engineers • issues • registration critical arrows point to parts – registration – aligning virtual and real – aircraft wiring looms – eye gaze direction • registration perhaps too hard, use schem atic applications of VR • sim ulation – gam es, m ilitary, training information and data visualisation • VR holidays – rainforest, safari, surf, ski and m oon walk … all from your own arm chair • m edical VR, 3D and 2D displays – surgery • scans and x-rays used to build model scientific and complex data then ‘practice’ operation • force feedback best interactivity central – phobia treatm ent • virtual lifts, spiders, etc. 5

scientific and technical data virtual wind tunnel • number of virtual dimensions that are ‘real’ • fluid dynam ics to sim ulate air flow • three dimensional space • virtual bubbles used to show movements – visualise invisible fields or values – e.g. virtual wind tunnel • ‘better’ than real • two dimensional space wind tunnel … – can project data value up from plane – no disruption of – e.g. geographic data air flow – N.B. viewing angle hard for static visualisation – cheaper and faster • no ‘real’ dimensions – 2D/ 3D histogram s, scatter plots, pie charts, etc. structured informnation visualising hiererchy • scientific data – just num bers • 2D organisation chart – fam iliar representation • inform ation system s … lots of kinds of data – what happens when it gets wide? • hierarchies managing – file trees, organisation charts director • networks – program flow charts, hypertext structure sales marketing production • free text … manager manager manager – docum ents, web pages F. Bloggs F. Bloggs A. Jones R.Carter K. West B. Firth J. Smith P. Larkin wide hierarchies … use 3D? networks in 2D managing director • network or ‘graph’: – nodes – e.g. web pages sales marketing production – links – m ay be directed or not – e.g. links manager manager manager • planar – can drawn without crossing F. Bloggs F. Bloggs A. Jones R.Carter K. West B. Firth • non-planar – any 2D layout has crossings J. Smith P. Larkin • cone trees (Xerox) • levels become rings • overlap ‘OK’ in 3D Planar graph Non-planar graph 6

time and interactivity • visualising in tim e – time dimension mapped to space – changing values: sales graphs, distance-time – events: Gantt chart, timelines, historical charts e.g. Lifelines – visualising medical and court records • using tim e – data dimension mapped to time – time to itself: fast/ slow replay of events – space to time: Visible Human Project • interactivity – change under user control e.g. influence explorer between two worlds • ubiquitous computing – com puters fill the real world • virtual reality and visualisation – real world represented in the com puter • augmented reality, ambient displays … – physical and digital interm ingled …m aturity – VR and visualisation – com m onplace – AR, ubiquity … com ing fast! 7