Assignment 3.5: New Lab Room Group Work Definitions & Theory Mediality, Codality & Form groups of 3-6 Modality Context Develop the outline of a project idea to change A120 into a Ambient Intelligent Systems room you would like to use: Descriptive Framework & Today, traditional computer lab Examples How to change it? Implementing & Evaluating Interior decor Furniture Human & Computer Technology Computing & Possible technologies: Culture Tab, Pads & Boards Required Reading Behavioural interfaces Natural language processing Pitch your idea in the course SoSe 2018 Jörg Cassens – Summary 16 / 139
Required Reading Tie-ins Definitions & Weiser, M. (1991). The computer for the 21st century . Theory Mediality, Codality & Scientific American, pages 94–104. Modality Context Aarts, E., R. Harwig, and M. Schuurmans. 2001. Ambient Ambient Intelligent Systems Intelligence. In The Invisible Future: The Seamless Descriptive Framework & Integration of Technology into Everyday Life , ed. P. J. Examples Denning, pp 235-250. New York: McGraw-Hill Companies. Implementing & Evaluating Dourish, Paul. “What we talk about when we talk about Human & Computer context.” Personal and ubiquitous computing 8, no. 1 Computing & (2004): 19-30. Culture De Ruyter, Boris, and Emile Aarts. “Experience research: a Required Reading methodology for developing human-centered interfaces.” In Handbook of ambient intelligence and smart environments, pp. 1039-1067. Springer, Boston, MA, 2010. SoSe 2018 Jörg Cassens – Summary 17 / 139
Weiser Definitions & Pads, tabs and boards Theory Mediality, Codality & Try it out in the lab Modality Context Only disappearing things help us focus Ambient Intelligent Systems Have to be trustworthy Descriptive Framework & Examples Information overload handled by machines Implementing Metaphor: utilities, electrical systems disappeared & Evaluating Human & Physical and virtual relations Computer No advanced AI needed Computing & Culture Interaction Required Reading Explicit vs. implicit Bridging the physical digital divide example: awareness systems SoSe 2018 Jörg Cassens – Summary 18 / 139
Aarts, Harwig, Schuurmans Definitions & Theory Aspects Mediality, Codality & Modality Ubiquity Context Ambient Intelligent Transparency (natural interaction?) Systems Intelligence Descriptive Framework & Emotions Examples Challenges Implementing & Evaluating Technical Human & Economic Computer Social Computing & Culture Productivity & Personal Time Required Interaction technology Reading Experience economy Ambient culture SoSe 2018 Jörg Cassens – Summary 19 / 139
Dourish Definitions & Viewpoints Theory Mediality, Codality & context as a data representation issue Modality Context Look at parameters Ambient Intelligent Systems context is not static Descriptive Framework & Look at activities Examples Question: what is ordinary? Implementing & Evaluating Static aspects and dynamic creation of ordinariness Human & Dialectic relation of top-down and bottom-up Computer We learn about structures and use them Computing & Culture Aspects Required information vs. relation Reading statically vs. dynamically defined stable vs. occasionally changing context separate vs. part of activity SoSe 2018 Jörg Cassens – Summary 20 / 139
de Ruyter & Aarts Definition Definitions & Ambient Intelligence refers to the embedding of technologies Theory Mediality, Codality & into electronic environments that are sensitive and responsive Modality Context to the presence of people. Ambient Intelligent Systems Descriptive Ambience refers to technology being embedded on a large Framework & Examples scale in such a way that it becomes unobtrusively Implementing integrated into everyday life and environments. Hence, the & Evaluating ambient characteristic of AmI has both a physical and Human & Computer social meaning. Computing & Intelligence reflects the situation in which the digital Culture Required surroundings exhibit specific forms of cognition, i.e. the Reading environments should be able to recognize the people that inhabit them, personalize according to individual preferences, adapt themselves to the users, learn from their behavior and possibly act upon their behalf. SoSe 2018 Jörg Cassens – Summary 21 / 139
Definitions & Theory Descriptive Framework & Examples Research Systems Implementing Descriptive Framework & Examples & Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 22 / 139
Descriptive Framework Version 4 Definitions & Theory Contextualisation Descriptive Framework & Examples Contextual Parameter Research Systems Environment – things, services, people Implementing & Evaluating Personal – mental & physical information about user Human & Social – roles & relations Computer Task – what is the user doing Computing & Culture Spatio-Temporal – when & where are we Other Required Reading Process of Contextualisation Awareness – what aspects are taken into account? Sensitivity – what aspects are changed? SoSe 2018 Jörg Cassens – Summary 23 / 139
Descriptive Framework Version 4 Definitions & Theory Descriptive Intelligence Framework & Examples Research Systems System Intelligence Implementing & Evaluating Personalized – tailored to individual needs Human & Adaptive – changing in response to user needs Computer Anticipatory – can act on its own on user’s behalf Computing & Culture Social Intelligence Required Socialized – compliant to social conventions Reading Empathic – take user’s inner states into account Conscious – introspection, has inner state SoSe 2018 Jörg Cassens – Summary 24 / 139
Descriptive Framework Version 4 Definitions & Ambience Theory Descriptive Perception Framework & Examples Mediality – media types Research Systems Codality – semantic representation Implementing & Evaluating Modality – human senses Human & Reasoning Computer Computing & Context Awareness Culture Context Sensitivity Required Other Reading Action Mediality – media types Codality – semantic representation Modality – human senses SoSe 2018 Jörg Cassens – Summary 25 / 139
Descriptive Framework Version 4 Interaction Definitions & Theory Descriptive Implicit vs. Explicit Framework & Examples Implicit input – through behaviour not primarily aimed at Research Systems interacting with the computerised system (walking through Implementing & Evaluating a door, using a whiteboard...) Human & Explicit input – primarily aimed at interacting with the Computer computerised system (voice or gesture commands...) Computing & Explicit output – designed to get the users’ attention (voice Culture output...) Required Reading Implicit output – change of material setting where the users’ attention is not the primary goal (opening doors...) Emotion Does the system sense emotions? Does the system show emotions? SoSe 2018 Jörg Cassens – Summary 26 / 139
Descriptive Framework Version 4 Embeddedness Definitions & Theory Descriptive Weaviness Framework & Examples Is the system woven into the background? Research Systems Is the interaction naturally/culturally sound? Implementing & Evaluating Enhancement Human & Does the system enhance or replace current solutions? Computer Current “technical” solutions – using (computerized) Computing & Culture artefacts Required Current “non-technical” solutions – not using Reading (computerized) artefacts Social Interaction Does the system enable/enhance social interaction amongst humans? Is the system targetting at supporting individual users? SoSe 2018 Jörg Cassens – Summary 27 / 139
Architecture Version 1 Definitions & Theory Descriptive Framework & Context Context Examples Sensing Acting Sensitivity Research Systems Awareness Implementing & Evaluating Human & Computer Sensors Actuators Computing & Culture Required World Reading Sensable Actable World World General, simplified architecture SoSe 2018 Jörg Cassens – Summary 28 / 139
Definitions & Theory Descriptive Framework & Examples Research Systems Implementing Research Systems & Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 29 / 139
Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 30 / 139
ShareBoard Definitions & Theory Building on top of existing projects Descriptive Easy and cheap to use electronic whiteboards Framework & Examples Touchscreen or Research Systems video projector and screen Implementing & Evaluating Wii remote controller and IR-Pen Human & Laptop with Webcam Computer Optional: kinect-like controller, leap motion controller Computing & Culture Drawings on canvases Required Freeform Reading Shapes Text (handwriting, virtual keyboard, speech recognition) Audio and video communication with other parties Automatic recognition of turn taking SoSe 2018 Jörg Cassens – Summary 31 / 139
Example: ShareBoard Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 32 / 139
Pervasive Games & Environments Pervasive (Learning) Games & Environments Definitions & Theory Breaking the magic circle, extending Descriptive spatial Framework & Examples temporal Research Systems social Implementing & Evaluating boundaries of game play Human & Computer This media informatics research area is still starting up Computing & First examples Culture Find It – Learning by Caching Required Reading City Explorer – Discover Würzburg Uburzis – competitive location-based game for school teams All designed and implemented taking instructional psychology into account Serious Games – gaming with a learning goal SoSe 2018 Jörg Cassens – Summary 33 / 139
City Explorer – Screenshots Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 34 / 139
Sliding Doors Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading Built as part of the Masters thesis of John Sverre Solem SoSe 2018 Jörg Cassens – Summary 35 / 139
Semantics Semantics as meaning potential or “what the person can Definitions & Theory mean” (Halliday, 1979, p.72) Descriptive Think of behaviour as semantic since there is a set of Framework & Examples behaviours that are at the individual’s disposal within a Research Systems particular context Implementing & Evaluating There is a limit to how truly individual it can be in most Human & Computer social contexts if the intention is to share meaning Computing & To share meaning you must share the code Culture Required It should be possible to model the meaning potential Reading available in a particular context Because communicating is multimodal, intention will not always be signalled entirely by behaviour The task in modelling intention is to find (patterns of) behaviours which carry the most significant meaning SoSe 2018 Jörg Cassens – Summary 36 / 139
Behaviour in Context Definitions & Theory How behaviour creates meaning and how we assign Descriptive meanings to behaviour is significantly related to situation Framework & Examples and context Research Systems Meaning is constituted in the interaction between the Implementing & Evaluating behavioural sign and its function within a context Human & Computer It is important to see expressive action as part of context Computing & and not as the product or effect of context Culture Required We can only assign meaning to behaviour through its Reading interaction with the context in which it is embedded If we are to find meaning in behaviour we primarily look to the dynamic relationship between the unfolding interaction and the context SoSe 2018 Jörg Cassens – Summary 37 / 139
Intention in Context Definitions & Theory Intention is something which is dynamic and emergent Descriptive Framework & from interaction rather than a static and predetermined Examples Research Systems feature of interaction Implementing & Evaluating Intention can thus be considered context sensitive Human & We have not attempted to model intention as a general or Computer Computing & context free concept Culture We have looked at the intention to walk through a door Required Reading rather than intention in general Our model of intention may be generalisable to contextually similar situations (waiting for a bus or train) SoSe 2018 Jörg Cassens – Summary 38 / 139
Sliding Doors Definitions & Theory Automatic sliding doors were chosen because of a rather Descriptive restricted behavioural set Framework & Examples Also, link between behaviour, intention and outcome is Research Systems much clearer and simpler than in other typical, but more Implementing & Evaluating complex situations Human & Computer The doors either open appropriately or they do not Computing & We were in no way suggesting that automatic doors should Culture respond to intention Required Reading Proximity is a good approximation of intention to go through a door But other people thought it would be good, so we reconsidered that position SoSe 2018 Jörg Cassens – Summary 39 / 139
CAKE and MATe Definitions & CAKE (Context Awareness and Knowledge Environment) is Theory a framework for building contextualized ambient Descriptive Framework & intelligent systems Examples Research Systems MATe (Mate for Awareness in Teams) is an application Implementing primarily aiming at improving situation awareness in work & Evaluating Human & teams Computer Designed to blend seamlessly with the team members’ Computing & Culture everyday routine, enabling unobtrusive in-situ interaction Required and facilitation of cooperation and communication Reading Knowledge is modelled in a user-centred process Technologies employed come from the semantic web community as well as artificial intelligence in general and machine learning in particular SoSe 2018 Jörg Cassens – Summary 40 / 139
Example: Exploring the Design Space Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading Very ambient, but hard to understand?
Example: Exploring the Design Space Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading Very traditional, but easy to grasp?
Example: Exploring the Design Space Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading Or something in between? SoSe 2018 Jörg Cassens – Summary 41 / 139
LADI Definitions & LADI – Location-Aware Device Integration Theory Descriptive Cross-Device Integration (XDI) Framework & Examples Utilises Research Systems Pads, Implementing & Evaluating Tabs, and Human & Boards Computer Location-centric, not network-centric Computing & Culture Storage (for example) in “the cloud” Required Reading Challenges: Indoor-localisation Media access Access control Device capabilities SoSe 2018 Jörg Cassens – Summary 42 / 139
Example: LADI in use Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 43 / 139
AmbieSense Context Model Definitions & Theory Descriptive Framework & Examples Research Systems Implementing & Evaluating Human & Computer Computing & Culture Required Reading User context in AmbieSense SoSe 2018 Jörg Cassens – Summary 44 / 139
Parts of the AmbieSense Context Model 1 Environmental context: Captures the users surroundings, Definitions & Theory such as things, services, light, people, and information Descriptive accessed by the user. Framework & Examples 2 Personal context: Mental and physical information about Research Systems Implementing the user, such as mood, expertise, disabilities and weight. & Evaluating 3 Social context: Social aspects of the user, such as Human & Computer information about friends, relatives and colleagues. Computing & Culture 4 Task context: Describe what the user is doing, it can Required describe the user’s goals, tasks, activities, etc. Reading 5 Spatio-temporal context: This type of context is concerned with attributes like: time, location and movement. The different aspects of the contexts are attribute-value tuples that are associated with the appropriate contexts. SoSe 2018 Jörg Cassens – Summary 45 / 139
Two-fold Use of Activity Theory Definitions & Theory Knowledge engineering with “Activity-Theoretic Goggles”: Descriptive we try to understand the basic properties of the workplace Framework & Examples using CHAT Research Systems Two-fold use of the theory Implementing & Evaluating Building the model: Building a knowledge model which can Human & capture the basic concepts of AT Computer General knowledge about human work processes together Computing & Culture with “best practice” knowledge is used to identify components of the context model Required Reading Populating the model: Using empirical evidence to fill the model Results from Activity-Theoretic field studies can be used to generate an initial knowledge model (that can be enhanced by online learning) SoSe 2018 Jörg Cassens – Summary 46 / 139
Mapping Definitions & Theory Descriptive Framework & Artefact Examples Research Systems Implementing & Evaluating Human & Computer Subject Object Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 47 / 139
Mapping Definitions & Theory Descriptive Framework & Artefact Examples Research Systems Implementing & Evaluating Human & Computer Subject Object Computing & Culture Required Reading Community Rules Division of Labour SoSe 2018 Jörg Cassens – Summary 47 / 139
Mapping Definitions & Theory Environmental Context Descriptive Framework & Artefact Examples Research Systems Implementing & Evaluating Personal Context Task Context Human & Computer Subject Object Computing & Culture Required Reading Community Rules Division of Labour Task Context Social Context Spatio-Temporal Context SoSe 2018 Jörg Cassens – Summary 47 / 139
Mapping Location? Instruments used? Definitions & Theory Environmental Context Descriptive Framework & Artefact Examples Research Systems Implementing User? Patient? Raw material? & Evaluating Personal Context Task Context Human & Computer Subject Object Computing & Culture Required Reading Community Rules Division of Labour Task Context Social Context Spatio-Temporal Context Rules? Time? Persons involved? Roles involved? SoSe 2018 Jörg Cassens – Summary 47 / 139
Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Implementing & Evaluating Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 48 / 139
Prevalent Paradigm Definitions & Theory Prevalent computing paradigm designed for personal Descriptive information management Framework & Examples desktops and laptops with fixed configurations of mouse, Implementing & Evaluating keyboard, and monitor Topics & Challenges Designing dedicated network services with fixed network addresses Implementing and locations Evaluating Human & printers Computer file servers Computing & ... Culture Required direct manipulation interfaces Reading representation and manipulation of files, documents, and applications SoSe 2018 Jörg Cassens – Summary 49 / 139
Ambient Interaction interaction mode goes beyond the one-to-one model prevalent for PCs Definitions & Theory many-to-many model where the same person uses multiple Descriptive devices Framework & Examples and several persons may use the same device Implementing interaction may be implicit, invisible, or through sensing & Evaluating natural interactions such as speech, gesture, or presence Topics & Challenges Designing wide range of sensors is required, both sensors built into Implementing Evaluating the devices as well as sensors embedded in the Human & environment Computer location tracking devices, cameras, and accelerometers Computing & Culture can be used to detect who is in a place and deduce what Required they are doing Reading provide the user with information relevant in a specific location adapt their device to a local environment or the local environment to them Networking is ofen wireless and ad hoc SoSe 2018 Jörg Cassens – Summary 50 / 139
Ambient Paradigm Definitions & Different paradigm of computing environment Theory Descriptive heterogeneous set of devices Framework & Examples invisible computers embedded in everyday objects such as Implementing cars and furniture & Evaluating mobile devices such as smartphones Topics & Challenges Designing personal devices such as laptops Implementing very large devices such as wall-sized displays and tabletop Evaluating Human & computers situated in the environments and buildings we Computer inhabit Computing & Culture All have different operating systems, networking interfaces, input capabilities, and displays Required Reading some are designed for end user interaction other devices, such as sensors, are not used directly by end users SoSe 2018 Jörg Cassens – Summary 51 / 139
Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Topics & Challenges Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 52 / 139
Resource-Constrained Devices wide range of new devices are built and introduced, which Definitions & ofen are resource-constrained Theory Descriptive devices such as mobile phones and music players have Framework & Examples limited CPU, memory, and network connectivity compared Implementing to a standard PC & Evaluating Topics & Challenges embedded platforms such as sensor networks and smart Designing Implementing cards are very limited compared to a PC or even a Evaluating smartphone Human & Computer it is important to recognize the constraints of the target Computing & Culture devices, and to recognize that hardware platforms are highly heterogeneous and incompatible with respect to Required Reading hardware specifications operating system input/output capabilities network ... SoSe 2018 Jörg Cassens – Summary 53 / 139
Distribution Definitions & Theory systems ofen distributed; they entail interaction between Descriptive different devices Framework & Examples mobile, embedded, or server-based Implementing these devices have different networking capabilities & Evaluating Topics & Challenges Spontaneous Designing Implementing devices continuously connect and disconnect Evaluating create and destroy communications links Human & Computer from a communication perspective, these devices may Computing & Culture leave the room (or run out of battery) at any time Required therefore, communication between the mobile devices and Reading the services in the smart room needs to gracefully handle such disconnection SoSe 2018 Jörg Cassens – Summary 54 / 139
Heterogeneous Execution Environments applications ofen involve a wide range of hardware, Definitions & Theory network technology, operating systems, input/output Descriptive capabilities, resources, sensors, etc. Framework & Examples in contrast to the traditional use of the term application, Implementing which typically refers to sofware that resides on one to & Evaluating three physical nodes, a ubiquitous application typically Topics & Challenges Designing spans several devices, which need to interact closely and in Implementing Evaluating concert in order to make up the application Human & a Smart Room is an application that relies on several Computer devices, services, communication links, sofware Computing & Culture components, and end user applications, which needs to Required work in complete concert Reading handling heterogeneity is not only a matter of being able to compile, build, and deploy an application on different target platforms—such as building a desktop application to run on different versions of Windows, Mac OS, and Linux SoSe 2018 Jörg Cassens – Summary 55 / 139
Invisible Computing Definitions & Handling and/or achieving invisibility is a core challenge Theory For example, monitoring human behaviour at home and Descriptive Framework & providing smart home control of the heating systems Examples In many of these cases, the computers are invisible to the Implementing & Evaluating users in a double sense Topics & Challenges Designing the computers are embedded into buildings, furniture, Implementing medical devices, etc., and are as such physically invisible to Evaluating Human & the human eye Computer Second, the computers operate in the periphery of the Computing & users’ attention and are hence mentally invisible Culture (imperceptible). Required Reading From a systems perspective, obtaining and handling invisible computing is a fundamental change from traditional computing SoSe 2018 Jörg Cassens – Summary 56 / 139
Invisibility as Fundamental Change traditional systems rely heavily on having the users’ Definitions & Theory attention; Descriptive users either use a computer or they don’t Framework & Examples This means, for example: Implementing the system sofware can rely on sending notifications and & Evaluating Topics & Challenges error messages to users, and expect them to react Designing ask for input in the contingency where the system needs Implementing Evaluating feedback in order to decide on further actions Human & ask the user to install hardware and/or sofware Computer components Computing & Culture can ask the user to restart the device Required Moving toward invisible computing, these assumptions Reading completely break down Mitigation strategies include autonomic computing, contingency management and graceful degradation SoSe 2018 Jörg Cassens – Summary 57 / 139
Security & Privacy: Trust Definitions & Trust Theory Descriptive First, trust is ofen lowered in volatile systems because the Framework & Examples principals whose components interact spontaneously may have Implementing no a priori knowledge of each other and may not have a trusted & Evaluating third party Topics & Challenges Designing Implementing Evaluating a new device that enters a hospital cannot be trusted to be Human & Computer used for displaying or storing sensitive medical data, and Computing & making the necessary configuration may be an Culture administrative overhead that would prevent any sort of Required Reading spontaneous use Hence, using the patient’s mobile phone may be difficult to set up SoSe 2018 Jörg Cassens – Summary 58 / 139
Security & Privacy: Assumptions Assumptions Definitions & Theory Second, conventional security protocols tend to make Descriptive Framework & assumptions about devices and connectivity that may not hold Examples Implementing & Evaluating portable devices more easily stolen and tampered with Topics & Challenges Designing resource-constrained embedded devices may not have Implementing Evaluating sufficient computing resources for asymmetric public key Human & Computer cryptography Computing & sofware does ofen not get updated afer initial release Culture “fire and forget”-strategy of vendors for cheap hardware Required Reading incompetence Many security protocols cannot rely on continuous online access to a server, which makes it hard to issue and revoke certificates SoSe 2018 Jörg Cassens – Summary 59 / 139
Security & Privacy: Context Context Definitions & Theory Third, the nature of ambient systems creates the need for a new Descriptive Framework & type of security based on location and context; service Examples authentication and authorization may be based on context and Implementing & Evaluating not the user Topics & Challenges Designing Implementing people entering a cafe may be allowed to use the café’s Evaluating Human & printer Computer if a device wants to use the café’s printer, it needs to be Computing & Culture verified that this device indeed is inside the cafe Required Reading it does not matter who uses the printer, the cafe cares only about where the user is Vice versa, the customer only cares that he connects to the printer in the café SoSe 2018 Jörg Cassens – Summary 60 / 139
Security & Privacy: User Data Sensors Definitions & Fourth, new privacy challenges emerge Theory Descriptive Framework & By introducing sensor technology, ambient systems may Examples gather extensive data on users including information on Implementing location, activity, social interaction, speech, video, and & Evaluating Topics & Challenges biological data Designing Implementing if these systems are invisible in the environment, people Evaluating may not even notice that data are being collected Human & Computer hence, designing appropriate privacy protection Computing & mechanisms is central Culture key challenge is to manage that users provide numerous Required Reading identifiers to the environment while moving around and using services networking IDs such as MAC, Bluetooth, and IP addresses (user-) names IDs of tags such as RFID tags payment IDs such as credit card numbers SoSe 2018 Jörg Cassens – Summary 61 / 139
Security & Privacy: Fluctuations Fluctuations Definitions & Fifh, the fluctuating usage scenarios also set up new challenges Theory for security Descriptive Framework & Examples numerous devices, users continuously create new Implementing & Evaluating associations Topics & Challenges Designing if all or some of these associations need to be secured, Implementing device and user authentication happens very ofen Evaluating Human & Existing user authentication mechanisms are, to a large Computer degree, designed for few (1–2) and long-lived (hours) Computing & associations between a user and a device or service Culture a user logs into a PC and uses it for the whole workday Required Reading in ambient scenarios, users may enter a smart room and use tens of devices and services in a short period (minutes) traditional user authentication e.g. using user names and passwords not feasible Moreover, if the devices are embedded or invisible, it may be difficult and awkward SoSe 2018 Jörg Cassens – Summary 62 / 139
Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Designing Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 63 / 139
Why? Definitions & Theory Prototyping future systems to explore ubiquity in practice Descriptive Empirical exploration of user reactions Framework & Examples Gathering datasets to tackle computational problems Implementing & Evaluating Creating experiences for public engagement or Topics & Challenges Designing performance Implementing Evaluating Creating research test beds to agglomerate activity and Human & Computer stimulate further research Computing & Explore a hypothesis more naturalistically Culture Required Test the limits of computational technologies Reading Addressing the perceived needs of a problem domain or pressing societal issue SoSe 2018 Jörg Cassens – Summary 64 / 139
Computational Knowledge Definitions & Theory Descriptive One needs to decide Framework & Examples what knowledge a system will need about the real world to Implementing function & Evaluating how it will get into the system Topics & Challenges Designing how to represent it Implementing Evaluating how this state will be maintained Human & what to do if it is incorrect Computer Unless this knowledge is easy to sense, or trivial to reason Computing & Culture with, one you must also decide Required what the implications are if the knowledge is imperfect or Reading conclusions are erroneously reached SoSe 2018 Jörg Cassens – Summary 65 / 139
Computational Knowledge about Physical World Definitions & Theory Descriptive Key questions you should ask yourself are Framework & Examples 1 What can be reliably sensed? Implementing & Evaluating 2 What can be reliably known? Topics & Challenges Designing 3 What can be reliably inferred? Implementing Evaluating Human & Computer The degree to which you can answer these questions for the Computing & Culture intended function of your system will help determine the Required feasible scope, or set some of the research challenges. Reading SoSe 2018 Jörg Cassens – Summary 66 / 139
Mental Models Definitions & Theory Key Question Descriptive Framework & Examples What do you intend for the user to understand or perceive of the Implementing & Evaluating system in operation? Topics & Challenges Designing Implementing To grow comfortable with it, adopt it, and potentially Evaluating Human & appropriate it, the user must be able to form a mental Computer model of cause and effect or a plausible rationale for its Computing & Culture behaviour Required Mental models on the user side can only be influenced by Reading induction SoSe 2018 Jörg Cassens – Summary 67 / 139
Always Runtime ambient systems are composed of distributed, potentially Definitions & Theory disjoint, and partially connected elements (sensors, mobile Descriptive devices, people, etc.) Framework & Examples “partially connected” here reflects that these elements will Implementing & Evaluating ofen not be reliably or continuously connected to each Topics & Challenges Designing other Implementing Evaluating the system is the product of spontaneous exchanges of Human & information when elements come together Computer Computing & interaction patterns and duration will vary with the design Culture and ambition of any given system, but it is important to Required consider a key precept: Reading once deployed, all changes happen at runtime typically no simultaneous access to all the elements to (for example) upgrade them or restart them SoSe 2018 Jörg Cassens – Summary 68 / 139
Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Implementing Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 69 / 139
UCD & Participatory Design Definitions & Theory Deploying systems for people to use is always a costly Descriptive Framework & process Examples Implementing Designing a system that meets peoples’ expectations, and & Evaluating indeed, helping set those expectations requires great care Topics & Challenges Designing and expertise Implementing Evaluating The key is identifying the stakeholders and involving them Human & in discussions from an early stage Computer Computing & User-Centred Design Processes Culture Participatory Design Processes Required Reading many issues due to the real world an d organizational settings that can catch the unwary developer by surprise SoSe 2018 Jörg Cassens – Summary 70 / 139
User Centred Design Definitions & Planning Theory UCD Descriptive Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading Finished Product SoSe 2018 Jörg Cassens – Summary 71 / 139
User Centred Design Definitions & Planning Theory Requirements UCD Descriptive Elicitation Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading Finished Product SoSe 2018 Jörg Cassens – Summary 71 / 139
User Centred Design Definitions & Planning Theory Requirements UCD Descriptive Elicitation Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Requirements Evaluating Specification Human & Computer Computing & Culture Required Reading Finished Product SoSe 2018 Jörg Cassens – Summary 71 / 139
User Centred Design Definitions & Planning Theory Requirements UCD Descriptive Elicitation Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Requirements Evaluating Specification Human & Computer Computing & Culture Required Reading Design & Finished Production Product SoSe 2018 Jörg Cassens – Summary 71 / 139
User Centred Design Definitions & Planning Theory Requirements UCD Descriptive Elicitation Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Requirements Evaluating Evaluation Specification Human & Computer Computing & Culture Required Reading Design & Finished Production Product SoSe 2018 Jörg Cassens – Summary 71 / 139
User Centred Design Definitions & Planning Theory Requirements UCD Descriptive Elicitation Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Requirements Evaluating Evaluation Specification Human & Computer Computing & Culture Required Reading Design & Finished Production Product SoSe 2018 Jörg Cassens – Summary 71 / 139
User Centred Design Definitions & Planning Theory Requirements UCD Descriptive Elicitation Framework & Process Examples Implementing & Evaluating Topics & Challenges Designing Implementing Requirements Evaluating Evaluation Specification Human & Computer Computing & Culture Required Reading Design & Finished Production Product SoSe 2018 Jörg Cassens – Summary 71 / 139
Participatory Design Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading ☞ Public hearing in urban planning (cc-by-sa Kaihsu Tai) SoSe 2018 Jörg Cassens – Summary 72 / 139
Low-Fidelity Prototyping Definitions & Theory Descriptive Framework & Examples graphical storyboards of proposed interactions. Implementing & Evaluating simple scenarios that can be discussed Topics & Challenges Designing paper prototypes Implementing Evaluating models of devices Human & Computer Anything that can add richness to the discussion of the system Computing & Culture with potential users Required Reading SoSe 2018 Jörg Cassens – Summary 73 / 139
Medium-Fidelity Prototyping Definitions & Theory Video prototypes Descriptive Framework & can communicate the concepts in the system quite Examples effectively Implementing & Evaluating act as a useful reference for explaining the system later on Topics & Challenges rapid prototypes of user interfaces using prototyping Designing Implementing toolkits can afford a more realistic synthesis of the intended Evaluating user experience Human & Computer Wizard of Oz Computing & prototypes of parts of the system Culture not implemented parts are simulated by human Required Reading behaviour of the system to be emulated and thus experienced by others SoSe 2018 Jörg Cassens – Summary 74 / 139
High-Fidelity Prototyping Partially working systems Definitions & Theory Horizontal prototype Descriptive all the intended functionality, but only at the top level Framework & Examples Example: initiate a shopping spree, but cannot order Implementing Good for testing high level goals and action plans & Evaluating Vertical prototype Topics & Challenges Designing only one or two tasks are implemented in detail Implementing Evaluating Example: shop til you drop, but cannot see shipping Human & information Computer Good when only few tasks are seen as particularly complex Computing & Culture or important Required Chauffered prototype Reading Considerable functionality, but little or no error detection How: A well trained assistant accepts and executes requests on behalf of the actual test user Orthogonal to vertical and horizontal SoSe 2018 Jörg Cassens – Summary 75 / 139
Real-World Issues The need to comply with health and safety or disabilities Definitions & legislation, which can constrain the citing of equipment Theory and place certain usability requirements for disabled users Descriptive Framework & To be sensitive to data protection legislation, which may Examples impact what data you can store, whether users have the Implementing & Evaluating right to opt-in, opt-out, or declare (e.g., with notices) that Topics & Challenges Designing the system is in operation Implementing Evaluating Environmental factors (including weather, pollution, etc.) Human & can have a devastating effect on equipment that is not Computer adequately protected Computing & Culture Privacy and organizational sensitivity Required potentially open vulnerabilities (perceived or actual) to Reading expose private information or interfere with existing systems or processes particularly true for organizations managing sensitive data or in high-pressure situations, such as healthcare and emergency services SoSe 2018 Jörg Cassens – Summary 76 / 139
Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Evaluating Topics & Challenges Designing Implementing Evaluating Human & Computer Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 77 / 139
Evaluation Definitions & Theory Simulation Descriptive In particular object-oriented simulations Framework & Examples Agents with particular goals, believes, intentions interact Implementing via simulated sensors with the real sofware & Evaluating Data and/or modelling necessary Topics & Challenges Designing Proof-of-concepts Implementing Evaluating field studies as done by Marc Weiser at PARC Human & Rudimentary and/or incomplete (see prototypes) Computer Computing & Implementing and Evaluating Applications Culture Large-scale implementations Required Reading Long running systems significant amount of users Field-study SoSe 2018 Jörg Cassens – Summary 78 / 139
Proof of Concept A PoC is a rudimentary and/or incomplete realization of a Definitions & certain technical concept or design to prove that it can Theory actually be realized and built, while also to some degree Descriptive Framework & demonstrating its feasibility in a real implementation Examples Not a theoretical (mathematical) proof of anything; it is Implementing & Evaluating merely a proof that the technical idea can actually be Topics & Challenges Designing designed, implemented, and run Implementing Evaluating Creating PoCs is the most prevalent evaluation strategy in Human & ambient systems Computer Weiser’s tabs, pads and boards Computing & Culture A PoC is a somewhat weak evaluation strategy Required Reading It basically shows only that the technical concept or idea can be implemented and realized Actually, however, a PoC tells us very little about how well this technical solution meets the overall goals and motivation of the research SoSe 2018 Jörg Cassens – Summary 79 / 139
End-User Applications Definitions & Theory Descriptive Framework & Examples A stronger evaluation approach is to build end user Implementing applications using ambient systems component and & Evaluating Topics & Challenges infrastructures, and then put these applications into Designing Implementing subsequent evaluation Evaluating Human & These applications can then be evaluated by end users in Computer either a simulated environment or in a real-world Computing & Culture deployment Required Reading SoSe 2018 Jörg Cassens – Summary 80 / 139
Released Systems The strongest evaluation of ambient systems components Definitions & Theory is to release them for third party use, for example, as open Descriptive Framework & source Examples In this manner, the system research is used and evaluated Implementing & Evaluating by other than its original designers, and the degree to Topics & Challenges Designing which the system components helps the application Implementing Evaluating programmers to achieve their goals directly reflects the Human & qualities of the system components Computer Computing & One may even argue that there is a direct correlation Culture between the number of application developers and Required Reading researchers using the system in their work, and the value and merits of the workReleasing and maintaining systems sofware does, however, require a substantial and continuing effort SoSe 2018 Jörg Cassens – Summary 81 / 139
Required Reading Tie-ins Weiser, M. (1991). The computer for the 21st century . Definitions & Scientific American, pages 94–104. Theory Descriptive Davies, N., & Gellersen, H. W. (2002). “Beyond prototypes: Framework & Examples Challenges in deploying ubiquitous systems.” IEEE Implementing Pervasive computing, 1(1), 26-35. & Evaluating Topics & Challenges Hansen, T. R., Bardram, J. E., & Soegaard, M. (2006). Designing Implementing “Moving out of the lab: Deploying pervasive technologies Evaluating in a hospital.” IEEE Pervasive Computing, 5(3), 24-31. Human & Computer Abowd, Gregory D., Elizabeth D. Mynatt, and Tom Rodden. Computing & Culture “The human experience” IEEE pervasive computing 1.1 Required (2002): 48-57. Reading De Ruyter, Boris, and Emile Aarts. “Experience research: a methodology for developing human-centered interfaces.” In Handbook of ambient intelligence and smart environments, pp. 1039-1067. Springer, Boston, MA, 2010. SoSe 2018 Jörg Cassens – Summary 82 / 139
Hansen, Bardram, Soegaard Definitions & Hospital room for operating room scheduling Theory Descriptive Not ubicomp, but having ubicomp aspects Framework & Awareness media Examples Implementing Challenges & Evaluating No added value for everyone Topics & Challenges Designing Concerns of privacy → reduced resolution, no-track-areas Implementing Evaluating Space for devices difficult to find Human & Reliability an issue → reliable over features Computer Iterative development Computing & Culture Guerilla teaching Required Deployment Reading Consider implicit & explicit organisational change Table with concerns SoSe 2018 Jörg Cassens – Summary 83 / 139
Davies & Gellersen Comments on Weiser Individual technology is developed, something missing Definitions & Theory Major problem: lack of integration Descriptive Walled gardens Framework & Examples How about interoperability certifications? Implementing Whole is more than sum of parts, what are the issues? & Evaluating Technology Topics & Challenges Designing Social & legal Implementing Evaluating Economic Human & Important to move past prototypes Computer Envisioned use cases not realized Computing & Culture Users find uses not envisioned Required Difficult to evaluate invisible technology without Reading deployment What is the value proposition of AmI? individual solutions only sell individual systems Examples: Xerox, Lancaster, mediacup Research Challenges SoSe 2018 Jörg Cassens – Summary 84 / 139
Abowd, Mynatt, Rodden Activity vs. task Definitions & Task: beginning and end, sequence of doing Theory Activity: Continuation of tasks, interruptions expected Descriptive Framework & Differences traditional and new Examples Traditional HCI example: HTA Implementing & Evaluating Cognitive view: how do we act in the world Topics & Challenges Examples for theories Designing Implementing Activity Theory Evaluating Human & hierarchical structure of activities, use of artefacts Computer Situated Action Computing & Culture improvised behaviour, use of external cues Required Distributed Cognition Reading Humans part of a larger systems Actor-Network Theory Inscribing human programs in artefacts Systemic-Functional Theory of Language Language in use, generic structure potential SoSe 2018 Jörg Cassens – Summary 85 / 139
de Ruyter & Aarts Levels of system intelligence Definitions & Context awareness & sensitivity Theory Personalised Descriptive Adaptive Framework & Examples Anticipatory Implementing Facets of social intelligence & Evaluating Topics & Challenges Socialised Designing Implementing Pleasant to interact with Evaluating Empathic Human & Computer View on internal state of human Computing & Conscious Culture Introspection of internal states Required Reading Definition of AmI Three-step approach to development Context studies Lab studies Field studies SoSe 2018 Jörg Cassens – Summary 86 / 139
Definitions & Theory Descriptive Framework & Examples Implementing & Evaluating Human & Computer Human & Computer Trust Explanations Black Boxing Context Systemic-Functional Theory of Language Abstract Concepts Computing & Culture Required Reading SoSe 2018 Jörg Cassens – Summary 87 / 139
Artificial Intelligence Definitions & Theory Descriptive Intelligent sofware applications are systems that realize Framework & Examples artificial intelligence in sofware: Implementing & Evaluating Human & What is Artifical Intelligence (AI)? Computer Trust “It is the science and engineering of making intelligent Explanations Black Boxing machines, especially intelligent computer programs. It is Context Systemic-Functional related to the similar task of using computers to understand Theory of Language Abstract Concepts human intelligence, but AI does not have to confine itself to Computing & Culture methods that are biologically observable.” (McCarthy, 2007) Required Reading SoSe 2018 Jörg Cassens – Summary 88 / 139
Intelligence Definitions & Theory No universally accepted answer, but few would argue that Descriptive intelligence is a capacity displayed by humans. Framework & Examples Implementing What is Intelligence? & Evaluating Human & “Intelligence is a very general mental capability that, among Computer Trust other things, involves the ability to reason, plan, solve Explanations Black Boxing problems, think abstractly, comprehend complex ideas, learn Context Systemic-Functional quickly and learn from experience. It is not merely book Theory of Language Abstract Concepts learning, a narrow academic skill, or test-taking smarts. Rather, Computing & it reflects a broader and deeper capability for comprehending Culture our surroundings – ‘catching on,’ ‘making sense’ of things, or Required Reading ‘figuring out’ what to do.” (Gottfredson, 1997) SoSe 2018 Jörg Cassens – Summary 89 / 139
Recommend
More recommend
