Ethnographic and field observation Dan Smith 2 Overview Aims - PDF document

6/11/18 Ethnographic and field observation Dan Smith 2 Overview • Aims • History • Practice 1

6/11/18 3 Ethnographic observation for HCI • Close observation of human behaviour in the field – What do people really do? – How do they interact with the computer? – What’ s the context? • Traditionally looking at office interactions • More important for mobile 4 Ethnography • Origins in the social sciences where it is used to display the social organisation of activities and so to understand work • Watch everything • Experiencing is paramount • Picks up what other methods may miss – how people do the ‘real’ work – in an organisation can be user-involvement procesess as well as data-gathering processes 2

6/11/18 5 Motivation and aims • “New technology may seem dazzling because people can now do the most familiar things in places they could not before, but if the technology does not support familiar activities its actual use can become problematic” • To give designers “… detailed analyses of the methodical organization of action and interaction in the settings under study to revise technical concepts …” (Crabtree et al. 2009) 6 Role of ethnographic studies • Ethnographic studies form a ‘bridge’ between users and designers • Provide a set of disciplines for observing and record behaviour and social interactions – qualitative data • Let the designer understand – what people do in a setting – how they organize their activities 3

6/11/18 7 Defamiliarisation • “Defamiliarisation” is intended to help designers rethink the assumptions built into familiar technologies • Tries to look at situations as if they are novel and alien • Seeing what is invisible to inhabitants – What they say vs. what they do • Making explicit the observer’ s bias – What you take for granted can blind you 8 FIELD OBSERVATION 4

6/11/18 9 Aims of ethnographic observation • To see how people actually use computers • Focus on close observation of everyday actions – otherwise it is just impressionistic storytelling • Try to discover interpretive framework of users in methodical detail. – uncovering what people do in the setting and how they organize what they do 10 5

6/11/18 11 12 6

6/11/18 13 Observation framework • Helps observers to keep their goals and questions in sight – Person • who is using the technology? – Place • where are they using it? – Thing • what are they doing with it? 14 Practical issues 1 • State your goals and questions clearly • How are you going to record events? – Notes, audio, video, or a combination – Set up the appropriate equipment • Go through your data soon after observation sessions – in case there is anything you need to revisit 7

6/11/18 15 Practical issues 2 • How are you going to gain acceptance and trust of those you are observing? • How will you handle sensitive issues like where you are allowed to go? • Observation teams? – More viewpoints, greater reliability and understanding • Plan to look at situation from different perspectives 16 Ethnographic observation in the workplace • Evaluator must be accepted into observation group • Data collection and interpretation may happen simultaneously • Identify problem or goal • Be sensitive to people’ s feelings • Collect a variety of data • Flexible movement between broad picture and specific questions • Analyse data within a context, after each session • Adjust your procedure to fit design process if necessary • Know when to stop observing 8

6/11/18 17 Data gathering in the workplace • Three dimensions 1. Distributed co-ordination • Where tasks take place and how they are co- ordinated 2. Plans and procedures • Focusing on organisational support for the work: workflow models and charts 3. Awareness of work • How people keep themselves aware of other’ s work: information sharing etc. 18 Outside the workplace • Emphasis on importance of setting – “Follow me home” technique – Observation can change what is observed • Time of day? • Location? 9

6/11/18 19 THEORETICAL BACKGROUND 20 Sociology: Ethnomethodology • The study of the everyday methods that people use for the production of social order – Originates from Harold Garfinkel (1954) • Focus is on the shared understandings of members of a social setting under study • Important in the development of ethnographic studies in HCI 10

6/11/18 21 Technomethodology • How do you generate technology design decisions from ethnographic observation? • How do anthropologists and ethnographers apply their insights to design problems? • How do designers make use of ethnographic studies? 22 A hybrid discipline? • Need to combine: – detailed small-scale observations from ethnomethodology – large-scale design decisions into a single framework • Several approaches have been proposed, but all have problems • Seems best to take a pragmatic approach, use studies for well-defined purposes 11

6/11/18 23 Observer or Hawthorne effect • Strong evidence from many studies that results are influenced by: – People feeling special because they’re being paid more attention – Being treated differently by being studied – can be positive • we’re important, the desire to please – can be negative • perceived goal is to reduce skill/autonomy/pay/head count 24 Implications for design • Provide suggestions for implementations • e.g., at the end of a study that demonstrated the impact of fears on archiving practices: “backup systems ought to be less transparent, assuaging the user’ s fears of loss. ” Kaye, J. 'J.', Vertesi, J., Avery, S., Dafoe, A., David, S., Onaga, L., Rosero, I., and Pinch, T. To Have and to Hold: Exploring the Personal Archive. Proc. CHI 2006. 12

6/11/18 25 Resources • Garfinkel H. (2002). Ethnomethodology's Program . New York: Rowman and Littlefield • Dourish P., Button G. (1998) On “Technomethodology”: Foundational Relationships between Ethnomethodology and System Design, Human Computer Interaction , 13(4), 395–432, 1998 • Crabtree A., Rodden T., Tolmie P., Button G. (2009) Ethnography Considered Harmful, CHI , 871-890 26 Key ethnomethodology studies in HCI • Bentley, R., Hughes, J.A., Randall, D., Rodden, T., Sawyer, P., Shapiro, D., Sommerville, I. Ethnographically-informed systems design for air traffic control. Proc. CSCW’92 123-129. • Harper, R. and Hughes, J.A. (1992), “What a F-ing System! Send em all to the same place and then expect us to stop em hitting”: Making technology work in air traffic control, in Technology and Working Order: Studies of Work, Interaction and Technology , ed, G. Button, 127-144, London: Routledge. • Harper, R., Hughes, J.A. and Shapiro, D. (1991), Harmonious Working and CSCW: Computer Technology and Air Traffic Control, in Studies in Computer Supported Cooperative Work: Theory, Practice and Design , ed. J. Bowers and S. Benford, Amsterdam, North-Holland. • MacKay, Wendy E. Is paper safer? The role of paper flight strips in air traffic control. ACM Transactions on Computer-Human Interaction 6(4) December 1999. • Heath, C. and Luff, P. (1992), Collaboration and control: Crisis management and multimedia technology in London Underground line control rooms, Computer Supported Cooperative Work , 1(1-2): 69-94. • Bowers, J., Button, G. and Sharrock, W ., "Workflow from Within and Without", Proc. ECSCW'95, Stockholm, Sweden , September 1995. • Gerson E, Star S L, 1986, "Analyzing Due Process in the Workplace" ACM Transactions on Office Information Systems 4 257-270 Suchman, Lucy. Office procedures as practical action: Models of work and system design. • ACM Transactions on Office Information Systems 1(4) October 1983 320-328. 13