Technical action research 3. Generalizing from TAR 4. Summary Roel - PDF document

17 ‐ 5 ‐ 2012 1. What is TAR? 2. Logical structure of TAR Technical action research 3. Generalizing from TAR 4. Summary Roel Wieringa University of twente 16th May, 2012 RCIS 2012, Valencia 1 16th May, 2012 RCIS 2012, Valencia 2 What is Technical Action Research? • Example – Researcher develops a technique to assess confidentiality 1. Wat is Technical Action risks in an IT architecture Research? – She applies it to a problem that a company has ... – producing an advice to the company ... – and drawing lessons learned about the method • She served two goals: – The company’s goal is to assess confidentiality risks – The researcher’s goal is to learn something about her method 16th May, 2012 RCIS 2012, Valencia 3 16th May, 2012 RCIS 2012, Valencia 4 Contrast with observational study What is Technical Action Research? (observational case study or survey) • The researcher plays three roles: • Example: – Designer: Designing a technique – Researcher observes one or more agile projects to investigate how requirements are prioritized – Helper: Using the technique to help others – Avoids influencing the projects – Researcher: Drawing lessons learned about technique – Observes, analyzes, concludes lessons learned • No change goal: The company is not influenced • The key to a proper methodology for TAR is keeping • Researcher’s goal is to learn about prioritization in agile projects these roles separate as it is currently happening • (the resulting knowledge may be useful to the companies) 16th May, 2012 RCIS 2012, Valencia 5 16th May, 2012 RCIS 2012, Valencia 6 1

17 ‐ 5 ‐ 2012 Contrast with “classical” action Contrast with consulting research • Consulting • In classsical AR, researcher helps client to identify and solve a problem – Consultant is paid by client – Consultant applies known techniques rather than – Emancipation of the powerless experimental technique – Learning about their situation – Reuse of techniques rather than critical evaluation • In TAR, the researcher wants to learn something – Aims at helping the client and acquiring repeat business, about a technique by using it to solve a client’s rather than testing a technique problem – Knowledge dissemination (if any) is internal 16th May, 2012 RCIS 2012, Valencia 7 16th May, 2012 RCIS 2012, Valencia 8 Action Research cycle AR in information systems research (Susman & Evered 1978) • AR in information systems – Identify problem in an organization – Jointly search for a solution – Implement it – Evaluate – Specify learning • TAR is technology ‐ driven, not problem driven – The technology may be motivated by a desire to solve a class of problems – Not a singlular problem 16th May, 2012 RCIS 2012, Valencia 9 16th May, 2012 RCIS 2012, Valencia 10 Why TAR for the researcher TAR and design science • Researcher developed a technique behind her desk • Design science is designing and investigating artifacts • Applied it to first to small and then to realistic • Characteristic for design science is scaling up to examples practice – Start at the desk, • Compared with other proposals – continue in the lab, • Then what? – end up in the field – Students will do as teacher tells: no realistic validation – In the field you do TAR and/or statistical field experiments – Best way to learn about the technique is to apply it – Similar to scaling up in pharmaceutical research yourself • Important to scale up from desk to practice 16th May, 2012 RCIS 2012, Valencia 11 16th May, 2012 RCIS 2012, Valencia 12 2

17 ‐ 5 ‐ 2012 Why TAR for the client • Risky project with large chance of non ‐ result 2. Logical structure of TAR • What is in it for the client? – Free consult – Potentially useful result – Advance knowledge of and experience with new techniques – Good relationships with university (PR, HRM) 16th May, 2012 RCIS 2012, Valencia 13 16th May, 2012 RCIS 2012, Valencia 14 Action Research cycle (Susman & Evered 1978) • This conflates two action cycles: – Action cycle of client – Action cycle of researcher • Each has a different goal and justification 16th May, 2012 RCIS 2012, Valencia 15 16th May, 2012 RCIS 2012, Valencia 16 The engineering cycle Problem context: Stakeholders, Software, • Rational action cycle Treatment Artifact Hardware, – Problem investigation Organizations, Processes, – Treatment ( = action) design … – Design validation – Treatment ( = action) implementation • Problem investigation – Implementation evaluation • Treatment design with new or existing artifacts • Design validation before applying the treatment • Treatment implementation by applying to the problem • Medical terminology • Implementation evaluation 16th May, 2012 RCIS 2012, Valencia 17 16th May, 2012 RCIS 2012, Valencia 18 3

17 ‐ 5 ‐ 2012 Stakeholders, goals • Problem investigation • Problem investigation Treatment = interaction between • Treatment design • Treatment design Phenomena, artifact and context diagnosis, • Design validation • Design validation evaluation • Treatment implementation • Treatment implementation • Implementation evaluation • Implementation evaluation • Interaction between Pill and Patient • Interaction between Software and its Context • Interaction between Method and its Context of use 16th May, 2012 RCIS 2012, Valencia 19 16th May, 2012 RCIS 2012, Valencia 20 Artifact & Context → Effects? • Problem investigation • Problem investigation Trade-off: Changes in artifact? • Treatment design • Treatment design Sensitivity: Changes in context? Effects serve Stakeholder goals? • Design validation • Design validation Transfer to practice! • Treatment implementation • Treatment implementation • Implementation evaluation • Implementation evaluation 16th May, 2012 RCIS 2012, Valencia 21 16th May, 2012 RCIS 2012, Valencia 22 • Problem investigation • Example: Extending an enterprise architecture (EA) method with goal ‐ oriented requirements • Treatment design engineering (GORE) to manage links to • Design validation business goals • Treatment implementation • Implementation evaluation Stakeholders, goals? Phenomena: Artifact & Context → Effects? Evaluation: Effects serve Stakeholder goals? 16th May, 2012 RCIS 2012, Valencia 23 16th May, 2012 RCIS 2012, Valencia 24 4

17 ‐ 5 ‐ 2012 Researcher’s cycle Client cycle Problem investigation: Problem investigation: Relation between EA and Goal of EA project? business objectives not known • Two goals Treatment design: – The client evaluates its redesigned EA against its goals Plan the project Treatment design: Extend EA method with – The researcher validates ARMOR against his goal GORE techniques (ARMOR) Design validation: Validate the plan • Three roles for the researcher Artifact validation: Usable? – Designing a technique Useful? Trade-offs? – Using it to help a client Sensitivity? Execute – Learning from it Implementation: • How do we use the client cycle to answer these validation Transfer to practice questions? EA evaluation EA satisfies client’s Evaluation: 16th May, 2012 RCIS 2012, Valencia goals? 25 16th May, 2012 RCIS 2012, Valencia 26 Monitor usage The empirical research cycle Conceptual framework, • Knowledge problem investigation • Rational choice of an answer to a knowledge question Research questions, • Research design – Knowledge problem investigation Population – Research design • Design validation – Design validation • Research execution – Research execution • Results evaluation – Results evaluation 16th May, 2012 RCIS 2012, Valencia 27 16th May, 2012 RCIS 2012, Valencia 28 • Knowledge problem investigation • Knowledge problem investigation Survey, observational case, • Research design • Research design Would this really answer our Experiment, Action case, • Design validation • Design validation questions? Simulation, ... • Research execution • Research execution Risk assessment of doing the • Results evaluation • Results evaluation wrong thing to answer the questions 16th May, 2012 RCIS 2012, Valencia 29 16th May, 2012 RCIS 2012, Valencia 30 5