17 ‐ 5 ‐ 2012 Design science • Design and investigation of artifacts Empirical validation research • Questions: methods – What artefact(s) are you developing? – How will you investigate its properties? Roel Wieringa University of Twente The Netherlands 17th May, 2012 RCIS 2012, Valencia 1 17th May, 2012 RCIS 2012, Valencia 2 The engineering cycle Implementation evaluation = Specification 1. The engineering cycle: developing artifacts Problem investigation implementation 2. The technical validation problem (1) Stakeholders? Goals? Choose an artifact! (2) Phenomena? Effects? Transfer to practice! 3. Technical validation research questions Causes? (3) Contribution of effects 4. Validation research methods to goals? 5. Examples Design validation Artifact design (1) Stakeholders, goals? Available artifacts? (2) Ar � fact X Context → E ff ects? Design new ones! Trade ‐ offs for different artifacts? Sensitivity to different Contexts? (3) Contribution of effects to goals? 17th May, 2012 RCIS 2012, Valencia 3 17th May, 2012 RCIS 2012, Valencia 4 Problems and artifacts Examples of problems (research goals) Problem context: • Developing an extension of i* Artifact: SW components & systems, • Developing a genome ontology HW components & systems, SW component, system, • Developing an argumentation ‐ based risk assessment People, HW component, Organizations, Interaction system, technique Business processes, Organization, Services, • …. Business process, Methods, Techniques, Service, • Questions: Conceptual structures, Method, Values, Desires, Fears, – What problem are you solving? Conceptual structure, Goals, Norms, Budgets, ... – For whom? Stakeholders. ... – What are current their experiences? Phenomena. Something to be influenced Something that is designed – Goals? Contribution? 17th May, 2012 RCIS 2012, Valencia 5 17th May, 2012 RCIS 2012, Valencia 6 1
17 ‐ 5 ‐ 2012 Examples of treatments • Developing a new technique to link business goals to 1. The engineering cycle enterprise architecture 2. The technical validation problem • Developing new techniques for agile requiremens prioritization 3. Technical validation research questions • Developing an algorithm for P2P document lookup with a 4. Validation research methods distributed hash table 5. Examples • Developing a directional antenna for TV reception in cars • Developing techniques to deal with interference in wireless city networks • What is the problem context in these cases? 17th May, 2012 RCIS 2012, Valencia 7 17th May, 2012 RCIS 2012, Valencia 8 The fundamental problem of validation Example: drug validation research • The artifact has not been implemented (transferred to • Drug = Medicine = Artifact practical problem contexts) yet. • Patient’s body = Problem context – It is not interacting with its intended problem context yet – You cannot have it interact with the intended problem context • Guidelines for validating medicine have been published by the U.S. Food and Drug Administration • So how to validate it? • http://www.fda.gov/cder/handbook/develop.htm • New drug development process: http://www.fda.gov/Drugs/DevelopmentApprovalProcess/Sm allBusinessAssistance/ucm053131.htm • See also http://en.wikipedia.org/wiki/Clinical_research 17th May, 2012 RCIS 2012, Valencia 17th May, 2012 RCIS 2012, Valencia 9 10 New Drug Application • Pre ‐ clinical research – Synthesis and purification. 1. Problem • (1) Investigate human body, 2. Design • (2) Design a new drug and 3. Validation • (3) Investigate its properties in the laboratory – Treat animals with it in short ‐ term 4. Validation experiments (few weeks) 5. Validation – Treat animals with it in long ‐ term experiments (few weeks to several Animals are years) models of patients 17th May, 2012 RCIS 2012, Valencia 17th May, 2012 RCIS 2012, Valencia 11 12 2
17 ‐ 5 ‐ 2012 • Pre ‐ clinical research • Pre ‐ clinical research – Synthesis and purification. (1) Investigate human body, (2) Design a new drug and (3) Investigate its properties in the laboratory – Synthesis and purification. (1) Investigate human body, (2) Design a new drug and (3) Investigate its properties in the laboratory – Treat animals with it in short ‐ term experiments (few weeks) – Treat animals with it in short ‐ term experiments (few weeks) – Treat animals with it in long ‐ term experiments (few weeks to several years) – Treat animals with it in long ‐ term experiments (few weeks to several years) • Clinical research • Clinical research 5. Validation – Phase 1: Treat healthy people with it to – Phase 1: Treat healthy people with it to understand the mechanisms and 6. Validation possible side effects of the drug (20 to 80 volunteers) understand the mechanisms and possible side 7. Validation – Phase 2: Treat ill people with it to understand mechanisms, side effects and effects of the drug (20 to 80 volunteers) contra ‐ indications (few hundred volunteers) – Phase 3: Treat ill people with it to investigate effectiveness, side effects and – Phase 2: Treat ill people with it to understand contra ‐ indications on a sufficiently large sample to generalize (few 100 to few 1000 volunteers) mechanisms, side effects and contra ‐ • Side effects, • Post availability indications (few hundred volunteers) • Mechanisms, – Phase 4: continue clinical studies to better – Phase 3: Treat ill people with it to investigate • Contra- Evaluation understand the drug effectiveness, side effects and contra ‐ indications – Surveillance studies in the field indications on a sufficiently large sample to generalize (few 100 to few 1000 volunteers) 17th May, 2012 RCIS 2012, Valencia 17th May, 2012 RCIS 2012, Valencia 13 14 More The engineering cycle realistic conditions of practice Implementation evaluation = Specification Ill Problem investigation implementation people Research (1) Stakeholders? Goals? Choose an artifact! Healthy subjects (2) Phenomena? Effects? Transfer to practice! people Causes? (3) Contribution of effects Animals Larger samples to goals? Small Large Population samples samples Design validation Artifact design • Once the drug can be produced (“purified”) in the lab, its use (1) Stakeholders, goals? Available artifacts? (2) Ar � fact X Context → E ff ects? is scaled up to conditions of practice Design new ones! Trade ‐ offs for different artifacts? Sensitivity to different Contexts? (3) Contribution of effects to goals? 17th May, 2012 RCIS 2012, Valencia 17th May, 2012 RCIS 2012, Valencia 15 16 Validation is modelling Validation is theory building • Development starts with an initial treatment theory (hunch, hope, hypothesis): • Animals, healthy volunteers, and ill volunteers are used as – “Artifact treats problem successfully”. models of arbitrary patients • Conclusions about the models are transferred to arbitrary • This theory is developed together with the artifact patients – It makes a cause e ff ect statement (Ar � fact X Context) → E ff ects • The model is a composite system (Problem X Artifact) – Side effects as well as desirable effects • The “X” is the treatment – It tries to explain this in terms of mechanisms – It claims this to be true of a certain class of contexts 17th May, 2012 RCIS 2012, Valencia 17th May, 2012 RCIS 2012, Valencia 17 18 3
Recommend
More recommend
