Human factors: Intertwining of formal and empirical methods in - PowerPoint PPT Presentation

Human factors: Intertwining of formal and empirical methods in software engineering HaPoC Symposium at IACAP-14 Conference of the International Association for Computing and Philosophy Thessaloniki, July 3rd 2014 Gonzalo Génova ggenovaf@gmail.com Knowledge Reuse Group Universidad Carlos III de Madrid Universidad de Santiago de Chile 1 Intertwining of formal and empirical methods in software engineering

Introduction • Question: What is the science in Computer Science? – A very broad term. – A plurality of research areas and methods. – Both in "pure" science and in applied science (i.e. engineering). – Empirical methods are not enough to account for all kinds of scientific activity in computing. • A three dimensional categorization of research methods in CS: – Formal vs Empirical – Natural vs Artificial, or Science vs Engineering – Things vs People, or Product vs Process • Focus on human factors in software engineering research. 2 Intertwining of formal and empirical methods in software engineering

Reference Gonzalo Génova, Juan Llorens, Jorge Morato Software engineering research: The need to strengthen and broaden the classical scientific method IGI-Global, 2012 3 Intertwining of formal and empirical methods in software engineering

It is often heard… The experimental-scientific method is the preferred way, even the only way, to attain knowledge of reality. A work only deserves to be qualified as “scientific” if it is supported by “empirical evidence”. But… too simplistic in general, and specifically for CS. 4 Intertwining of formal and empirical methods in software engineering

1st dimension: Formal vs Empirical Formal Sciences Empirical Sciences deductive method inductive method definitions, axioms, theorems observation, experimentation laws are established laws are discovered modeling languages network traffic cryptographic algorithms convergence of genetic algorithms neural networks frequency of “bugs” inheritance laws in OO improving productivity with OO Can we experiment with formal systems? The dice ... 5 Intertwining of formal and empirical methods in software engineering

2nd dimension: Natural vs Artificial Analysis Design (pure science) (scientific engineering) observation of natural phenomena identification of needs hypothesis formulation artifact construction experimental confirmation / refutation artifact evaluation in search for truth (know) in search for utility (transform) • Herbert Simon. The Sciences of the Artificial, 1969. • Artifacts: – physical machines – conceptual or social systems (information structures, methods, processes, organizations…) 6 Intertwining of formal and empirical methods in software engineering

Combining both dimensions Formal Empirical Analysis formal analysis empirical analysis (Observation) Design formal design empirical design (Construction) 7 Intertwining of formal and empirical methods in software engineering

3rd dimension: Things vs People Inanimate products Human processes pieces of information (machinery and) humans involved inanimate entities developers and users physics, chemistry, biology economics, psychology, pedagogy • Two very different types of indeterminism – Inanimate beings: inherent vs improper indeterminism – Human beings: able to change behavior through cognition and free will 8 Intertwining of formal and empirical methods in software engineering

Plurality of research methods Formal Empirical formal analysis empirical analysis Entities of natural entities of natural entities Analysis (Observation of nature) formal analysis empirical analysis Processes of natural processes of natural processes formal design empirical design Products of artificial products of artificial products Design (Construction of artifacts) formal design empirical design Processes of artificial processes of artificial processes • Human Factors in the design of processes to improve: – Production (developers viewpoint) – Interaction (users viewpoint) 9 Intertwining of formal and empirical methods in software engineering

Difficulties • My area of interest: – Lack of adequate consideration of human factors in the justification of new artifacts that supposedly improve the software development process. • Difficulties in the empirical validation of production artifacts: – Are developers less demanding and more gullible than end users? – Fallacies against empirical methods (they are expensive...) – Distinguish experimentation and application. • Too naïve experiments: – Influence of culture and education. – People learn and change, they are not inanimate atoms. – Repeatability: we cannot isolate variables, or rewind history. – The Hawthorne effect. 10 Intertwining of formal and empirical methods in software engineering

Solutions? • Increase and vary the sample – Still, little predictive value in individual cases • Once again: research subjects are rational beings – Need of re-education and rational persuasion – Qualitative rather than quantitative arguments • Who am I? – Formal or empirical scientist? – Observation or construction scientist? – Things or people scientist? 11 Intertwining of formal and empirical methods in software engineering

Conclusions • Don’t let you be measured with other’s rule: – Particularly, empirical methods for inanimate beings. – Study and strengthen your own methods. • Increase knowledge of methods in social sciences: – How to isolate variables, how to repeat experiments. – Less ambitious experiments ( � HCI). • Adapting empirical methods is not enough. “Not everything that can be counted counts, and not everything that counts can be counted.” • Questions? Feel free! (ggenovaf@gmail.com) 12 Intertwining of formal and empirical methods in software engineering