PREFACE SOFTWARE TESTING What is it? Ben Simo - PDF document

9/8/2012 THE ART OF SOFTWARE INVESTIGATION Ben Simo Ben@QualityFrog.com PNSQC 2012 Based on THE ART OF SCIENTIFIC INVESTIGATION By W. I. B. Beveridge 1950 PREFACE SOFTWARE TESTING • What is it? Ben Simo Ben@QualityFrog.com Sep-12 2 1

9/8/2012 PREFACE THE ART OF SCIENTIFIC INVESTIGATION By William Ian Beardmore Beveridge An entirely fresh approach to the intellectual adventure of scientific research 1950 Ben Simo Ben@QualityFrog.com Sep-12 3 PREFACE COMPARING SOFTWARE TESTING TO SCIENTIFIC INVESTIGATION Ben Simo Ben@QualityFrog.com Sep-12 4 2

9/8/2012 CONTENTS • PREFACE • INTUITION • PREPARATION • REASON • SELECTION • OBSERVATION • SEQUENCE • INVESTIGATORS • EXPERIMENTATION • DISCUSSION • CHANCE • HYPOTHESIS Scientific research is not itself a science; • IMAGINATION it is still an art or craft. - W. H. George Ben Simo Ben@QualityFrog.com Sep-12 5 PREPARATION SELF-DIRECTED LEARNING • Knowledge o Build a foundation o Keep current o Maintain independence The research worker o Cultivate diversity remains a student all o Understand history his life. Preparation • Fluency for his work is never finished for he has to o Communicate & think with clarity keep abreast with the • Confer growth of knowledge. o Participate in the greater community - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 6 3

9/8/2012 SELECTION CHOOSE YOUR OWN WORK • Interest encourages success o If work is chosen for you, seek out an aspect that provokes interest • Select work that Start with a problem in which there is a o has a chance of success good chance of his o Is within your technical abilities accomplishing something, and which is not beyond [your] technical capabilities. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 7 SEQUENCE ITERATE 1. Review The most effective experimenters are 2. Observe usually those who give much thought to the 3. Analyze problem beforehand and resolve it into 4. Guess crucial questions and then give much 5. Experiment thought to designing experiments to answer the questions. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 8 4

9/8/2012 EXPERIMENTATION TWO TYPES OF INVESTIGATION • Observational o Collection of data from naturally occurring phenomena • Experimental A basic concept … is that there is an Collection of data from o infinitely large, an event made to occur hypothetical under controlled conditions population of which the experimental group or data are a random sample. All investigation is sampling - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 9 EXPERIMENTATION EXECUTION • Start modestly • Be competent Pilot o o Techniques Sighting o Tools o Screening o • Take notes It happens surprisingly often Document as you go o that one needs to refer back • Iterate to some detail whose Design later experiments based significance one did not o on results of earlier ones realize when the experiment • Stop was carried out. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 10 5

9/8/2012 EXPERIMENTATION STATISTICS • Caution o People give numbers more credence than they deserve o Averages are often The use of statistics does misleading not lessen the necessity o Graphs are often for using common sense misleading in interpreting results, a point which is sometimes forgotten. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 11 EXPERIMENTATION MISLEADING EXPERIMENTS • Mistakes o “Honest” mistakes o Incompetent experimenters Experimentation, like other • Contamination measures employed in research, is not infallible. o Accidental or unknown influences Inability to demonstrate a • Difficult to supposition experimentally prove a negative does not prove that it is incorrect. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 12 6

9/8/2012 EXPERIMENTATION EUREKA • Reproduce it • Look at it from multiple perspectives The real and lasting • Connect it with pleasure in a discovery other knowledge comes not so much from the accomplishment itself • Seek new avenues as from the possibility of using it as a stepping of investigation stone for fresh advances. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 13 CHANCE THE ROLE OF CHANCE • Chance plays an important part in discovery o Chance alone does not discover o Chance provides opportunity to the keen observer o Significance comes from an observer relating observations to other knowledge In the field of observation, chance favors only the prepared mind. - Pasteur Ben Simo Ben@QualityFrog.com Sep-12 14 7

9/8/2012 CHANCE COURTING CHANCE • Prepare your mind to recognize useful information • Entertain ideas that contradict beliefs • Be unconventional • Maximize the risk of having a fortunate accident • Postpone demand for evidence • Perform many experiments Chance favors only those who know how to court her. - Charles Nicolle Ben Simo Ben@QualityFrog.com Sep-12 15 CHANCE RECOGNIZE & EXPLOIT • Be alert for the unexpected Acute powers of • Don’t be blinded observation are often by hypothesis required to notice the clue, and especially o Follow up the ability to remain on interesting alert and sensitive for side-issues the unexpected while watching for the expected. - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 16 8

9/8/2012 HYPOTHESIS A TOOL FOR DISCOVERY • Suggests new o Experiments o Observations In science • Helps provide significance the primary to what we observe duty of ideas is to be useful • Most will be wrong and interesting even more than o Be prepared to abandon them to be 'true'. - Wilfred Trotter Ben Simo Ben@QualityFrog.com Sep-12 17 HYPOTHESIS PRECAUTIONS • Once an opinion is formed, it becomes difficult to think of alternatives Men who have • Don’t get too attached to excessive faith in your brainchild their theories or ideas are not only • Let go of a hypothesis ill-prepared for proved wrong making discoveries; they also make poor observations. - Claude Bernard Ben Simo Ben@QualityFrog.com Sep-12 18 9

9/8/2012 HYPOTHESIS SAFEGUARDS • Subordinate ideas to facts • Have multiple hypotheses • Make special note of data My business is to unfavorable to your hypothesis teach my aspirations to • Don’t embrace conjecture conform themselves • Once the experiment begins, to fact, not to try to make facts throw out the hypothesis harmonize with my aspirations. -Thomas Huxley Ben Simo Ben@QualityFrog.com Sep-12 19 IMAGINATION PRODUCTIVE THINKING • Ideas “occur” to us Can’t deliberately create ideas o May come during o To be genuinely • reflective thinking thoughtful, we • daydreaming must be willing • Fertilize your imagination to sustain and protract that o Variety of knowledge and experience state of doubt o Focus thinking which is the o Stay curious stimulus to • Temporarily suspend judgment thorough enquiry... • Use reason to make ideas useful - Dewey Ben Simo Ben@QualityFrog.com Sep-12 20 10

9/8/2012 IMAGINATION CAN BE DANGEROUS • Don’t repress it o Risk going astray What merely annoys and • Balance it discourages a person not o Criticism accustomed to o Judgment thinking ... is a stimulus and guide to the • Most hypotheses are wrong trained o Check your work enquirer. o Detect and correct mistakes quickly -Dewey Ben Simo Ben@QualityFrog.com Sep-12 21 IMAGINATION GETTING UNSTUCK Temporary Abandonment • Let it be • Return once old thought In research most of the time associations are less strong progress is • Flaws in thinking difficult and often one is up against become apparent what appears to be a "brick wall". - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 22 11

9/8/2012 IMAGINATION GETTING UNSTUCK Discussion • Useful suggestions • Pooling information may trigger new ideas • Detection of error • Stimulating, refreshing • Escape conditioned thinking Productive mental o Explaining a problem effort is often helped requires clarifying information by intellectual o Questioning by others disturbs intercourse. our lines of thought - W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 23 INTUITION SUDDEN ENLIGHTENMENT • Arises from the subconscious • Capture it The really valuable factor is intuition. - Albert Einstein Ben Simo Ben@QualityFrog.com Sep-12 24 12

9/8/2012 REASON LIMITATIONS • Logic has very little to do with discovery or invention o Logic builds on what is already thought to be so Great discoveries have o Discovery often been made by means of requires disregard experiments devised with complete disregard for for current beliefs well accepted beliefs. -W. I. B. Beveridge Ben Simo Ben@QualityFrog.com Sep-12 25 REASON SAFEGUARDS • Don’t confuse interpretation with Research is fundamentally a state of results mind involving continual o Recognize that re-examination of doctrines and axioms generalizations can upon which current never be proved thought and action are based. It is, therefore, o Don’t place excessive critical of trust in existing practices generalizations - Theobald Smith Ben Simo Ben@QualityFrog.com Sep-12 26 13