Chapter 11, Testing ! Function testing Types of errors ! Structure - PDF document

Outline Object-Oriented Software Engineering ♦ Terminology ♦ System testing Chapter 11, Testing ! Function testing ♦ Types of errors ! Structure Testing ♦ Dealing with errors ! Performance testing Using UML, Patterns, and Java ♦ Quality assurance vs Testing ! Acceptance testing ♦ Component Testing ! Installation testing ! Unit testing ! Integration testing ♦ Testing Strategy ♦ Design Patterns & Testing Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 2 What is this? Erroneous State (“Error”) A failure? An error? A fault? Need to specify the desired behavior first! Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 3 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 4 Algorithmic Fault Mechanical Fault Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 5 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 6 Page 1

Terminology ♦ Reliability: The measure of success with which the observed behavior of a system confirms to some specification of its behavior. ♦ Failure: Any deviation of the observed behavior from the specified behavior. How do we deal with Errors and Faults? ♦ Error: The system is in a state such that further processing by the system will lead to a failure. ♦ Fault (Bug): The mechanical or algorithmic cause of an error. There are many different types of errors and different ways how we can deal with them. Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 7 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 8 Modular Redundancy? Verification? Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 9 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 10 Patching? Declaring the Bug as a Feature? Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 11 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 12 Page 2

Testing? Examples of Faults and Errors ♦ Faults in the Interface ♦ Mechanical Faults (very specification hard to find) ! Mismatch between what the ! Documentation does not client needs and what the match actual conditions or server offers operating procedures ! Mismatch between ♦ Errors requirements and ! Stress or overload errors implementation ! Capacity or boundary errors ♦ Algorithmic Faults ! Timing errors ! Missing initialization ! Throughput or performance ! Branching errors (too soon, errors too late) ! Missing test for null Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 13 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 14 Dealing with Errors Another View on How to Deal with Errors ♦ Error prevention (before the system is released): ♦ Verification: ! Use good programming methodology to reduce complexity ! Assumes hypothetical environment that does not match real ! Use version control to prevent inconsistent system environment ! Proof might be buggy (omits important constraints; simply wrong) ! Apply verification to prevent algorithmic bugs ♦ Error detection (while system is running): ♦ Modular redundancy: ! Testing: Create failures in a planned way ! Expensive ! Debugging: Start with an unplanned failures ♦ Declaring a bug to be a “feature” ! Monitoring: Deliver information about state. Find performance bugs ! Bad practice ♦ Error recovery (recover from failure once the system is released): ♦ Patching ! Data base systems (atomic transactions) ! Slows down performance ! Modular redundancy ♦ Testing (this lecture) ! Recovery blocks ! Testing is never good enough Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 15 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 16 Some Observations Testing takes creativity ♦ It is impossible to completely test any nontrivial module or any ♦ Testing often viewed as dirty work. system ♦ To develop an effective test, one must have: ! Theoretical limitations: Halting problem " Detailed understanding of the system ! Practial limitations: Prohibitive in time and cost " Knowledge of the testing techniques ♦ Testing can only show the presence of bugs, not their absence " Skill to apply these techniques in an effective and efficient manner (Dijkstra) ♦ Testing is done best by independent testers ! We often develop a certain mental attitude that the program should in a certain way when in fact it does not. ♦ Programmer often stick to the data set that makes the program work ! "Don’t mess up my code!" ♦ A program often does not work when tried by somebody else. ! Don't let this be the end-user. Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 17 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 18 Page 3

Testing Activities Testing Activities continued Requirements Requirements Subsystem Unit Client’s System Analysis Analysis Code Test Global User Design Understanding Document Document Requirements Tested User of Requirements Environment Document Subsystem Manual Subsystem Unit Code Test Accepted Validated Functioning Tested Integration System System Functional System Performance Acceptance Installation Subsystem Test Test Test Test Test Functioning Integrated System Subsystems Usable Tested Subsystem Tests by client System Subsystem Unit Tests by developer Code Test User’s understanding All tests by developer System in Use Tests (?) by user Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 19 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 20 Fault Handling Techniques Quality Assurance encompasses Testing Quality Assurance Fault Handling Usability Testing Fault Tolerance Scenario Prototype Product Fault Avoidance Fault Detection Testing Testing Testing Fault Avoidance Fault Tolerance Design Atomic Modular Reviews Methodology Transactions Redundancy Atomic Modular Configuration Configuration Transactions Redundancy Verification Verification Management Management Fault Detection Testing Debugging Reviews Unit Integration System Correctness Performance Debugging Walkthrough Inspection Debugging Debugging Testing Testing Testing Testing Correctness Performance Debugging Debugging Unit Integration System Testing Testing Testing Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 21 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 22 Types of Testing System Testing ♦ Unit Testing: ♦ System Testing: ! Individual subsystem ! The entire system ! Carried out by developers ! Carried out by developers ! Goal: Confirm that subsystems is correctly coded and carries out the intended functionality ! Goal: Determine if the system meets the requirements (functional ♦ Integration Testing: and global) ! Groups of subsystems (collection of classes) and eventually the entire ♦ Acceptance Testing: system ! Evaluates the system delivered by developers ! Carried out by developers ! Goal: Test the interface among the subsystem ! Carried out by the client. May involve executing typical transactions on site on a trial basis ! Goal: Demonstrate that the system meets customer requirements and is ready to use ♦ Implementation (Coding) and testing go hand in hand Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 23 Bernd Bruegge & Allen H. Dutoit Object-Oriented Software Engineering: Using UML, Patterns, and Java 24 Page 4