Object-Oriented Software Engineering Conquering Complex and Changing Systems Testing Chapter 9,
Preliminaries ♦ Written exam on � for Bachelors of Informatik, and � for other students who are not in the Informatik Diplom track ♦ Date: February 8th, 14:30-16:00 ♦ Room: S1128 Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 2
Outline ♦ Terminology ♦ System testing � Function testing ♦ Types of errors � Structure Testing ♦ Dealing with errors � Performance testing ♦ Quality assurance vs Testing � Acceptance testing ♦ Component Testing � Installation testing � Unit testing � Integration testing ♦ Testing Strategy ♦ Design Patterns & Testing Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 3
Terminology ♦ Reliability: The measure of success with which the observed behavior of a system confirms to some specification of its behavior. ♦ Failure: Any deviation of the observed behavior from the specified behavior. ♦ Error: The system is in a state such that further processing by the system will lead to a failure. ♦ Fault (Bug): The mechanical or algorithmic cause of an error. There are many different types of errors and different ways how we can deal with them. Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 4
What is this? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 5
Erroneous State (“Error”) Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 6
Algorithmic Fault Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 7
Mechanical Fault Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 8
How do we deal with Errors and Faults? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 9
Verification? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 10
Modular Redundancy? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 11
Declaring the Bug as a Feature? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 12
Patching? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 13
Testing? Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 14
Examples of Faults and Errors ♦ Faults in the Interface ♦ Mechanical Faults (very ♦ Faults in the Interface ♦ Mechanical Faults (very specification hard to find) specification hard to find) � Mismatch between what the � Documentation does not � Mismatch between what the � Documentation does not client needs and what the match actual conditions or client needs and what the match actual conditions or server offers operating procedures server offers operating procedures � Mismatch between ♦ Errors � Mismatch between ♦ Errors requirements and requirements and � Stress or overload errors � Stress or overload errors implementation implementation � Capacity or boundary errors � Capacity or boundary errors ♦ Algorithmic Faults ♦ Algorithmic Faults � Timing errors � Timing errors � Missing initialization � Missing initialization � Throughput or performance � Throughput or performance � Branching errors (too soon, � Branching errors (too soon, errors errors too late) too late) � Missing test for nil � Missing test for nil Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 15
Dealing with Errors ♦ Verification: � Assumes hypothetical environment that does not match real environment � Proof might be buggy (omits important constraints; simply wrong) ♦ Modular redundancy: � Expensive ♦ Declaring a bug to be a “feature” � Bad practice ♦ Patching � Slows down performance ♦ Testing (this lecture) � Testing is never good enough Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 16
Another View on How to Deal with Errors ♦ Error prevention (before the system is released): � Use good programming methodology to reduce complexity � Use version control to prevent inconsistent system � Apply verification to prevent algorithmic bugs ♦ Error detection (while system is running): � Testing: Create failures in a planned way � Debugging: Start with an unplanned failures � Monitoring: Deliver information about state. Find performance bugs ♦ Error recovery (recover from failure once the system is released): � Data base systems (atomic transactions) � Modular redundancy � Recovery blocks Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 17
Some Observations ♦ It is impossible to completely test any nontrivial module or any system � Theoretical limitations: Halting problem � Practial limitations: Prohibitive in time and cost ♦ Testing can only show the presence of bugs, not their absence (Dijkstra) Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 18
Testing takes creativity ♦ Testing often viewed as dirty work. ♦ To develop an effective test, one must have: � Detailed understanding of the system � Knowledge of the testing techniques � Skill to apply these techniques in an effective and efficient manner ♦ Testing is done best by independent testers � We often develop a certain mental attitude that the program should in a certain way when in fact it does not. ♦ Programmer often stick to the data set that makes the program work � "Don’t mess up my code!" ♦ A program often does not work when tried by somebody else. � Don't let this be the end-user. Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 19
Testing Activities Requirements Requirements Subsystem Unit System Analysis Analysis Test Code Design Document Document Tested User Document Subsystem Manual Subsystem Unit Code Test Integration Tested Functional Subsystem Test Test Functioning Integrated System Subsystems Tested Subsystem Subsystem Unit Code Test All tests by developer All tests by developer Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 20
Testing Activities ctd Client’s Global Understanding User Requirements of Requirements Environment Accepted Validated Functioning System System System Performance Acceptance Installation Test Test Test Usable Tests by client System Tests by client Tests by developer Tests by developer User’s understanding System in Use Tests (?) by user Tests (?) by user Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 21
Fault Handling Techniques Fault Handling Fault Tolerance Fault Avoidance Fault Detection Design Atomic Modular Reviews Methodology Transactions Redundancy Configuration Verification Management Testing Debugging Component Integration System Correctness Performance Testing Testing Testing Debugging Debugging Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 22
Quality Assurance encompasses Testing Quality Assurance Usability Testing Scenario Prototype Product Testing Testing Testing Fault Avoidance Fault Tolerance Atomic Modular Configuration Transactions Redundancy Verification Management Fault Detection Reviews Debugging Walkthrough Inspection Testing Correctness Performance Debugging Debugging Component Integration System Testing Testing Testing Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 23
Component Testing ♦ Unit Testing: � Individual subsystem � Carried out by developers � Goal: Confirm that subsystems is correctly coded and carries out the intended functionality ♦ Integration Testing: � Groups of subsystems (collection of classes) and eventually the entire system � Carried out by developers � Goal: Test the interface among the subsystem Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 24
System Testing ♦ System Testing: � The entire system � Carried out by developers � Goal: Determine if the system meets the requirements (functional and global) ♦ Acceptance Testing: � Evaluates the system delivered by developers � Carried out by the client. May involve executing typical transactions on site on a trial basis � Goal: Demonstrate that the system meets customer requirements and is ready to use ♦ Implementation (Coding) and testing go hand in hand Bernd Bruegge & Allen Dutoit Object-Oriented Software Engineering: Conquering Complex and Changing Systems 25
Recommend
More recommend
