State-of-the-Art ! 30-85 errors are made per 1000 lines of source CS 619 Introduction to OO Design code and Development ! extensively tested software contains 0.5-3 errors per 1000 lines of source code Testing ! error distribution: – 60% design, – 40% implementation. Fall 2013 66% of the design errors are not discovered until the software has become operational ! testing is postponed, as a consequence: the later an error is discovered, the more it costs to fix it. � When to Test: the V-model Relative cost of error correction Acceptance Test Requi quireme ments Depl eployed Docum cuments nts System Syste 100 System 50 Syste stem Relea leased Test Speci pecificat System Syste 20 ion Integration Test Syste stem System Syste 10 Design sign In Integra gration 5 Unit Test Modu dules Modu odules 2 Speci pecificat Impl pleme Prepare tests here … … and run them here ! ion nta ntation 1 RE design code test operation SPECIFY & DESIGN WITH TESTABILITY IN MIND � �
When to Test: UP Process What is Testing Error of omission: Neglected specifications: SPECIFICATION Omissions (not found with tests) SYSTEM Errors/Failures = mismatch between specification & system (found with tests) Surprises (sometimes found with tests) Error of commission: net defined by the specification Can reveal the presence of errors NOT their absence � � Terminology Drivers and Stubs Component • A part of the system that can be isolated for testing + an object, a group of objects, one or more subsystems Unit Test Case Driver Stub procedure Under Test procedure • A set of inputs and expected results that exercise a component with the purpose of causing errors and failures call call + a predicate method that answers “true” when the component answers with the expected results for the given input and “false” access to global otherwise variables - “expected results” includes exceptions, error codes,... Test Stub ! Driver: simulates a module that calls the module currently being • A partial implementation of components on which the tested component tested depends + dummy code that provides the necessary input values and ! Stub : simulates a module called by the module currently being behaviour to run the test cases tested Test Driver • A partial implementation of a component that depends on the tested component + a “main()” function that executes a number of test cases � �
Equivalence Partitioning : Example Black-box Testing Example: Binary search ! A.K.A Functional Test, Specification based private int[] _elements; public boolean find(int key) { ... } ! Focus: I/O behavior. If for any given input, we can •pre-condition(s) - Array has at least one element predict the output, then the module passes the test. - Array is sorted – Almost always impossible to generate all possible inputs •post-condition(s) (The element is in _elements and the result is true) ("test cases") or (The element is not in _elements and the result is false) Check input partitions: ! Goal: Reduce number of test cases by equivalence • Do the inputs satisfy the pre-conditions? partitioning: • Is the key in the array? ➡ leads to (at least) 2x2 equivalence classes – Divide input conditions into equivalence classes – Choose test cases for each equivalence class. Check boundary conditions • Is the array of length 1 ? E.g. If an object is supposed to accept a negative number, • Is the key at the start or end of the array? testing one negative number is enough. ➡ leads to further subdivisions � �� (not all combinations make sense) White-box Testing Equivalence Partitioning: Test Data Generate test data that cover all meaningful equivalence partitions. ! A.K.A. Structural testing; program-based. Test Cases Input Output Array length 0 key = 17, elements = { } FALSE ! Treat a component as a “white box”, i.e. you Array not sorted key = 17, elements = { 33, 20, 17, 18 } exception can inspect its internal structure Array size 1, key in array key = 17, elements = { 17 } TRUE ! Internal structure is also design specs; e.g. Array size 1, key not in array key = 0, elements = { 17 } FALSE sequence diagrams, state charts, ... Array size > 1, key is first element key = 17, elements = { 17, 18, 20, 33 } TRUE ! Derive test cases to maximize coverage of that Array size > 1, key is last element key = 33, elements = { 17, 18, 20, 33 } TRUE structure, yet minimize number of test cases Array size > 1, key is in middle key = 20, elements = { 17, 18, 20, 33 } TRUE Array size > 1, key not in array key = 50, elements = { 17, 18, 20, 33 } FALSE … … … �� ��
Statement vs. Branch Coverage White-box Testing Coverage Criteria assignAbsolute(int x) { Consider this program segment, the test set if (x < 0) T = {x=-1} will give statement coverage, ! Coverage metrics x := -x; however not branch coverage z := x; } – Statement coverage : all statements in the programs B0 should be executed at least once Control Flow Graph: (x < 0) true false B1 – Branch coverage : all branches in the program should be executed at least once x := -x Test set {x=-1} does not execute this edge, so it doesn’t – Path coverage : all execution paths in the program give branch coverage B2 should be executed at lest once z := x �� �� Finding the test cases Path Coverage: Determining the Paths Start FindMean (FILE ScoreFile) { float SumOfScores = 0.0; 1 int NumberOfScores = 0; a (Covered by any data) 1 ! float Mean=0.0; float Score; 2 Read(ScoreFile, Score); while (! EOF(ScoreFile) { b (Data set must contain at least one value 2 ! if (Score > 0.0 ) { 3 ! 3 (Positive score) d e (Negative score) SumOfScores = SumOfScores + Score; 4 ! c 5 4 NumberOfScores++; (Data set must h (Reached if either f or } g f be empty) 5 ! 6 e is reached) Read(ScoreFile, Score); 6 ! 7 } j i (Total score > 0.0) (Total score < 0.0) /* Compute the mean and print the result */ 9 8 if (NumberOfScores > 0) { 7 ! k l Mean = SumOfScores / NumberOfScores; 8 ! Exit printf(“ The mean score is %f\n”, Mean); } else printf (“No scores found in file\n”); 9 ! �� �� }
Integration Testing Testing stages Black- & White-Box Testing Module techniques Driver under Test ! Unit testing Test Cases Testing of individual components Test Cases – Test Cases Results Stub Module Module Test Cases ! Integration testing Testing to expose problems arising from the – • Why ? combination of components + The sum is more then its parts, i.e. interfaces (and calls to them) may contain defects too. ! System testing • Who ? Testing the complete system prior to delivery + Person developing the module writes the tests. – • When ? ! Acceptance testing + Top-down: main module before constituting modules + Bottom-up: constituting modules before integrated module Testing by users to check that the system – + In practice: a little bit of both satisfies requirements. Sometimes called alpha testing ## The distinction between unit testing and integration testing is not that sharp! �� �� Regression Testing Acceptance Testing Regression Testing ensures that all things that used to work still work after Acceptance Tests changes. • conducted by the end-user (representatives) • check whether requirements are correctly implemented Regression Test + borderline between verification (“Are we building the system • = re-execution of some subset of tests to ensure that changes have not right?”) and validation (“Are we building the right system?”) caused unintended side effects • tests must avoid regression (= degradation of results) • Regression tests must be repeated often (after every change, every Alpha- & Beta Tests night, with each new unit, with each fix,...) • acceptance tests for “off-the-shelves” software (many unidentified users) • Regression tests may be conducted manually + Alpha Testing + Execution of crucial scenarios with verification of results + Manual test process is slow and cumbersome - end-users are invited at the developer’s site ➡ preferably completely automated - testing is done in a controlled environment + Beta Testing Advantages - software is released to selected customers • Helps during iterative and incremental development - testing is done in “real world” setting, + during maintenance without developers present Disadvantage • Up front investment in maintainability is difficult to sell to the customer �� ��
Recommend
More recommend
