What’s Next INF 117 Project in Software Engineering Lecture Notes - Spring Quarter, 2008 Michele Rousseau Set 8 - Testing Set 8 - Testing 2 Announcements Today’s Class k Drop Boxes k Testing ● We will use drop boxes for the remainder of the qtr ● Coverage-Based Testing ● Please still post all deliverables ◘ EXCEPT: Team Appraisals, Peer Evals & Course Logs ● Equivalence Partitioning k Due: Thursday ● Boundary Value Testing ● Design Iteration #3 ● Code Iteration #1 ● Project Plan #3 k Friday: Cust Approval of Design Set 8 - Testing 3 Set 8 - Testing 4 Motivation The Purpose of Testing Design and coding are creative. but… k People are not perfect k Testing is Destructive ● We make errors in design and code ● The primary goal is to “break” the software k Very often the same person does both ● Goal of testing: given some code, coding and testing g g uncover as many errors are possible ibl ● This is not ideal… why? k Important and expensive activity ● Need “split personality”: ◘ when you start testing, become paranoid and ● Not unusual to spend 30-40% of malicious ● Surprisingly hard to do: people don’t like total project effort on testing finding out that they made mistakes Set 8 - Testing 5 Set 8 - Testing 6 1
Testing Approach Testing Testing is a process of executing software k Unit Testing with the intent of finding errors k Integration Testing k Good testing has a high probability of k System Testing finding as-yet-undiscovered errors finding as-yet-undiscovered errors k Regression Testing k Successful testing discovers unknown errors k If did not find any errors, need to ask “Was our testing approach is good?” Set 8 - Testing 7 Set 8 - Testing 8 V-Model of Development & Testing Fundamental Testing Questions (the big picture) k Test Criteria: What should we test? Develop Requirements Execute System Tests Requirements Review k Test Oracle: Is the test correct? Develop Acceptance Tests Acceptance Test Review k Test Adequacy: How much is Design Execute Integration Tests Design Review enough? Develop Integration Tests Integration Tests Review k Test Process: Is our testing Code Execute Unit Tests effective? Code Review Develop Unit Tests How to make the most of limited resources? Set 8 - Testing Unit Tests Review 9 Set 8 - Testing 10 Some Commonly Used Testing Coverage-Based Testing Approaches k Flow Graphs k Coverage or Control-flow based ● Control Flow k Data-flow based ◘ Partial order of Statement Execution k Equivalence Partitioning k Equivalence Partitioning k Data Flow k D Fl k Boundary Value Analysis ◘ Flow of values from Definition to Variables Graph representation of control flow and data flow relationships Set 8 - Testing 11 Set 8 - Testing 12 2
Prog P’s Control Flow Graph All-Statements Coverage (CFG) k Select test cases such that every 6 7 10 node in the graph is visited F T T T 9 F T ● Also called node coverage 9 ´ 2,3,4 5 9 a 9 b 14 F ◘ G ara tees that e er state e t i ◘ Guarantees that every statement in F F the source code is executed at least 12 once 1 1 function P return INTEGER is 2 2 begin 9 if (Y < 20 and then X mod 2 = 0) then 3 k Selects minimal number of test 3 X, Y: INTEGER; 10 Y := Y + 20; 4 4 READ(X); READ(Y); 11 else 5 5 while (X > 10) loop 12 Y := Y – 20; cases 6 6 X := X – 10; 13 end if ; 7 7 exit when X = 10; 14 return 2*X + Y; 8 8 end loop ; 15 end P; Set 8 - Testing 13 Set 8 - Testing 14 All-Branches Coverage All-Statements Coverage of P k Select test cases such that every At least 2 test cases needed branch in the graph is visited 6 7 10 ◘ Guarantees that every branch in the F T T T source code is executed at least once source code is executed at least once F F 2,3,4 5 9 14 k More thorough than All- F Statements coverage Example all-statements-adequate 12 test set: ● More likely to reveal logical errors (X = 20, Y = 10) < 2,3,4,5,6,7,9,10,14> (X = 20, Y = 30) < 2,3,4,5,6,7,9,12,14> Set 8 - Testing 15 Set 8 - Testing 16 All-Branches Coverage of P All-Edges Coverage At least 2 test cases needed k Select test cases such that every edge in the graph is visited 6 7 10 F T ◘ Takes complex statements into T T F consideration treats them as consideration – treats them as 2,3,4 2,3,4 5 5 9 9 14 14 separate nodes F k More thorough than All-Branches 12 Example all-branches-adequate test set: coverage (X = 20, Y = 10) < 2,3,4,5,6,7,9,10,14> ● More likely to reveal logical errors (X = 15, Y = 30) < 2,3,4,5,6,7,5,9,12,14> Set 8 - Testing 17 Set 8 - Testing 18 3
All-Edges Coverage of P All-Paths Coverage At least 3 test cases needed k Path coverage 6 7 10 ● Select test cases such that every F T T T path in the graph is visited F T 2,3,4 5 9a 9b 14 ● L ● Loops are a problem bl F F ◘ 0, 1, average, max iterations 12 Example all-edges-adequate test set: (X = 20, Y = 10) k Most thorough… < 2,3,4,5,6,7,9a,9b,10,14> (X = 5, Y = 30) …but is it feasible? < 2,3,4,5,9a,12,14> (X = 21, Y = 10) < 2,3,4,5,6,7,5,6,7,5,9a,9b,12,14> Set 8 - Testing 19 Set 8 - Testing 20 Subsumption of Criteria All-Paths Coverage of P k C1 subsumes C2 if any C1- Infinitely many test cases needed adequate test T is also C2- 6 7 10 adequate F T T T F T ● But some T1 satisfying C1 may ● B t T1 ti f i C1 2,3,4 2 3 4 5 5 9 9 a 9 b 9 b 14 14 detect fewer faults than some T2 F F Example all-paths- satisfying C2 adequate test set: 12 (X = 5, Y = 10) (X = 15, Y = 10) (X = 25, Y = 10) (X = 35, Y = 10) … Set 8 - Testing 21 Set 8 - Testing 22 Data-flow Def-Use Pair k A def-use (DU) pair for variable x x is a Test connections between variable definitions (“write”) and variable uses k pair of nodes (n1,n (n1,n2) ) such that (“read”) ● x x is in DEF(n1) DEF(n1) k Variation of the control flow graph ● the definition of x x at n1 n1 reaches n2 n2 ● A node represents a single statement, not a ● A ode represe ts a si gle state e t ot a ● x is in USE( USE( USE(n2) USE( 2) 2) 2) single-entry-single-exit chain of statements k i.e., the value that is assigned to x x at n1 n1 k Set DEF(n) DEF(n) contains variables that are is used at n2 n2 defined at node n (i.e., they are written) ● Since the definition reaches n2 n2, the k Set USE(n USE(n): : variables that are read value is not “killed” along some path n1… n1…n2 Set 8 - Testing 23 Set 8 - Testing 24 4
Test Adequacy P’s Control and Data Flow Graph Tells you when to stop testing k Coverage-Based Testing X Y ● Coverage metrics X ◘ when sufficient percentage of the program structure has been 6 7 10 exercised X Y k Fault-Based Testing F X X X T T T T T ● Empirical assurance ● E i i l F T ◘ when failures/test curve flatten out 2,3,4 5 9 a 9 b 14 ● Error seeding F ◘ percentage of seeded faults found is proportional to the percentage X F of real faults found Y k Error-Based Testing Y 12 ● faults found in common are representative of total X population of faults Y X ● Equivalence Partitioning Set 8 - Testing 25 Set 8 - Testing 26 Test Criteria Equivalence Partitioning (EQP) k Testing must select a subset of test cases that are likely to reveal failures k Testing technique Testing technique k Test Criteria provide the guidelines, rules, strategy by which test cases are selected ● Reduces the # of test cases ● actual test data ◘ Make the # of test cases manageable ● conditions on test data ● requirements on test data ● requirements on test data ◘ Systematic derivation of test cases S t ti d i ti f t t k Equivalence partitioning is the typical approach ● Reasonably tests the system ● a test of any value in a given class is equivalent to a test of any other value in that class ● if a test case in a class reveals a failure, then any other test case in that class should reveal the failure Basic Principle: ● some approaches limit conclusions to some chosen Some distinctions don’t make a difference class of errors and/or failures Set 8 - Testing 27 Set 8 - Testing 28 EQP : How does it work EQP: Reduces test cases Basic Method: Notice when any element in the partition will produce the same results Divid Divide inputs into inputs into equivalent Input domain (ie find the same faults) Input domain partitioned into four partitions partitions sub-domains. ● i.e. Find a small # set of ● ie Find a small # set of 2 2 1 re representative entative input values 3 4 ● For each Class program behaves in Four test inputs, one an “equivalent” way Large set of selected from each test inputs. sub-domain. ● Smaller test set – but equally effective Set 8 - Testing 29 Set 8 - Testing 30 5
Recommend
More recommend
