MaTRIX Maintenance-Oriented Test Requirements Identifier and - PowerPoint PPT Presentation

MaTRIX Maintenance-Oriented Test Requirements Identifier and Examiner Mary Jean Harrold † Taweesup (Term) Apiwattanapong, † Raúl Santelices, † Pavan Kumar Chittimalli, ‡ Alessandro Orso † †College of Computing, Georgia Institute of Technology ‡Tata Research Development & Design Centre, TCS Limited Supported by Tata Consultancy Services (TCS) Limited and by NSF

Regression Testing Program T P P’ Version of P TAIC PART August 2006 2

Regression Testing T-T’ Program T P T’ T’ P’ Version of P T’ T’’ TAIC PART August 2006 3

Regression Testing How well do T, T’,T,’’ How well do T, T’,T,’’ T-T’ or any test suites or any test suites Program exercise P’ with exercise P’ with T P respect to changes? respect to changes? T’ T’ Is there suitable P’ Version guidance for creating of P T’ T’’ new test cases that target the modified behavior? TAIC PART August 2006 4

Motivating Example public class E { n void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 5

Motivating Example change public class E { n void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 6

Motivating Example branches public class E { n void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 7

Motivating Example du-pairs du-pairs public class E { n void simple (int i) { s1 (s1,s2,x) s1 int x = i; c: if (x >= 5){ (s1,s3,x) s2 if (x > 5){ T s2 (s1,s4,x) s3 x = (5/(x-5)); F s3 (s3,s4,x) } s4 (s4,s5,x) s4 x = x - 1; (s4,s6,x) T s5 (s4,s7,x) s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 8

Motivating Example public class E { i=6 n i=1 void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 9

Motivating Example change public class E { i=6 n i=1 void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 10

Motivating Example branches public class E { i=6 n i=1 void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 11

Motivating Example du-pairs du-pairs public class E { i=6 n i=1 i=6 i=1 void simple (int i) { s1 X X (s1,s2,x) s1 int x = i; c: if (x >= 5){ X (s1,s3,x) s2 if (x > 5){ T s2 X (s1,s4,x) s3 x = (5/(x-5)); F s3 X (s3,s4,x) } s4 X X (s4,s5,x) s4 x = x - 1; X (s4,s6,x) T s5 X (s4,s7,x) s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 } x } … } TAIC PART August 2006 12

Motivating Example public class E { i=6 n i=1 void simple (int i) { s1 s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ T s2 s3 x = (5/(x-5)); F s3 } s4 s4 x = x - 1; T s5 s5 if (x == 0){ F s6 print(x); s6 } else { s7 print(10/x); s7 Tests satisfy test requirements } x for criteria but don’t reveal } … fault in s3 } TAIC PART August 2006 13

Motivating Example Criteria require Criteria require public class E { i=6 n i=1 void simple (int i) { s1 • Execution of the • Execution of the s1 int x = i; c: if (x >= 5){ s2 if (x > 5){ change and entities change and entities T s2 s3 x = (5/(x-5)); F affected by change affected by change s3 } s4 s4 x = x - 1; But don’t require T s5 s5 if (x == 0){ • Infection of the state F s6 print(x); s6 after change } else { s7 print(10/x); • Propagation of state s7 Tests satisfy test requirements } to output where it x for criteria but don’t reveal } can be observed … fault in s3 } TAIC PART August 2006 14

Computation of Testing Requirements Criteria require Criteria require i=6 n i=1 s1 • Execution of the • Execution of the change and entities change and entities T s2 F affected by change affected by change s3 s4 But don’t require the criteria { Our technique T s5 • Infection of the state F s6 adds these after change requirements to • Propagation of state s7 to output where it x can be observed TAIC PART August 2006 15

Computation of Testing Requirements public class E { PC SS(x) PC’ SS’(x) void simple (int i) { s1 int x = i; true i 0 true i 0 s2 if (x > 5){ if (x >= 5){ ” ” ” ” s3 x = (5/(x-5)); (i 0 >5) 5/(i 0 -5) (i 0 >=5) 5/(i 0 -5) } s4 x = x - 1; (i 0 <=5) i 0 -1 (i 0 <5) i 0 -1 or or (i 0 >5) 5/(i 0 -5)-1 (i 0 >=5) 5/(i 0 -5)-1 s5 if (x == 0){ ” ” ” ” s6 print(x); (i 0 ==0) 0 (i 0 ==0) 0 } else { (i 0 <=5) ∧ i 0 -1 s7 print(10/x); (i 0 <5) ∧ i 0 -1 … (i 0 !=0) } … (i 0 !=0) or or (i 0 >5) ∧ (i 0 >=5) ∧ 5/(i 0 -5)-1 5/(i 0 -5)-1 } ..(i 0 !=0) ..(i 0 !=0) … } PC—path condition SS—symbolic state TAIC PART August 2006 16

Computation of Testing Requirements public class E { Conditions for propagation of infected PC SS(x) PC’ SS’(x) void simple (int i) { states: s1 int x = i; true i 0 true i 0 s2 if (x > 5){ if (x >= 5){ ” ” ” ” 1. The execution in P’ reaches s i ’ and the s3 x = (5/(x-5)); (i 0 >5) 5/(i 0 -5) (i 0 >=5) 5/(i 0 -5) } execution in P does not reach s i ; or s4 x = x - 1; (i 0 <=5) i 0 -1 (i 0 <5) i 0 -1 or or 2. The execution in P’ reaches s i ’ and the (i 0 >5) 5/(i 0 -5)-1 (i 0 >=5) 5/(i 0 -5)-1 s5 if (x == 0){ execution in P reaches s i ; however, s i ’ and s i ” ” ” ” s6 print(x); (i 0 ==0) 0 (i 0 ==0) 0 have different symbolic states. } else { (i 0 <=5) ∧ i 0 -1 s7 print(10/x); (i 0 <5) ∧ i 0 -1 … (i 0 !=0) } … (i 0 !=0) or or (i 0 >5) ∧ (i 0 >=5) ∧ 5/(i 0 -5)-1 5/(i 0 -5)-1 } ..(i 0 !=0) ..(i 0 !=0) … } PC—path condition SS—symbolic state TAIC PART August 2006 17

Computation of Testing Requirements public class E { But (as we discussed yesterday) But (as we discussed yesterday) PC SS(x) PC’ SS’(x) void simple (int i) { s1 int x = i; • symbolic execution is expensive • symbolic execution is expensive true i 0 true i 0 s2 if (x > 5){ if (x >= 5){ ” ” ” ” • won’t scale to large programs • won’t scale to large programs s3 x = (5/(x-5)); (i 0 >5) 5/(i 0 -5) (i 0 >=5) 5/(i 0 -5) } • can’t be applied for entire paths • can’t be applied for entire paths s4 x = x - 1; (i 0 <=5) i 0 -1 (i 0 <5) i 0 -1 or or (i 0 >5) 5/(i 0 -5)-1 (i 0 >=5) 5/(i 0 -5)-1 • etc. • etc. s5 if (x == 0){ ” ” ” ” s6 print(x); (i 0 ==0) 0 (i 0 ==0) 0 } else { (i 0 <=5) ∧ i 0 -1 s7 print(10/x); (i 0 <5) ∧ i 0 -1 Our technique has two ways to … (i 0 !=0) } … (i 0 !=0) or or improve efficiency (i 0 >5) ∧ (i 0 >=5) ∧ 5/(i 0 -5)-1 5/(i 0 -5)-1 } ..(i 0 !=0) ..(i 0 !=0) … } PC—path condition SS—symbolic state TAIC PART August 2006 18

Computation of Testing Requirements public class E { PC SS(x) PC’ SS’(x) void simple (int i) { s1 int x = i; s2 if (x > 5){ if (x >= 5){ s3 x = (5/(x-5)); } s4 x = x - 1; 1. Perform partial symbolic execution (PSE) beginning immediately before the change s5 if (x == 0){ s6 print(x); • computes conditions in terms of variables } else { immediately before change s7 print(10/x); } • avoids symbolic execution from beginning } of program to change … } TAIC PART August 2006 19

Computation of Testing Requirements public class E { PC SS(x) PC’ SS’(x) void simple (int i) { -- -- -- -- s1 int x = i; s2 if (x > 5){ if (x >= 5){ true x 0 true x 0 s3 x = (5/(x-5)); (x 0 >5) 5/(x 0 -5) (x 0 >=5) 5/(x 0 -5) } s4 x = x - 1; s5 if (x == 0){ s6 print(x); } else { s7 print(10/x); } } … } TAIC PART August 2006 20