efficient and effective mutation testing supporting the
play

Efficient and Effective Mutation Testing: Supporting the - PowerPoint PPT Presentation

Nov 14, 2022 •573 likes •2.33k views

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/ gkapfham/

  1. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Case? Expected Expected Test Output Output Set Up Method Test Input Output Output Under Test Oracle Test Test Test Clean Up Verdict Verdict The test case fails and a defect is found! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  2. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  3. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  4. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 T 5 T 6 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  5. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  6. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  7. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  8. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  9. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  10. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  11. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  12. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  13. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  14. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  15. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 B 2 B 2 B 2 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  16. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  17. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  18. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  19. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 How Good is Test Suite T ? R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  20. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 How Good is Test Suite T ? Coverage Analysis R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  21. Introduction Mutation Analysis Empirical Evaluation Conclusion Software Testing What is a Test Suite? Test Suite T = � T 1 , T 2 , . . . , T 9 , T 10 � T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 How Good is Test Suite T ? Coverage Analysis Mutation Analysis R 1 R 2 R 3 R 4 R 5 R 6 F 1 F 2 F 3 F 4 B 1 B 2 Requirements R = { R 1 , . . . , R 6 } , Features F = { F 1 , . . . , F 4 } , Bug Fixes B = { B 1 , B 2 } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  22. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  23. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) if(a >= 10) Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  24. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) if(a >= 10) Implemented Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  25. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults if(a > 10) if(a >= 10) Implemented Potential Fault Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  26. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a if(a > 10) if(a >= 10) true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  27. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  28. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 0 if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  29. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 0 if(a > 10) if(a >= 10) true false false true false false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  30. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 15 if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  31. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 15 if(a > 10) if(a >= 10) true true false true true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  32. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  33. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  34. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false Can the tests differentiate between implemented and potential fault ? Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  35. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false If yes , then the tests are adequate ! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  36. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false If no , then the tests must be improved ! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  37. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Conceptual Faults a a a = 10 if(a > 10) if(a >= 10) true false false true true false Purposefully insert faults in order to implement quality software! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  38. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  39. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  40. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  41. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  42. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  43. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  44. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  45. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Methodically inject small syntactical faults into the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  46. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Mutation Mutation Mutation Mutation Operator Operator Operator Operator Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  47. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  48. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  49. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  50. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  51. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  52. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  53. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  54. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  55. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Execute the test suite after enabling a single mutant in the program under test Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  56. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 The test suite Execute the cannot kill the test suite after mutant – either enabling a a test suite single mutant weakness or in the program an equivalent under test mutant! Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  57. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Overview of Mutation Analysis Test Case T 1 Test Case T 2 Test Case T 3 Test Case T 4 Repeat this Execute the process for test suite after all of the enabling a test cases single mutant and mutants in the program – calculate mutation score under test when finished Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  58. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Mutation Analysis Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  59. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  60. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  61. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  62. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Syntax Tree Transformation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  63. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Syntax Tree Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  64. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Mutation Analysis Solutions Conditional Mutation Compiler Syntax Tree Integrated Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  65. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Challenges Comprehensive Mutation Empirical Study Analysis Solutions Conditional Mutation Compiler Syntax Tree Integrated Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  66. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Contributions of this Presentation Efficient Technique - Fully Integrated into the Java 6 SE Compiler Efficient Challenges Comprehensive Mutation Empirical Study Analysis Solutions Conditional Mutation Compiler Syntax Tree Integrated Transformation Expressions and Statements Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  67. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max( int a, int b){ int max = a; if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  68. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; y = a * x; Methodically y += b; inject small return y; } syntactical public int max( int a, int b){ faults into int max = a; the program under test if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  69. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; y = a * x; y += b; return y; } public int max( int a, int b){ int max = a; if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  70. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ int a=3, b=1, y; • y = a - x; • y = a + x; ⇒ y = a * x; = • y = a / x; y += b; return y; } public int max( int a, int b){ int max = a; • if(b < a) • if(b != a) if (b>a){ ⇒ = max=b; • if(b == a) } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  71. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ Unbiased int a=3, b=1, y; and powerful y = a * x; method for assessing y += b; oracles and return y; } input values public int max( int a, int b){ int max = a; if (b>a){ max=b; } return max; } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  72. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Understanding Mutation Analysis public int eval( int x){ Unbiased int a=3, b=1, y; and powerful y = a * x; method for assessing y += b; oracles and return y; } input values public int max( int a, int b){ Useful method int max = a; for fault seeding if (b>a){ during the max=b; empirical study } of testing return max; techniques } Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  73. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutant Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  74. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Mutation Mutant Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  75. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Program Mutation Mutant Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  76. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Program Mutation Mutant Mutants Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  77. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  78. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation High Time Over- head for Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  79. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation Mutation Analysis High Time Over- head for Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

  80. Introduction Mutation Analysis Empirical Evaluation Conclusion Fundamental Concepts Mutation Analysis Challenges Often Yields a Substantial Num- Program ber of Mutants Mutation Mutant Mutants Operators Generation Mutation Tests Analysis High Time Over- head for Generation Kapfhammer Allegheny College Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects

Recommend

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with Mark Harman and William Langdon Agenda Why Higher Order Mutation Testing? Search for interesting HOMs Milu mutation testing tool Scalability and

669 views • 38 slides
Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis Ren Just 1 , 2 &amp; Gregory M. Kapfhammer 3 &amp; Franz Schweiggert 2 1 University of Washington, USA 2 Ulm University,

1.18k views • 85 slides
Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings College, London W. B. Langdon, Crest 1 Introduction What is mutation testing 2 objectives: Hard to kill, little change to source Higher

417 views • 29 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

350 views • 6 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

306 views • 7 slides
Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Introduction Mutation Operators Conclusion Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015 P. Delgado-Prez UCASE (University of Cdiz) Advances in Mutation Testing Research for C++ TAROT:

694 views • 45 slides
Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis Department of Informatics Athens University of Economics and Business 1 Mutation Testing White-box, fault-based testing technique Considered as

540 views • 22 slides
Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15 2 Sunday, October 4, 15 3 Sunday, October 4, 15 Mutation Testing 4 Sunday, October 4, 15 What is mutation testing? Code under test + test

449 views • 32 slides
MAJOR: An Efficient and Extensible Tool for Mutation Analysis in a Java Compiler e Just 1 , Franz

MAJOR: An Efficient and Extensible Tool for Mutation Analysis in a Java Compiler e Just 1 , Franz

MAJOR: An Efficient and Extensible Tool for Mutation Analysis in a Java Compiler e Just 1 , Franz Schweiggert 1 , and Gregory M. Kapfhammer 2 Ren 1 Ulm University, Germany 2 Allegheny College, USA 26th International Conference on Automated

743 views • 32 slides
Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff, David R. White and John A. Clark. The 13th CREST Open Workshop Thursday 12 May 2011 Outline Invariants Using GP to find Invariants Identifying

417 views • 38 slides
Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common way to determine the quality of a test suite How good are my tests? is to measure its coverage - we ve already seen quite a number of

423 views • 38 slides
GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra, J. de Lara Modelling&amp;Software Engineering Research Group http://miso.es Universidad Autnoma de Madrid (Spain) Mercedes G. Merayo Design

350 views • 13 slides
The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

Software Testing Challenges Structural Testing Regression Testing Mutation Testing Future Work Conclusion The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer Department of Computer Science

578 views • 23 slides
Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi ciently deep ? 2 Test suite depth Mutation testing 3 Program Generate Mutants 4 Program Generate Mutants 5 Program Generate Mutants Mutant

653 views • 38 slides
Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa Misailovic 2 , August Shi 2 ICSE NIER 2017 Buenos Aires, Argentina May 24, 2017 1 2 CCF-1409423, CCF-1421503, CCF-1566363, CCF-1629431, CCF-1319688,

445 views • 31 slides
1 Mutation - selection equilibrium 1. Mutation pressure: ( ) ( ) = Let = the

1 Mutation - selection equilibrium 1. Mutation pressure: ( ) ( ) = Let = the

Population Genetics 7: transient verses equilibrium polymorphism Mutation - selection equilibrium Mutation pressure and selection can operate in opposite directions as a force for change in allele frequencies in populations. Note that

359 views • 9 slides
Semantic Mutation Testing John A. Clark, Haitao Dan, Robert M Hierons The 8th CREST Open

Semantic Mutation Testing John A. Clark, Haitao Dan, Robert M Hierons The 8th CREST Open

Semantic Mutation Testing John A. Clark, Haitao Dan, Robert M Hierons The 8th CREST Open Workshop, 27-10-2010 An example: cruise control Question What happens in no_vehicle_in_front if brake and level=increase? Another question What

376 views • 21 slides
Effective Policies for Supporting Education and Employment of Women in Science and Technology :

Effective Policies for Supporting Education and Employment of Women in Science and Technology :

Effective Policies for Supporting Education and Employment of Women in Science and Technology : Case of Korea Kong-Ju-Bock Lee National Institute for Supporting Women in S&amp;T Ewha Womans University - 1 - NIS WIST UNESCO Paris France

335 views • 15 slides
Title II-A: Supporting Effective Instruction Consolidated Federal and State Grant Application

Title II-A: Supporting Effective Instruction Consolidated Federal and State Grant Application

Title II-A: Supporting Effective Instruction Consolidated Federal and State Grant Application Spring 2019 Teresa A. A. Bu Burgess Program Coordinator Supporting Schools and Students to Achieve SHERRI YBARRA, ED.S., SUPERINTENDENT OF PUBLIC

694 views • 20 slides
Mutation Testing How good your tests are 2017 whoami iOS Developer by day compiler

Mutation Testing How good your tests are 2017 whoami iOS Developer by day compiler

Mutation Testing How good your tests are 2017 whoami iOS Developer by day compiler hacker by night https://twitter.com/1101_debian https://lowlevelbits.org https://systemundertest.org Mars Climate Orbiter Mars Climate

1k views • 85 slides
Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 &amp; Gregory

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 &amp; Gregory

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 &amp; Gregory M. Kapfhammer 2 &amp; Franz Schweiggert 1 Ren 1 Ulm University, Germany 2 Allegheny College, USA 6th International Workshop on the Automation of

946 views • 73 slides
EASI Emission Testing for PTI Dietmar Muchitsch Head of Product Management, AVL DiTest, Austria

EASI Emission Testing for PTI Dietmar Muchitsch Head of Product Management, AVL DiTest, Austria

EASI Emission Testing for PTI Dietmar Muchitsch Head of Product Management, AVL DiTest, Austria 05.03.2014 | Page 2 E E A S I Emission A S I Emission testing for PTI testing for PTI Lets make it EFFECTIVE: Efficient

412 views • 17 slides
Killing Bugs in a Black Box with Model-based Mutation Testing Bernhard K. Aichernig Institute of

Killing Bugs in a Black Box with Model-based Mutation Testing Bernhard K. Aichernig Institute of

Institute of Software Technology tugraz Killing Bugs in a Black Box with Model-based Mutation Testing Bernhard K. Aichernig Institute of Software Technology Graz University of Technology, Austria MT CPS Workshop Vienna, 11 Apr 2016 B.K.

1.49k views • 111 slides
efficient and cost effective utilization of electrical power in accelerator based research

efficient and cost effective utilization of electrical power in accelerator based research

EnEfficient sustainable and energy efficient technologies M.Seidel, PSI network related to: efficient and cost effective utilization of electrical power in accelerator based research facilities EuCARD-2 is co-funded by the partners and the

338 views • 15 slides

More recommend