mutation testing of memory related operators
play

Mutation Testing of Memory- Related Operators Jay Nanavati, Fan Wu, - PowerPoint PPT Presentation

Mar 29, 2024 •312 likes •606 views

Mutation Testing of Memory- Related Operators Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL, UK Mutation Testing Test case Test case Test case Test case Test case Test case Test case Jay Nanavati, Fan Wu, Mark

  1. Mutation Testing of Memory- Related Operators Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL, UK

  2. Mutation Testing Test case Test case Test case Test case Test case Test case Test case Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  3. Mutation Testing Test case Test case Test case Test case Test case Test case Test case Mutants Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  4. Motivation Bug #68942 Use after free Submitted: 2015-01-29 07:20 UTC Reference: https://bugs.php.net/bug.php?id=68942 if (zend_hash_find(...) == SUCCESS) { if (zend_hash_find(...) == SUCCESS) { convert_to_long(*z_timezone_type); if (SUCCESS == timezone_initialize(...)) { return SUCCESS; } Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  5. Motivation Bug #68942 Use after free Submitted: 2015-01-29 07:20 UTC Reference: https://bugs.php.net/bug.php?id=68942 if (zend_hash_find(...) == SUCCESS) { if (zend_hash_find(...) == SUCCESS) { convert_to_long(*z_timezone_type); if (SUCCESS == timezone_initialize(...)) { return SUCCESS; } Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  6. Motivation - if (zend_hash_find(...) == SUCCESS) { + if (zend_hash_find(...) == SUCCESS && Z_TYPE_PP(z_timezone_type) == IS_LONG) { if (zend_hash_find(...) == SUCCESS) { - convert_to_long(*z_timezone_type); if (SUCCESS == timezone_initialize(...)) { return SUCCESS; } Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  7. Motivation Test case Test case Test case Test case Test case Test case Test case Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  8. Motivation Test case Test case Test case Test case Test case Test case Test case Mutants Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  9. Motivation Test case Test case Test case Test case Test case Test case Test case weakpoint Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  10. Memory Mutation Operators Uninitialized Memory Access Faulty Memory Allocation Faulty Heap Management Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  11. Memory Mutation Operators Uninitialized Memory Access Uninitialized memory REC2M calloc(k, sizeof(T)) malloc(k*sizeof(T)) Use-after-free RMNA str = NULL str Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  12. Memory Mutation Operators Faulty Memory Allocation Use-before-allocation REDAWN malloc(k*sizeof(T)) NULL Buffer overflow REDAWZ malloc(k*sizeof(T)) malloc(0) RESOTPE malloc(k*sizeof(T)) malloc(k*sizeof(T*)) REMSOTP malloc(k*sizeof(T*)) malloc(k*sizeof(T)) Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  13. Memory Mutation Operators Faulty Heap Management Memory leaks RMFS free(str) REM2A malloc(k*sizeof(T)) alloc(k*sizeof(T)) REC2A calloc(k, sizeof(T)) alloc(k*sizeof(T)) Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  14. Weakly Killing Criteria Memory Fault Detection Control Flow Deviation Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  15. Weakly Killing Criteria Memory Fault Detection (MFD)   t, MFD(M, t) MFD(P, t) Valgrind Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  16. Weakly Killing Criteria Control Flow Deviation (CFD) P M Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  17. Weakly Killing Criteria Control Flow Deviation (CFD) P M   t, CFD(M, t) CFD(P, t) Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  18. Research Questions RQ1 What are the characteristics of the proposed Memory Mutation Operators? RQ1a What is the prevalence of Memory Mutants? RQ1b How effective is each Memory Mutation Operator in inserting memory faults? RQ1c What is the Mutation Score for the Traditional criterion applied against the Memory Mutants? Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  19. Research Questions RQ2 What is the reduction rate of survived mutants after introducing Memory Fault Detection and Control Flow Deviation criteria? RQ3 What is the relation between MFD and CFD criteria? All Mutants MFD T CFD Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  20. Experiments No. Program LoC 1 PeerWireProtocol 1547 2 Craft 731 3 CfixedArraylist 497 4 ChashMapViaLinkedList 488 5 CAVLTree 405 6 CpseudoLRU 384 7 CHashMapViaQuadraticProbing 1097 8 CtextureAtlas 745 9 Csplaytree 834 10 CstreamingBencodeReader 371 11 CSparseCounter 328 12 Cheap 207 13 CcircularBuffer 118 14 ClinkedListQueue 200 15 CbipBuffer 118 16 Cbitfield 87 Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  21. Results (RQ1) RQ1a What is the prevalence of Memory Mutants? Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  22. Results (RQ1) RQ1b How effective is each Memory Mutation Operator in inserting memory faults? RQ1c What is the Mutation Score for the Traditional criterion applied against the Memory Mutants? Category Mutation Generated Survived Mutation Operator Mutants Mutants Score Uninitialized REC2M 30 25 0.167 Memory Access RMNA 39 21 0.462 Faulty Memory REDAWN 65 12 0.815 Allocation REDAWZ 63 35 0.444 RESOTPE 48 28 0.417 REMSOTP 5 5 0.000 Faulty Heap RMFS 53 53 0.000 Management REM2A 27 16 0.407 REC2A 29 6 0.793 All 359 201 0.440 Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  23. Results (RQ2) RQ2 What is the reduction rate of survived mutants after introducing Memory Fault Detection and Control Flow Deviation criteria? Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  24. Results (RQ3) RQ3 What is the relation between MFD and CFD criteria? All Mutants MFD T CFD     MFD T CFD CFD T MFD   c c MFD CFD   MFD  CFD T MFD  CFD T Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  25. Results (RQ3) RQ3 What is the relation between MFD and CFD criteria? All Mutants MFD T CFD Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  26. Conclusion & Extension Proposed Memory Mutation Operators Introduced MFD & CFD, reduced survived mutants Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  27. Conclusion & Extension Compare with traditional operators Extend the comparison between traditional strong killing criterion and MFD/CFD All Mutants MFD T CFD Jay Nanavati, Fan Wu, Mark Harman, Yue Jia, Jens Krinke UCL

  28. All Mutants MFD T CFD Category Mutation Generated Survived Mutation Operator Mutants Mutants Score Uninitialized REC2M 30 25 0.167 Memory Access RMNA 39 21 0.462 Faulty Memory REDAWN 65 12 0.815 Allocation REDAWZ 63 35 0.444 RESOTPE 48 28 0.417 REMSOTP 5 5 0.000 Faulty Heap RMFS 53 53 0.000 Management REM2A 27 16 0.407 REC2A 29 6 0.793 All 359 201 0.440

Recommend

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015

Introduction Mutation Operators Conclusion Advances in Mutation Testing Research for C++ Pedro Delgado-Prez TAROT: Intro Talk June 2015 P. Delgado-Prez UCASE (University of Cdiz) Advances in Mutation Testing Research for C++ TAROT:

694 views • 45 slides
Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and

Using Non-Redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis Ren Just 1 , 2 & Gregory M. Kapfhammer 3 & Franz Schweiggert 2 1 University of Washington, USA 2 Ulm University,

1.18k views • 85 slides
Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with

Milu: A Higher Order Mutation Testing Tool Yue Jia University College London Joint work with Mark Harman and William Langdon Agenda Why Higher Order Mutation Testing? Search for interesting HOMs Milu mutation testing tool Scalability and

669 views • 38 slides
Escaping Large Deceptive Basins of Attraction with Heavy-Tailed Mutation Operators Tobias

Escaping Large Deceptive Basins of Attraction with Heavy-Tailed Mutation Operators Tobias

Escaping Large Deceptive Basins of Attraction with Heavy-Tailed Mutation Operators Tobias Friedrich, Francesco Quinzan, Markus Wagner How to mutate? I mean: mutation rate, ? Many packages do this: if n is the length of a solution, then

654 views • 26 slides
Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings

Multi-Objective Higher Order Mutation Testing with Genetic Programming W. B. Langdon Kings College, London W. B. Langdon, Crest 1 Introduction What is mutation testing 2 objectives: Hard to kill, little change to source Higher

417 views • 29 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

306 views • 7 slides
1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

1 Mutation pressure Let = the mutation rate from A a Let = the mutation rate from a

Population Genetics 5: Mutation pressure Mutation pressure Table 1 : Estimates of per generation mutation rates for a range of organisms Organism Per nucleotide rate Genomic rate RNA GENOMES 1.97 10 -5 Poliovirus 0.15 1.10 10 -4

350 views • 6 slides
Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis

Automating the Generation of Mutation-based Test Cases Mik ike e Pap apad adakis akis Department of Informatics Athens University of Economics and Business 1 Mutation Testing White-box, fault-based testing technique Considered as

540 views • 22 slides
Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15

Mutation Testing in Python Austin Bingham @austin_bingham @sixty_north 1 Sunday, October 4, 15 2 Sunday, October 4, 15 3 Sunday, October 4, 15 Mutation Testing 4 Sunday, October 4, 15 What is mutation testing? Code under test + test

449 views • 32 slides
Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff,

Searching for Program Invariants using Genetic Programming and Mutation Testing Sam Ratcliff, David R. White and John A. Clark. The 13th CREST Open Workshop Thursday 12 May 2011 Outline Invariants Using GP to find Invariants Identifying

417 views • 38 slides
Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by

Efficient and Effective Mutation Testing: Supporting the Implementation of Quality Software by Purposefully Inserting Defects Gregory M. Kapfhammer Department of Computer Science Allegheny College http://www.cs.allegheny.edu/ gkapfham/

2.33k views • 174 slides
Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common

Mutation Testing Software Engineering Gordon Fraser Saarland University 1 The most common way to determine the quality of a test suite How good are my tests? is to measure its coverage - we ve already seen quite a number of

423 views • 38 slides
GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra,

GENERATION O OF MUTATION T TESTING T TOOLS WITH WODEL-TEST TEST P. Gmez-Abajo , E. Guerra, J. de Lara Modelling&Software Engineering Research Group http://miso.es Universidad Autnoma de Madrid (Spain) Mercedes G. Merayo Design

350 views • 13 slides
The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer

Software Testing Challenges Structural Testing Regression Testing Mutation Testing Future Work Conclusion The Theory and Practice of Software Testing: Can we Test it? Yes we Can! Gregory M. Kapfhammer Department of Computer Science

578 views • 23 slides
Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi

Mutation Testing Reid Holmes Key questions Is a test suite: Su ffi ciently broad ? Su ffi ciently deep ? 2 Test suite depth Mutation testing 3 Program Generate Mutants 4 Program Generate Mutants 5 Program Generate Mutants Mutant

653 views • 38 slides
Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa

Mutation Testing Meets Approximate Computing Milos Gligoric 1 , Sarfraz Khurshid 1 , Sasa Misailovic 2 , August Shi 2 ICSE NIER 2017 Buenos Aires, Argentina May 24, 2017 1 2 CCF-1409423, CCF-1421503, CCF-1566363, CCF-1629431, CCF-1319688,

445 views • 31 slides
DESIGN, SIMULATION, TESTING CSSE 120 Rose Hulman Institute of Technology += and related

DESIGN, SIMULATION, TESTING CSSE 120 Rose Hulman Institute of Technology += and related

DESIGN, SIMULATION, TESTING CSSE 120 Rose Hulman Institute of Technology += and related operators (- =, *=, ) a += b is equivalent to a = a + b IDLE 1.2.1 >>> x = 5 >>> x += 6; print x 11 >>> x *= 2; print x

653 views • 31 slides
1 Mutation - selection equilibrium 1. Mutation pressure: ( ) ( ) = Let = the

1 Mutation - selection equilibrium 1. Mutation pressure: ( ) ( ) = Let = the

Population Genetics 7: transient verses equilibrium polymorphism Mutation - selection equilibrium Mutation pressure and selection can operate in opposite directions as a force for change in allele frequencies in populations. Note that

359 views • 9 slides
Semantic Mutation Testing John A. Clark, Haitao Dan, Robert M Hierons The 8th CREST Open

Semantic Mutation Testing John A. Clark, Haitao Dan, Robert M Hierons The 8th CREST Open

Semantic Mutation Testing John A. Clark, Haitao Dan, Robert M Hierons The 8th CREST Open Workshop, 27-10-2010 An example: cruise control Question What happens in no_vehicle_in_front if brake and level=increase? Another question What

376 views • 21 slides
Mutation Testing How good your tests are 2017 whoami iOS Developer by day compiler

Mutation Testing How good your tests are 2017 whoami iOS Developer by day compiler

Mutation Testing How good your tests are 2017 whoami iOS Developer by day compiler hacker by night https://twitter.com/1101_debian https://lowlevelbits.org https://systemundertest.org Mars Climate Orbiter Mars Climate

1k views • 85 slides
Memory Testing 1 Memory Cells Per Chip 2 1 Test Time in Seconds (Memory Size n Bits, Memory

Memory Testing 1 Memory Cells Per Chip 2 1 Test Time in Seconds (Memory Size n Bits, Memory

Memory Testing 1 Memory Cells Per Chip 2 1 Test Time in Seconds (Memory Size n Bits, Memory Cycle Time 60ns) Size Number of Test Algorithm Operations n 2 n 3/2 n n n X log 2 n 64.5 0.06 1 Mb 1.26 18.3 hr 515.4 4 Mb 0.25

558 views • 30 slides
Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 & Gregory

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 & Gregory

Using Conditional Mutation to Increase the Efficiency of Mutation Analysis e Just 1 & Gregory M. Kapfhammer 2 & Franz Schweiggert 1 Ren 1 Ulm University, Germany 2 Allegheny College, USA 6th International Workshop on the Automation of

946 views • 73 slides
What is testing? A technical investigation done to expose quality-related information about the

What is testing? A technical investigation done to expose quality-related information about the

The Essence of Testing Slides are courtesy of Vassilios Tzerpos What is testing? A technical investigation done to expose quality-related information about the product under test EOT2 Defining Testing Technical Logic,

709 views • 35 slides
Operators of equivalent sorting power and related Wilf-equivalences Mathilde Bouvel (LaBRI,

Operators of equivalent sorting power and related Wilf-equivalences Mathilde Bouvel (LaBRI,

Operators of equivalent sorting power and related Wilf-equivalences Mathilde Bouvel (LaBRI, Bordeaux, France) joint work with Michael Albert (University of Otago, New Zealand) CanaDAM, June 10, 2013 Operators of equivalent sorting power . . .

764 views • 48 slides

More recommend