concolic testing
play

Concolic Testing Dynamic Symbolic Execution Marco Probst - PowerPoint PPT Presentation

Dec 19, 2022 •29 likes •919 views

Concolic Testing Dynamic Symbolic Execution Marco Probst Albert-Ludwigs-Universitt Freiburg January 25th, 2016 Marco Probst Concolic Testing 1 / 22 Overview Code Example 1 Unit Testing 2 Random Testing Symbolic Execution Concolic

  1. Concolic Testing Dynamic Symbolic Execution Marco Probst Albert-Ludwigs-Universität Freiburg January 25th, 2016 Marco Probst Concolic Testing 1 / 22

  2. Overview Code Example 1 Unit Testing 2 Random Testing Symbolic Execution Concolic Testing 3 DART Summary 4 Marco Probst Concolic Testing 2 / 22

  3. Overview Code Example 1 Unit Testing 2 Random Testing Symbolic Execution Concolic Testing 3 DART Summary 4 Marco Probst Concolic Testing 3 / 22

  4. Program Under Test Developers writing code [1] . . . 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Marco Probst Concolic Testing 4 / 22

  5. Program Under Test Developers writing code [1] . . . 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } . . . need to test Marco Probst Concolic Testing 4 / 22

  6. Overview Code Example 1 Unit Testing 2 Random Testing Symbolic Execution Concolic Testing 3 DART Summary 4 Marco Probst Concolic Testing 5 / 22

  7. Unit Testing Ensure overall software quality Individual components (e.g. functions) Marco Probst Concolic Testing 6 / 22

  8. Unit Testing Ensure overall software quality Individual components (e.g. functions) Goals ◮ Detect errors ◮ Check corner cases ◮ Provide high code coverage (e.g. path coverage) Marco Probst Concolic Testing 6 / 22

  9. Path Coverage Marco Probst Concolic Testing 7 / 22

  10. Path Coverage Code Example ⇒ Control Flow ⇒ Execution Paths 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Marco Probst Concolic Testing 7 / 22

  11. Path Coverage Code Example ⇒ Control Flow ⇒ Execution Paths 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Marco Probst Concolic Testing 7 / 22

  12. Path Coverage Code Example ⇒ Control Flow ⇒ Execution Paths 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Marco Probst Concolic Testing 7 / 22

  13. Path Coverage Code Example ⇒ Control Flow ⇒ Execution Paths 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Marco Probst Concolic Testing 7 / 22

  14. Path Coverage Code Example ⇒ Control Flow ⇒ Execution Paths 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Marco Probst Concolic Testing 7 / 22

  15. Path Coverage Code Example ⇒ Control Flow ⇒ Execution Paths 1 f(int x, int y) { if (x*x*x > 0) { 2 if (x > 0 && y == 10) { 3 fail(); 4 } 5 } else { 6 if (x > 0 && y == 20) { 7 fail(); 8 } 9 } 10 11 complete(); 12 13 } Contradiction: x <= 0 && x > 0 ⇒ not executable Marco Probst Concolic Testing 7 / 22

  16. Path Coverage 3 possible execution paths Corresponding path conditions Marco Probst Concolic Testing 7 / 22

  17. Path Coverage 3 possible execution paths Corresponding path conditions Optimal: cover all paths Find input set to run program along different paths Marco Probst Concolic Testing 7 / 22

  18. Random Testing Marco Probst Concolic Testing 8 / 22

  19. Random Testing Most naive way of testing Generate random inputs Concrete input values Dynamic execution of program Observe behavior Compare against expected behavior e.g. output or "do not crash" Marco Probst Concolic Testing 8 / 22

  20. Random Testing on Code Example Marco Probst Concolic Testing 9 / 22

  21. Random Testing on Code Example Random inputs for f(int x, int y) Marco Probst Concolic Testing 9 / 22

  22. Random Testing on Code Example Random inputs for f(int x, int y) x = 700, y = 500 Marco Probst Concolic Testing 9 / 22

  23. Random Testing on Code Example Random inputs for f(int x, int y) x = 700, y = 500 x = -700, y = 500 Marco Probst Concolic Testing 9 / 22

  24. Random Testing on Code Example Random inputs for f(int x, int y) x = 700, y = 500 x = -700, y = 500 Similar values are very likely Marco Probst Concolic Testing 9 / 22

  25. Random Testing on Code Example Necessary inputs x > 0, y = 10 Marco Probst Concolic Testing 9 / 22

  26. Random Testing on Code Example Necessary inputs x > 0, y = 10 Assume 32-bit integers ⇒ 1 out of 2 32 Marco Probst Concolic Testing 9 / 22

  27. Random Testing on Code Example Necessary inputs x > 0, y = 10 Assume 32-bit integers ⇒ 1 out of 2 32 Very low probability Marco Probst Concolic Testing 9 / 22

  28. Random Testing on Code Example Necessary inputs x > 0, y = 10 Assume 32-bit integers ⇒ 1 out of 2 32 Very low probability Long run . . . � Marco Probst Concolic Testing 9 / 22

  29. Random Testing on Code Example Necessary inputs x > 0, y = 10 Assume 32-bit integers ⇒ 1 out of 2 32 Very low probability Long run . . . � Another technique! Marco Probst Concolic Testing 9 / 22

  30. Symbolic Execution [2] & [3] Symbols instead of concrete values Marco Probst Concolic Testing 10 / 22

  31. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Marco Probst Concolic Testing 10 / 22

  32. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Marco Probst Concolic Testing 10 / 22

  33. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values 1 y = read(); 2 y = 2 * y; 3 4 if (y == 12) { fail(); 5 6 } 7 8 complete(); Marco Probst Concolic Testing 10 / 22

  34. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; 3 4 if (y == 12) { fail(); 5 6 } 7 8 complete(); Marco Probst Concolic Testing 10 / 22

  35. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; y = 2 * y ⇒ y = 2 * s 3 4 if (y == 12) { fail(); 5 6 } 7 8 complete(); Marco Probst Concolic Testing 10 / 22

  36. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; y = 2 * y ⇒ y = 2 * s 3 4 if (y == 12) { Branching point in line 4 fail(); 5 y == 12 ⇒ 2 * s == 12 6 } y != 12 ⇒ 2 * s != 12 7 8 complete(); Marco Probst Concolic Testing 10 / 22

  37. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; y = 2 * y ⇒ y = 2 * s 3 4 if (y == 12) { Branching point in line 4 fail(); 5 y == 12 ⇒ 2 * s == 12 6 } y != 12 ⇒ 2 * s != 12 7 8 complete(); Marco Probst Concolic Testing 10 / 22

  38. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; y = 2 * y ⇒ y = 2 * s 3 4 if (y == 12) { Branching point in line 4 fail(); 5 y == 12 ⇒ 2 * s == 12 6 } y != 12 ⇒ 2 * s != 12 7 8 complete(); Marco Probst Concolic Testing 10 / 22

  39. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; y = 2 * y ⇒ y = 2 * s 3 4 if (y == 12) { Branching point in line 4 fail(); 5 y == 12 ⇒ 2 * s == 12 6 } y != 12 ⇒ 2 * s != 12 7 8 complete(); Which input leads to fail() ? Marco Probst Concolic Testing 10 / 22

  40. Symbolic Execution [2] & [3] Symbols instead of concrete values Connected to path constraints (or path conditions) Constraint solver computes concrete values Introduces symbol s for read() y = read() ⇒ y = s 1 y = read(); 2 y = 2 * y; y = 2 * y ⇒ y = 2 * s 3 4 if (y == 12) { Branching point in line 4 fail(); 5 y == 12 ⇒ 2 * s == 12 6 } y != 12 ⇒ 2 * s != 12 7 8 complete(); Which input leads to fail() ? Constraint solver yields 6 Marco Probst Concolic Testing 10 / 22

Recommend

Efficient Concolic Testing of MPI Applications Hongbo Li Zizhong Chen Rajiv Gupta CC19,

Efficient Concolic Testing of MPI Applications Hongbo Li Zizhong Chen Rajiv Gupta CC19,

Efficient Concolic Testing of MPI Applications Hongbo Li Zizhong Chen Rajiv Gupta CC19, Washington DC, USA Feb. 17, 2019 Concolic Testing Symbolic Execution Concrete Execution It Is Popular Programming languages: Binary machine

769 views • 37 slides
LCT: An Open Source Concolic Testing Tool for Java Programs Kari Khknen, Tuomas Launiainen,

LCT: An Open Source Concolic Testing Tool for Java Programs Kari Khknen, Tuomas Launiainen,

LCT: An Open Source Concolic Testing Tool for Java Programs Kari Khknen, Tuomas Launiainen, Olli Saarikivi, Janne Kauttio, Keijo Heljanko and Ilkka Niemel BYTECODE 2011 Overview Concolic Testing Tool Overview Experiments

395 views • 13 slides
CUTE: A Concolic Unit Testing Engine for C (ACM SIGSOFT Impact Award 2019) Koushik Sen, UC

CUTE: A Concolic Unit Testing Engine for C (ACM SIGSOFT Impact Award 2019) Koushik Sen, UC

CUTE: A Concolic Unit Testing Engine for C (ACM SIGSOFT Impact Award 2019) Koushik Sen, UC Berkeley Darko Marinov, UIUC Gul Agha, UIUC Programs Have Bugs 2 Why Program Testing? Programmer familiarity Concrete input for debugging

1.16k views • 68 slides
CAB-Fuzz: Practical Concolic Testing Techniques for COTS Operating Systems Su Yong Kim, Sangho

CAB-Fuzz: Practical Concolic Testing Techniques for COTS Operating Systems Su Yong Kim, Sangho

CAB-Fuzz: Practical Concolic Testing Techniques for COTS Operating Systems Su Yong Kim, Sangho Lee, Insu Yun, Wen Xu, Byoungyoung Lee, Youngtae Yun, Taesoo Kim USENIX Annual Technical Conference July 14, 2017 The Affiliated Institute of ETRI

812 views • 46 slides
MACE: Model-inference-Assisted Concolic Exploration Domagoj Babic

MACE: Model-inference-Assisted Concolic Exploration Domagoj Babic

MACE: Model-inference-Assisted Concolic Exploration Domagoj Babic h-p://www.domagoj.info/ joint work with Chia Yuan Cho, Pongsin Poosankam, Kevin Zhijie Chen, Edward XueJun

508 views • 21 slides
QSYM : A PRACTICAL CONCOLIC EXECUTION ENGINE TAILORED FOR HYBRID FUZZING Insu Yun, Sangho Lee,

QSYM : A PRACTICAL CONCOLIC EXECUTION ENGINE TAILORED FOR HYBRID FUZZING Insu Yun, Sangho Lee,

QSYM : A PRACTICAL CONCOLIC EXECUTION ENGINE TAILORED FOR HYBRID FUZZING Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang and Taesoo Kim, FINDING SECURITY BUGS Fuzzing Automated test to monitor exceptions (crashes &amp; memory leaks)

417 views • 17 slides
Testing Terminology System testing Types of errors Function testing Structure

Testing Terminology System testing Types of errors Function testing Structure

Outline Testing Terminology System testing Types of errors Function testing Structure Testing Dealing with errors Performance testing Quality assurance vs Acceptance testing Testing Installation testing

396 views • 11 slides
Chapter 1 Fundamentals of testing 1. Why is testing necessary? 2. What is testing? 3. Test

Chapter 1 Fundamentals of testing 1. Why is testing necessary? 2. What is testing? 3. Test

Chapter 1 Fundamentals of testing 1. Why is testing necessary? 2. What is testing? 3. Test principles 4. Fundamental test process 5. The psychology of testing 1. Why is testing necessary 1.1 Software system context 1.2 Causes of

568 views • 36 slides
Levels of Testing Chapter 12 Beyond unit testing Developer Testing stages Unit testing

Levels of Testing Chapter 12 Beyond unit testing Developer Testing stages Unit testing

Levels of Testing Chapter 12 Beyond unit testing Developer Testing stages Unit testing Testing of individual components Integration testing Testing to expose problems arising from the combination of components System

173 views • 15 slides
Software Testing Software testing 1 V model Software testing 2 Program testing goals To

Software Testing Software testing 1 V model Software testing 2 Program testing goals To

Software Testing Software testing 1 V model Software testing 2 Program testing goals To demonstrate to the developer and the customer that the software meets its requirements. =&gt; leads to validation testing To discover situations

264 views • 11 slides
Lecture 8 Purpose of testing Kinds of testing Testing Heuristics for good test

Lecture 8 Purpose of testing Kinds of testing Testing Heuristics for good test

CSE 331 Outline Software Design and Implementation Why correct software matters Motivates testing and more than testing, but now seems like a fine time for the discussion Testing principles and strategies Lecture 8 Purpose

506 views • 13 slides
Testing Overview Introduction (Proving and Testing) Types of Testing Unit,

Testing Overview Introduction (Proving and Testing) Types of Testing Unit,

CPSC 310 Software Engineering Testing Overview Introduction (Proving and Testing) Types of Testing Unit, Integration, Regression, System, Acceptance Testing Tactics Functional (Black Box), Structural (White Box)

982 views • 61 slides
Testing 5 March 2020 OSU CSE 1 Importance of Testing Testing is a ubiquitous and expensive

Testing 5 March 2020 OSU CSE 1 Importance of Testing Testing is a ubiquitous and expensive

Testing 5 March 2020 OSU CSE 1 Importance of Testing Testing is a ubiquitous and expensive software engineering activity It is not unusual to spend 30-40% of total project effort on testing For big and/or life-critical systems

701 views • 47 slides
Overview Objective Types of testing ECE 553: TESTING AND Verification testing

Overview Objective Types of testing ECE 553: TESTING AND Verification testing

9/1/2014 Overview Objective Types of testing ECE 553: TESTING AND Verification testing TESTABLE DESIGN OF Characterization testing Manufacturing testing DIGITAL SYSTES DIGITAL SYSTES Acceptance testing

368 views • 7 slides
Chapter 11, Testing ! Function testing Types of errors ! Structure Testing Dealing with

Chapter 11, Testing ! Function testing Types of errors ! Structure Testing Dealing with

Outline Object-Oriented Software Engineering Terminology System testing Chapter 11, Testing ! Function testing Types of errors ! Structure Testing Dealing with errors ! Performance testing Using UML, Patterns, and Java Quality

425 views • 7 slides
Functional Testing Review Chapter 8 Functional Testing We saw three types of functional

Functional Testing Review Chapter 8 Functional Testing We saw three types of functional

Functional Testing Review Chapter 8 Functional Testing We saw three types of functional testing Boundary Value Testing Equivalence Class Testing Decision Table-Based Testing What is the common thread among the above methods?

685 views • 33 slides
Page 1 Unit Testing Informal -- Incremental coding Static Analysis: Hand execution:

Page 1 Unit Testing Informal -- Incremental coding Static Analysis: Hand execution:

Podcast Ch11-02 Title : Types of Testing Description : Unit Testing, Black Box Testing, White Box Testing, Using Flow Diagrams Participants : Barry Kurtz (instructor); Brandon Winters, Sara Hyde, Cheng Vue, Dan Baehr (students)

792 views • 8 slides
CSE507 Computer-Aided Reasoning for Software Model Checking I

CSE507 Computer-Aided Reasoning for Software Model Checking I

CSE507 Computer-Aided Reasoning for Software Model Checking I courses.cs.washington.edu/courses/cse507/14au/ Emina Torlak emina@cs.washington.edu Today 2 Today Last lecture Symbolic execution and concolic testing 2 Today Last lecture

776 views • 62 slides
Testing 6.1 Specification testing Michel Bierlaire A short reminder on hypothesis testing

Testing 6.1 Specification testing Michel Bierlaire A short reminder on hypothesis testing

Testing 6.1 Specification testing Michel Bierlaire A short reminder on hypothesis testing Hypothesis testing is a method to contradict a theoretical assumption using data. In his seminal book on design of experiments, Fisher (1937) uses

526 views • 3 slides
Property-Based Testing Matt Bachmann @mattbachmann Testing is Important Testing is Important

Property-Based Testing Matt Bachmann @mattbachmann Testing is Important Testing is Important

Property-Based Testing Matt Bachmann @mattbachmann Testing is Important Testing is Important Testing is Important Testing is Hard Testing is Hard Testing is Hard Capture the Important Cases Minimize The Coding Overhead Sorting a list of

1.66k views • 118 slides
Testing a URL Class http://www.askigor.org/status.php?id=sample Query Protocol Host Path 31

Testing a URL Class http://www.askigor.org/status.php?id=sample Query Protocol Host Path 31

Making Programs Fail Andreas Zeller 1 Two Views of Testing Testing means to execute a program with the intent to make it fail. Testing for validation: Finding unknown failures (classical view) Testing for debugging: Finding a

459 views • 17 slides
Structural Testing Also known as glass/white/open box testing Structural testing is based

Structural Testing Also known as glass/white/open box testing Structural testing is based

Path Testing and Test Coverage Chapter 9 Structural Testing Also known as glass/white/open box testing Structural testing is based on using specific knowledge of the program source text to define test cases Contrast with functional

999 views • 60 slides
Black Box Testing (A&amp;O Ch. 4) (2 nd Ed. Ch. 6) Course Software Testing &amp; Verification

Black Box Testing (A&amp;O Ch. 4) (2 nd Ed. Ch. 6) Course Software Testing &amp; Verification

Black Box Testing (A&amp;O Ch. 4) (2 nd Ed. Ch. 6) Course Software Testing &amp; Verification 2019/20 Wishnu Prasetya &amp; Gabriele Keller Plan Partitioning based testing (A&amp;O Ch 4/2 nd Ed Ch. 6). Model based testing Note: black

772 views • 39 slides
Web testing Image by C Watts What is web testing? Testing web applications Applications of which

Web testing Image by C Watts What is web testing? Testing web applications Applications of which

Web testing Image by C Watts What is web testing? Testing web applications Applications of which the client runs in a web browser In this lecture on web testing What to test How to test it What to test Back end Front end HTTP (server)

356 views • 31 slides

More recommend