cyberdyne
play

CYBERDYNE: Automatic bug-finding at scale Peter Goodman - PowerPoint PPT Presentation

Feb 13, 2024 •167 likes •808 views

CYBERDYNE: Automatic bug-finding at scale Peter Goodman COUNTERMEASURE 2016 Cyberdyne (ex)terminates bugs Finds bug in binaries Combines different techniques Coverage-guided fuzzing Symbolic execution Trail of Bits |

  1. CYBERDYNE: Automatic bug-finding at scale Peter Goodman COUNTERMEASURE 2016

  3. Cyberdyne (ex)terminates bugs  Finds bug in binaries  Combines different techniques  Coverage-guided fuzzing  Symbolic execution Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 2

  4. Get to know the mind of the machine  Part 1: high level architecture  How to coordinate bug-finding tools  Part 2: low level tools  How do the bug-finding tools work? Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 3

  5. History: Cyber Grand Challenge (1) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 4

  6. History: Cyber Grand Challenge (2)  Capture - the - flag (CTF) competition  Goal: find and exploit bugs in binaries  Goal: patch binaries  Competitors were programs  “Cyber Reasoning Systems” (CRS) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 5

  7. History: Cyber Grand Challenge (3)  Shaped the design of Cyberdyne  Distributed system  Runs on any number of nodes  Automated system  No human intervention required Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 6

  8. Part 1 Skeleton of a bug-finding system Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 7

  9. Ideally, a bug-finding system should …  Find bugs  Simple, right?  Work on real programs  Be easy to scale Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 8

  10. When I grow up … Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 9

  11. First kill: simple fuzzing (1) Byte flips Bit flips Splice Slice Mutation Seed Inputs Mutated Inputs Engine Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 10

  12. First kill: simple fuzzing (1) Radamsa, zzuf, etc. Mutation Seed Inputs Mutated Inputs Engine Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 11

  13. First kill: simple fuzzing (2)  Mutate inputs 12  11 1 10 2 3 9 8 4 7 5 6 Execute inputs  … Terminator Profit?  pag@sloth:~/ cyberdyne start pag@sloth:~/ cyberdyne analyze – program foo – binaries bar pag@sloth:~/ cyberdyne seed – program foo – inputs ./inputs/*  Find bugs! Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 12

  14. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  12 11 1 10 2 3 9 8 4 7 5 6 … Terminator Profit?  pag@sloth:~/ cyberdyne start pag@sloth:~/ cyberdyne analyze – program foo – binaries bar pag@sloth:~/ cyberdyne seed – program foo – inputs ./inputs/*  Find bugs! Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 13

  15. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  … Terminator Profit? 12  11 1 pag@sloth:~/ cyberdyne start 10 2 pag@sloth:~/ cyberdyne analyze – program 3 9 foo – binaries bar 8 4 pag@sloth:~/ cyberdyne seed – program foo – 7 5 6 inputs ./inputs/*  Find bugs! Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 14

  16. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  … Terminator Profit? 12  11 1 pag@sloth:~/ cyberdyne start 10 2 pag@sloth:~/ cyberdyne analyze – program 3 9 foo – binaries bar 8 4 pag@sloth:~/ cyberdyne seed – program foo – 7 5 6 inputs ./inputs/*  Find bugs!  Right???? Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 15

  17. First kill: simple fuzzing (2)  Mutate inputs   Execute inputs  … Terminator Risk of loss!  pag@sloth:~/ cyberdyne start pag@sloth:~/ cyberdyne analyze – program foo – binaries bar pag@sloth:~/ cyberdyne seed – program foo – inputs ./inputs/*  No bugs found  Lost cycles, time Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 16

  18. Misfire: Check your targets  Searching for bugs takes time  Need accountability  Is it worth it to keep searching?  Is progress being made?  How do we measure progress? Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 17

  19. Reload: Track bug-finding progress  Idea: has something new happened?  Track when new code is executed  Code coverage: Instrument program to detect when new code is executed  Inputs that cover new code signal progress Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 18

  20. Need more ammo  Eventually hit a “coverage ceiling”  Decreasing marginal returns  Need heavier guns  Coverage-guided fuzzing: re-seed with inputs that got new coverage (next)  Symbolic execution (later) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 19

  21. Coverage-guided mutational fuzzing (1) Terminator cyberdyne start cyberdyne analyze –pr… cyberdyne launch nukes Step p 3 Ste tep p 1 Ste tep p 2 Ste tep p 4 Gets new Mutate Execute Re-seed Crashes! inputs mutations Coverage? mutator Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 20

  22. Coverage-guided mutational fuzzing (1) AFL Terminator cyberdyne start cyberdyne analyze –pr… cyberdyne launch nukes Step p 3 Ste tep p 1 Ste tep p 2 Ste tep p 4 Gets new Mutate Execute Re-seed Crashes! inputs mutations Coverage? mutator Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 21

  23. Coverage-guided mutational fuzzing (2)  Trivially parallelizable  Run mutation engines concurrently  Scaling fuzzing in Cyberdyne  Fuzzer service internalizes mutation, execution, code coverage  Runs many fuzzers, one mutator each Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 22

  24. Look under the skin of Cyberdyne (1) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 23

  25. Look under the skin of Cyberdyne (2) Terminator cyberdyne start cyberdyne analyze –pr… cyberdyne launch nukes Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 24

  26. Look under the skin of Cyberdyne (3) Fuz uzze zer r (with GRR) Terminator cyberdyne start cyberdyne analyze –pr… Mutates and  cyberdyne launch nukes executes inputs Easy to scale  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 25

  27. Look under the skin of Cyberdyne (4) Py PySy SymE mEmu mu Terminator cyberdyne start cyberdyne analyze –pr… Coverage-guided  cyberdyne launch nukes binary symbolic executor Harder to scale  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 26

  28. Look under the skin of Cyberdyne (5) KLEE KL EE (with McSema) Terminator cyberdyne start cyberdyne analyze –pr… LLVM bitcode  cyberdyne launch nukes symbolic executor Hard to use  Hard to scale  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 27

  29. Look under the skin of Cyberdyne (6) Or Orac acle le Terminator cyberdyne start cyberdyne analyze –pr…  Gatekeeper cyberdyne launch nukes for minset  Detects crashes  Easy to scale Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 28

  30. Look under the skin of Cyberdyne (7) Mins Mi nset Terminator cyberdyne start cyberdyne analyze –pr… Finds inputs that get  cyberdyne launch nukes new code coverage One input at a time  Bottleneck?  Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 29

  31. Part 2 The servos and the gears Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 30

  32. How it works: Minset (1)  What is it?  Minimum set of inputs that produce maximum code coverage  Why use it?  Identify “interesting” inputs  Good candidates for exploration Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 11.17.2016 | trailofbits.com 31

  33. How it works: Minset (2) 4 3 2 1 1 4 3 2 3 1 2 4 Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 09.17.2016 | trailofbits.com 32

  34. How it works: Minset (3) 4 3 2 1 1 4 3 2 3 1 2 4 Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 09.17.2016 | trailofbits.com 33

  35. How it works: Minset (4) 4 3 2 1 1 4 3 2 3 1 2 4 𝐷𝑝𝑤(𝐽 3 ) ⊆ 𝐷𝑝𝑤(𝐽 1 ) ∪ 𝐷𝑝𝑤(𝐽 2 ) Trail of Bits | CYBERDYNE: Automatic Bug-Finding at Scale | 09.17.2016 | trailofbits.com 34

Recommend

More recommend