Fuzzing the Solidity Compiler Bhargava Shastry @ibags Ethereum - PowerPoint PPT Presentation

Fuzzing the Solidity Compiler Bhargava Shastry @ibags Ethereum Foundation bshastry Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

2 whoami ● Security engineer, Solidity team ● Semantic testing of Solidity compiler Find security-critical bugs in the compiler before it is shipped Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

3 tl;dr: ● Threat model: Incorrect code generation ● Randomly generated valid Solidity (yul) programs test compiler ● Found 10 bugs using semantic fuzzing ● Continuous fuzzing for early bug discovery Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

4 Introduction Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

5 Threat model ● Compiler user (programmer) is not malicious ● Bugs introduced by the optimizer Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

6 Fuzz testing in a nutshell while not ctrl + c do input=gen_input() runProgram(input) done Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

7 Limitation of random fuzzing contract C { contract C { function foo() fu#!3ion foo() public { puX^&c { do_something(); do_something(); Mutation } } } } Accepted by parser Rejected by parser Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

8 Fuzzing a compiler requires generating valid programs... … generating a valid program requires structure awareness Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

9 Approach Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

10 Write a specification Specification written in protobuf language message Block { repeated Statement stmts; } ... message program { repeated Block blocks; } Full spec: https://github.com/ethereum/solidity/blob/develop/test/tools/ossfuzz/yulProto.proto Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

11 Input generation ● Input generated and mutated by libprotobuf-mutator ● Each input is a tree blocks { stmts { ifstmt { condition { binaryOp { eq { op1: varref{id: 0} op2: 0} } } } } } Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

12 Input conversion ● Converter is source-to-source translator ● Input: protobuf serialization format ● Output: yul program Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

13 Example blocks { stmts { ifstmt { condition { binaryOp { eq { op1: varref{id: 0} op2: 0} } } } } } Conversion if x_0 == 0 Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

14 Test program generation { Libprotobuf function f() Protobuf Message func { { + Block b = 1; ... Converter } mutator } } Protobuf Test program specification Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

15 Correctness testing requires encoding expectation somehow Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

16 Differential fuzzing ● Track side-effects of execution ● Run program ● Run optimized program ● Compare side-effects Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

17 Yul interpreter ● Interprets arbitrary yul program ● Outputs side-effects as a trace (string) Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

18 Yul interpreter { function f() MLOAD { MSTORE Interpreter ... … } DATACOPY } Test program Execution trace Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

19 Fuzzing Setup Program MLOAD MSTORE generator … DATACOPY == Interpreter MLOAD MSTORE Optimizer … DATACOPY Trace Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

20 Custom Fuzz Mutator Program Custom Generator Mutator if x_0 == 0 if x_0 != 0 Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

21 Results Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

22 Bugs by component 3 7 Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

23 Bugs by impact 5 5 Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

24 Bugs by severity Found via custom 1 mutation 2 6 1 Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

25 Current work Antlr based custom mutator { { function f() function g() Antlr Solidity { { ... g() unparser mutator } } } } Test program Mutation Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

26 Conclusion Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

27 Conclusion ● Continuous structure-aware fuzzing for early bug discovery ● Useful for testing optimizer and data en/decoding ● Decent assurance ○ Evidence that it works ○ No formal guarantees though Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler

28 Thank you! ethereum/solidity.git gitter.im/ethereum/solidity-dev Fuzzcon Europe 2020 | Fuzzing the Solidity Compiler