Learning to Fuzz from Symbolic Execution with Application to Smart - PowerPoint PPT Presentation

Learning to Fuzz from Symbolic Execution with Application to Smart Contracts Jingxuan Mislav Nodar Petar Martin He Balunovic Ambroladze Tsankov Vechev

Random Fuzzing vs. Symbolic Execution Random Fuzzing Symbolic Execution Fast Slow Speed Inputs Ineffective Effective Coverage Low Low 1

Smart Contract Testing: Challenge Initial , deposit(), P ether . . . . . . , setOwner( ) . . . . . . , withdraw(P) . . . . . . steals P ether from 2

Smart Contract Testing: Challenge Wanted: Transaction sequences that thoroughly explore the state space 3

4

Random Fuzzing vs. Symbolic Execution Imitation Learning based Fuzzer ~120K contracts ~16K clusters - Analyzing Ethereum’s Contract Topology. Kiffer et al.. IMC ’18 Random Fuzzing Symbolic Execution ILF (this work) Fast Slow Fast Speed Inputs Effective Effective Ineffective Coverage Low High Low 5

Imitation Learning Demonstration Robot Human expert Demonstration Fuzzer Symbolic execution 6

Learning to Fuzz from Symbolic Execution Symbolic execution expert ≈ 15K contracts Smart contracts Transaction sequences Training Fuzzing Coverage Vulnerability New contract Fuzzing policy Report (neural networks) 7

Learning to Fuzz from Symbolic Execution Symbolic execution expert Smart contracts Transaction sequences Training Fuzzing Coverage Vulnerability New contract Fuzzing policy Report (neural networks) 8

̅ ̅ Smart Contract Fuzzing Policy + + + = may modify blockchain state ! = ($ & , ()*+),, -./0*!) ()*+), -./0*! $ & Example: a Uniformly Random Policy Transaction : 2*3$/,.(4) : 2*3$/,.(536*-!0,)($)) Fuzzing Tested Policy Contract : 2*3$/,.(5JKLJM5) Feedback 72*3$/,. 0, 9: $ is payable : C(0) = 1 otherwise 9

̅ Neural Network Fuzzing Policy Features of 4 GRU fuzz FCN func $ Q at step 3 − 1 hidden ℎ QRS state GRU int + & Q FCN int GRU fuzz Feature hidden state at step 3 of $ ℎ Q QRS FCN sender ()*+), Q ℎ Q GRU fuzz -./0*! Q FCN amount at step 3 + 1 10

Neural Network Fuzzing Policy – Fuzzing State GRU fuzz at step 3 − 1 e.g., Coverage, opcodes, Last function name. (can be dynamic) hidden [3.5, 0.3, 4.0, …] state Current hidden state Feature of $ QRS GRU fuzz [1, 6.2, 5, …] [1.2, 8.7, 2.5, …] at step 3 11

̅ Neural Network Fuzzing Policy Features of 4 GRU fuzz FCN func $ Q at step 3 − 1 hidden ℎ QRS state GRU int + & Q FCN int GRU fuzz Feature hidden state at step 3 of $ ℎ Q QRS FCN sender ()*+), Q ℎ Q GRU fuzz -./0*! Q FCN amount at step 3 + 1 12

Neural Network Fuzzing Policy – Function [[1, 6.2, 5, …], [4, 3.7, 6, …], Feature of 4 … SetOwner FCN func [2, 9.2, 7, …]] + Withdraw Deposit Softmax Withdraw Current hidden state [1.2, 8.7, 2.5, …] 13

̅ Neural Network Fuzzing Policy Features of 4 GRU fuzz FCN func $ Q at step 3 − 1 hidden ℎ QRS state GRU int + & Q FCN int GRU fuzz Feature hidden state at step 3 of $ ℎ Q QRS FCN sender ()*+), Q ℎ Q GRU fuzz -./0*! Q FCN amount at step 3 + 1 14

Neural Network Fuzzing Policy – Arguments 1 1 0x800 0x800 0x200 1 0x10 0x10 0x200 . . . 0x200 . . . Distribution over 50 seed integer values from expert FCN int FCN int . . . Current hidden state GRU int GRU int [1.2, 8.7, 2.5, …] at step 0 at step 1 One-hot [0, 0, 1, 0, …] 15

̅ Neural Network Fuzzing Policy Features of 4 GRU fuzz FCN func $ Q at step 3 − 1 hidden ℎ QRS state GRU int + & Q FCN int GRU fuzz Feature hidden state at step 3 of $ ℎ Q QRS FCN sender ()*+), Q ℎ Q GRU fuzz -./0*! Q FCN amount at step 3 + 1 16

Learning to Fuzz from Symbolic Execution Symbolic execution expert Smart contracts Transaction sequences Training Fuzzing Coverage Vulnerability New contract Fuzzing policy Report (neural networks) 17

Symbolic Execution Expert Revisit Revisit Revisit T QUQV T S T W ! S Execute . . . e t u c e x E ! W Symbolic : VerX S&P 2020 18

Learning to Fuzz from Symbolic Execution Symbolic execution expert Smart contracts Transaction sequences Training Fuzzing Coverage Vulnerability New contract Fuzzing policy Report (neural networks) 19

̅ Training Neural Network Fuzzing Policy NN Policy at step 3 − 1 Cross-Entropy loss Back Prop. Hidden $ Q State & Q Features Inference NN Policy ! Q by expert ()*+), at step 3 Q -./0*! Q ! Q by expert 20

Learning to Fuzz from Symbolic Execution Symbolic execution expert Smart contracts Transaction sequences Training Fuzzing Coverage Vulnerability New contract Fuzzing policy Report (neural networks) 21

ILF System: Coverage & Vulnerability Detection • Instruction coverage. • Basic block coverage. • Locking : The contract cannot send out but can receive ether. • Leaking : An attacker can steal ether from the contract. • Suicidal : An attacker can deconstruct the contract. • Block Dependency : Ether transfer depends on block state variables. • Unhandled Exception : Root call does not catch exceptions from child calls. • Controlled Delegatecall : Transaction parameters explicitly flow into arguments of a delegatecall instruction. 22

Evaluation • 18,496 Contracts (5,013 Large & 13,483 Small) • 5-fold Cross Validation • UNIF • Echidna • ContractFuzzer • EXPERT • MAIAN • Coverage & Speed • Vulnerability Detection • Fuzzing Components • Case Study 23

Coverage: ILF vs. Fuzzers Instr. Coverage Instr. Coverage 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% 0% 0 200 400 600 800 1000 0 200 400 600 800 1000 Number of Transactions Number of Transactions ILF UNIF Echidna ILF UNIF Echidna Small contracts Large contracts 24

Coverage: ILF vs. Symbolic Expert Instr. Coverage 100% Small: 30 txs, 547s 90% Large: 49 txs, 2,580s 80% EXPERT ILF (#tx same as EXPERT) 70% ILF (2k txs) 60% Small: 13s 50% Large: 17s Small Contracts Large Contracts 148 txs/s 25

Vulnerability Detection ILF: 0 FPs ∪ ∪ 13 FPs % of True Vulnerabilities % of True Vulnerabilities 100% 100% 80% 80% 60% 60% 6 FPs 40% 40% 20% 20% 0% 0% Leaking Suicidal Locking Block Unhandled Controlled Dependency Exception Delegatecall ILF UNIF MAIAN ILF UNIF ContractFuzzer 26

Importance of Policy Components All components are necessary 100% 90% ILF Most ILF-func important 80% ILF-args 70% ILF-sender 60% ILF-amount 50% Coverage Leaking 27

Summary Q & A ? 28