Symbolic Execution and Fuzz Testing ISSISP Summer School 2018 Prof. Abhik Roychoudhury National University of Singapore 1
Thanks to organizers and ISSISP • Steve Blackburn • Adrian Herrera ISSISP Summer School 2018 • Tony Hosking • Shane McGrath and all organizers of the event. 2
Ack. to former students and grant Marcel. Boehme, PhD. NUS 2014, Post-doc NUS -> Lecturer Monash Van Thuan Pham, PhD. 2017 Sergey Mechtaev, PhD. 2018 -> Lecturer University College London Shin Hwei Tan, PhD. 2018 -> Asst Prof, SUSTech, Shenzen. China ISSISP Summer School 2018 Jooyong Yi, past post-doc -> Asst Prof. Innopolis ACKNOWLEDGEMENT: National Cyber Security Research program from NRF Singapore http://www.comp.nus.edu.sg/~tsunami/ and DSO National Labs 3
COTS-integrated Platforms Trustworthy System Flaws Malicious Vulnerability Behavior ISSISP Summer School 2018 Outsourced and Shared Data Data Breach Binary analysis of paramount need for software acquisition or assembly. 4 http://www.comp.nus.edu.sg/~tsunami
Enhancing local capabilities Education – NUS (New degree program) Vulnerability Binary Discovery Hardening Industry Agency Collaboration Collaboration – ST, Symantec, ISSISP Summer School 2018 DSTA, … NEC, … Data Verification Protection Research Outputs – Publications, Tools, Academic Collaboration, 5 Exchanges, Seminars, Workshops
Short Videos • https://youtu.be/C1hl_ujw6B0 Plan • (1 Minute) • History of Symbolic execution • https://youtu.be/EHBjMSQvIpg Symbolic Execution and Program Testing • (1 Minute) • Use in fuzz testing ISSISP Summer School 2018 • Lead up to specification inference • How the ideas of symbolic execution can be transported to automated program repair 6
In this(?) talk … Search Symbolic Execution • Enhance the effectiveness of search • Explore capabilities of symbolic techniques, with symbolic execution execution beyond search as inspiration • Automated Program Repair • Systematic Fuzz Testing ISSISP Summer School 2018 7
“ Program testing and program proving can be considered as extreme alternatives. …. This paper describes a practical approach between these two extremes … ISSISP Summer School 2018 Each symbolic execution result may be equivalent to a large number of normal tests ” 8
Testing BLACK-BOX Requirements ISSISP Summer School 2018 9
Testing Require ments ISSISP Summer School 2018 WHITE-BOX 10
Proving via SW Model Checking ISSISP Summer School 2018 11
Proving: SW Model Checking ISSISP Summer School 2018 12
Blurring the lines: Symbolic Exec. SEARCH(A, 1, 5, X, found, j) SEARCH( A, L, U, X, found, j){ int j, found = 0; while (L <= U && found == 0){ X == A[3] found == 1 j == 3 j = (L+U)/2; if (X == A[j]){ found = 1;} X == A[1] && X < A[3] found == 1 j == 1 else if (X < A[j]){ U = j -1; } X < A[1] && X <A[3] found == 0 j == 0 else{ L = j +1; } ISSISP Summer School 2018 } X = A[2] && X > A[1] && X <A[3] found == 1 j == 2 if (found == 0){ j = L – 1;} …. } Testing ? Comprehension?? Verification ??? 13
Blurring the lines: Symbolic Exec. SEARCH( A, L, U, X, found, j){ SEARCH(A, 1, 5, 20, found, j) int j, found = 0; while (L <= U && found == 0){ SEARCH(A, 1, 5, X, found, j) j = (L+U)/2; if (X == A[j]){ found = 1;} else if (X < A[j]){ U = j -1; } SEARCH(A, N, N+4, X, found, j) else{ L = j +1; } ISSISP Summer School 2018 } if (found == 0){ j = L – 1;} SEARCH(A, 1, M, X, found, j) } Testing ? Comprehension?? Verification ??? 14
Primer on SE ISSISP Summer School 2018 Abhik Roychoudhury National University of Singapore 15
Concrete execution Concrete input Concrete input in == 1 in == 1 Program Program out = in + 1 out = in * 2 P Q ISSISP Summer School 2018 Concrete Concrete output output out == 2 out == 2 No observable difference! 16
Execution with symbolic inputs Symbolic input Symbolic input in == q in == q Program Program out = in + 1 out = in * 2 P Q ISSISP Summer School 2018 Concrete output Concrete output out == 2* q out == q + 1 To expose difference, try to find q such that q + 1 ¹ 2 * q 17
Path exploration based symbolic execution in == q input in; in >= 0 input in; No Yes Keep both if (in >= 0) a = in; else a = -1; a = in; a = -1; ISSISP Summer School 2018 return a; q <0 Þ q ≥ 0 Þ return a out == -1 out == q 18
On-the-fly path exploration Instead of analyzing the whole program, shift from one program path to another. in == 0 in == 5 input in; z = 0; x = 0; Sample exploration: Continue the if (in > 0){ search for failing inputs. Try ISSISP Summer School 2018 z = in *2; those which do not go through the x = in +2; “same” path. x = x + 2; } How to perform symbolic else … execution along a single if ( z > x){ path? return error; } X Ö 19
Exploring one path in==0 input in; in >= 0 Useful to find: No Yes “the set of all inputs which trace a given a = -1; a = in; path” ISSISP Summer School 2018 Path condition return a; in ≥ 0 20
Path condition computation in == 5 Line# Assignment store Path condition 1 input in; 1 {} true 2 z = 0; x = 0; 2 {(z,0),(x,0)} true 3 if (in > 0){ 4 z = in *2; 3 {(z,0),(x,0)} in > 0 5 x = in +2; 4 {(z,2*in), (x,0)} in > 0 6 x = x + 2; 7 } 5 {(z,2*in), (x,in+2)} in > 0 ISSISP Summer School 2018 8 else … 6 {(z,2*in), (x, in+4)} in > 0 9 if ( z > x){ 7 {(z, 2*in), (x, in+4)} in > 0 return error; } 9 {(z, 2*in), (x, in+4)} in>0 Ù (2*in > in +4) 21
Directed testing • Start with a random input I. • Execute program P with I Suppose I executes path p in program P. While executing p, collect a symbolic formula f which captures the set of all inputs which execute path p in program P. f is the path condition of path p traced by input i. • Minimally change f, to produce a formula f1 ISSISP Summer School 2018 Solve f1 to get a new input I1 which executes a path p1 different from path p. 22
Concrete Symbolic Execution Execution concrete state symbolic state constraints main(){ int t1 = randomInt(); int t2 = randomInt(); t1=0, t2=457 t1=m, t2=n test_me(t1,t2); } int add100(int x){ return x + 100;} int test_me(int Climb, int Up){ int sep, upward; if (Climb > 0){ ISSISP Summer School 2018 sep = Up;} else {sep = add100(Up);} if (sep > 150){ upward = 1; } else {upward = 0;} if (upward < 0){ abort; } else return upward; 23 }
Concrete Symbolic Execution Execution concrete state symbolic state constraints main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int add100(int x){ return x + 100;} int test_me(int Climb, int Up){ Climb=0, Up=457 Climb=m, Up=n int sep, upward; if (Climb > 0){ ISSISP Summer School 2018 sep = Up;} else {sep = add100(Up);} if (sep > 150){ upward = 1; } else {upward = 0;} if (upward < 0){ abort; } else return upward; 24 }
Concrete Symbolic Execution Execution concrete state symbolic state constraints main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int add100(int x){ return x + 100;} int test_me(int Climb, int Up){ int sep, upward; if (Climb > 0){ ISSISP Summer School 2018 sep = Up;} Climb=0, Up=457, sep= 457 Climb=m, Up=n sep= n m ≤ 0 else {sep = add100(Up);} if (sep > 150){ upward = 1; } else {upward = 0;} if (upward < 0){ abort; } else return upward; 25 }
Concrete Symbolic Execution Execution concrete state symbolic state constraints main(){ int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int add100(int x){ return x + 100;} int test_me(int Climb, int Up){ int sep, upward; if (Climb){ ISSISP Summer School 2018 sep = Up;} else {sep = add100(Up);} Climb=0, Up=457 sep= 557 Climb=m, Up=n sep= if (sep > 150){ m ≤ 0 && n > 50 n+100 upward = 1; } else {upward = 0;} if (upward < 0){ abort; } else return upward; 26 }
Concrete Symbolic Execution Execution main(){ concrete state symbolic state constraints int t1 = randomInt(); int t2 = randomInt(); test_me(t1,t2); } int add100(int x){ return x + 100;} Solve int test_me(int Climb, int Up){ m ≤ 0 && n ≤ 50 int sep, upward; if (Climb){ m == 0, n == 50 sep = Up;} ISSISP Summer School 2018 else {sep = add100(Up);} if (sep > 150){ upward = 1; } else {upward = 0;} Climb=0, Up=457, sep= 557 Climb=m, Up=n, sep= m ≤ 0 && n > 50 if (upward < 0){ n+100, upward =1 abort; } else return upward; } 27 Ack: Koushik Sen (Berkeley)
Concrete Symbolic Execution Execution concrete state symbolic state constraints main(){ int t1 = randomInt(); int t2 = randomInt(); t1=0, t2=50 t1=m, t2=n test_me(t1,t2); } int add100(int x){ return x + 100;} int test_me(int Climb, int Up){ int sep, upward; if (Climb > 0){ ISSISP Summer School 2018 sep = Up;} else {sep = add100(Up);} if (sep > 150){ upward = 1; } else {upward = 0;} if (upward < 0){ abort; } else return upward; 28 }
Recommend
More recommend
