Complete Shadow Symbolic Execution with Java PathFinder Hoang Lam Yannic Noller Minxing Tang Timo Kehrer Lars Grunske Nguyen Humboldt-Universität zu Berlin yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 1
Problem Background Solution Evaluation Future Work Summary Regression Testing 1 int foo ( int x) { 2 int y; 3 if (x < 0) { 4 y = -x; 5 } else { 6 y = 2 * x; 7 } assertion error for x=-1 8 if (y > 1) { 9 return 0; 10 } else { 11 if (y == 1) 12 assert( false ); 13 } 14 return 1; 15 } yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 2
Problem Background Solution Evaluation Future Work Summary Regression Testing 1 int foo ( int x) { 2 int y; 3 if (x < 0) { assertion error 4- y = -x; for x=-1 is fixed 4+ y = x * x; (returns 0) 5 } else { 6 y = 2 * x; 7 } introduced new 8+ y = y + 1; assertion error 9 if (y > 1) { for x=0 10 return 0; (previously returned 1) 11 } else { 12 if (y == 1) → Regression Bug 13 assert( false ); 14 } 15 return 1; 16 } yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 3
Problem Background Solution Evaluation Future Work Summary Symbolic Execution [Clarke1976, King1976] (a short recap) path condition [ TRUE ] x= 𝕐 1 int foo ( int x) { 2 int y; 3 if (x < 0) { [ TRUE ] 𝕐 < 0 ? 4 y = -x; true false 5 } else { [ 𝕐 <0] y := - 𝕐 [ 𝕐 ≥ 0] y := 2* 𝕐 6 y = 2 * x; 7 } 8 if (y > 1) { [ 𝕐 ≥ 0] 2* 𝕐 > 1 ? [ 𝕐 <0] - 𝕐 > 1 ? 9 return 0; true false true false … 10 } else { [ 𝕐 >0] [ 𝕐 <0 ⋀ - 𝕐 ≤ 1] [ 𝕐 <0 ⋀ - 𝕐 >1] 11 if (y == 1) [ 𝕐 <-1] [ 𝕐 =0] return 0; - 𝕐 = 1 ? return 0; 12 assert( false ); return 1; true false 13 } [ 𝕐 <0 ⋀ - 𝕐 ≤ 1 ⋀ - 𝕐 =1] [ 𝕐 <0 ⋀ - 𝕐 ≤ 1 ⋀ - 𝕐 ≠ 1] 14 return 1; [ 𝕐 =-1] assert( false ); return 1; 15 } UNSAT yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 4
Problem Background Solution Evaluation Future Work Summary Shadow Symbolic Execution (Palikareva, Kuchta, and Cadar; ICSE 2016) Goal: generate test cases to expose diverging behavior of two software versions yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 5
Problem Background Solution Evaluation Future Work Summary [Palikareva2016] 1 int foo ( int x) { 1 int foo ( int x) { 2 int y; 2 int y; 3 if (x < 0) { 3 if (x < 0) { 4- y = -x; 4 y = change(-x, x*x); 4+ y = x * x; 5 } else { 5 } else { 6 y = 2 * x; 6 y = 2 * x; 7 } 7 } 8 y = change(y, y + 1); 8+ y = y + 1; 9 if (y > 1) { 9 if (y > 1) { 10 return 0; 10 return 0; 11 } else { 11 } else { 12 if (y == 1) 12 if (y == 1) 13 assert( false ); 13 assert( false ); 14 } 14 } 15 return 1; 15 return 1; 16 } 16 } yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 6
Problem Background Solution Evaluation Future Work Summary [Palikareva2016] 2 1 Bounded Concolic Symbolic Execution Execution Two-way Forking (BSE) Four-way Forking [ TRUE ] 𝛃 ? true false [ TRUE ] change( 𝛃 , 𝛄 ) ? [ 𝛃 ] … [¬ 𝛃 ] … old: true old: false old: false old: true new: true new: false new: true new: false [ 𝛃∧𝛄 ] … [¬ 𝛃∧ ¬ 𝛄 ] … [ 𝛃∧ ¬ 𝛄 ] … [¬ 𝛃∧𝛄 ] … same TRUE same FALSE diff TRUE diff FALSE yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 7
���������������������������� ������������������������������������������ Problem Background Solution Evaluation Future Work Summary [Palikareva2016] ACM SIGSOFT Software Engineering Notes Page 1 October 2017 Volume 42 Number 4 Shadow Symbolic Execution with Java PathFinder Yannic Noller Hoang Lam Nguyen Humboldt University of Berlin Humboldt University of Berlin yannic.noller@informatik.hu-berlin.de nguyenhx@informatik.hu-berlin.de Minxing Tang Timo Kehrer Humboldt University of Berlin Humboldt University of Berlin tangminx@informatik.hu-berlin.de timo.kehrer@informatik.hu-berlin.de ABSTRACT bolic execution-based technique, which they refer to as shadow symbolic execution . Their technique is designed to generate test Regression testing ensures that a software system when it evolves still performs correctly and that the changes introduce no unin- inputs that cover new program behaviors introduced by a patch. Shadow symbolic execution works by executing both the old (bug- tended side-e ff ects. However, the creation of regression test cases gy) and new (patched) version in the same symbolic execution that show divergent behavior needs a lot of e ff ort. A solution is the idea of shadow symbolic execution , originally implemented instance, with the old version shadowing the new one. Therefore, it is necessary to manually merge both programs into a change- based on KLEE for programs written in C, which takes a unified annotated, unified version. Based on such a unified version, the version of the old and the new program and performs symbolic execution guided by concrete values to explore the changed behav- technique detects divergences along the execution path of an in- put that exercises the patch. Their tool Shadow , which we refer ior. In this work, we apply the idea of shadow symbolic execution to as Shadow KLEE , is implemented on top of the KLEE symbolic to Java programs and, hence, provide an extension of the Java (Noller et al.; JPF 2017) yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 8
Problem Background Solution Evaluation Future Work Summary Limitations (1) Deeper divergences might be missed in the BSE phase due to narrow path conditions based on concrete inputs. 1 int foo ( int x) { x=-1 (fully covers the changes) 2 int y; 3 if (x < 0) { 4 y = change(-x, x*x); path condition up to line 9: 5 } else { [ 𝕐 < 0 ] 6 y = 2 * x; 7 } 8 y = change(y, y + 1); to reach assertion error BSE 9 if (y > 1) { 10 return 0; needs to follow false branch 11 } else { with condition: [ 𝕐 2 + 1 ≤ 1 ] 12 if (y == 1) 13 assert( false ); 14 } only possible for x=0 , but [ 𝕐 < 0 ] 15 return 1; 16 } yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 9
Problem Background Solution Evaluation Future Work Summary Limitations (2) The initial input has to cover not only changed locations, but also potential divergence points. divergence only possible in line 4 1 int bar ( int x, int y) { 2 int z = change(x, y); 3 if ((x+y) == 5) { collect change and then reach 4 if (z == -100) divergence (point) 5 assert( false ); 6 } 7 return 0; all inputs with x+y ≠ 5 would 8 } miss the divergence yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 10
Problem Background Solution Evaluation Future Work Summary Shadow Symbolic Execution strongly depends on concrete inputs Complete Shadow Symbolic Execution 1 combines bounded symbolic execution with four-way forking full exploration of same TRUE/FALSE paths, as long as they can or have 2 reached a change 3 exploration of di ff TRUE/FALSE paths only for the new version yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 11
Problem Background Solution Evaluation Future Work Summary 1 int foo ( int x) { 2 int y; 3 if (x < 0) { 4 y = change(-x, x*x); 5 } else { 6 y = 2 * x; 7 } 8 y = change(y, y + 1); 9 if (y > 1) { 10 return 0; 11 } else { 12 if (y == 1) 13 assert( false ); 14 } 15 return 1; 16 } yannic.noller@hu-berlin.de Java Pathfinder Workshop 2019 12
Recommend
More recommend
