Finding Code That Explodes Under Symbolic Evalua<on James - PowerPoint PPT Presentation

Finding Code That Explodes Under Symbolic Evalua<on James Bornholt Emina Torlak University of Washington unsat.org

Automated reasoning tools help us solve hard programming problems

Automated reasoning tools help us solve hard programming problems Does my program s8ll 👸 Verifica8on work a:er the file system crashes? [ASPLOS’16]

Automated reasoning tools help us solve hard programming problems Does my program s8ll 👸 Verifica8on work a:er the file system crashes? [ASPLOS’16] How do I compile code 
 architecture? [PLDI’14]  Synthesis for this weird new

Automated reasoning tools help us solve hard programming problems Does my program s8ll 👸 Verifica8on work a:er the file system crashes? [ASPLOS’16] How do I compile code 
 architecture? [PLDI’14]  Synthesis for this weird new How do I teach kids 
  “Programs” the rules of algebra effec8vely? [VMCAI’18]

Symbolic evaluators Does my program s8ll How do I compile code 
 👸 architecture? [PLDI’14]  work a:er the file system for this weird new crashes? [ASPLOS’16]

Symbolic evaluators Does my program s8ll How do I compile code 
 👸 architecture? [PLDI’14]  work a:er the file system for this weird new crashes? [ASPLOS’16] Interpreter for file system Interpreter for new architecture opera8ons instruc8ons

Symbolic evaluators Does my program s8ll How do I compile code 
 👸 architecture? [PLDI’14]  work a:er the file system for this weird new crashes? [ASPLOS’16] Interpreter for file system Interpreter for new architecture opera8ons instruc8ons Symbolic evaluator Sketch, RoseWe, …

Symbolic evaluators Does my program s8ll How do I compile code 
 👸 architecture? [PLDI’14]  work a:er the file system for this weird new crashes? [ASPLOS’16] Interpreter for file system Interpreter for new architecture opera8ons instruc8ons Symbolic evaluator Sketch, RoseWe, … Angelic Verifica8on Synthesis for free! Execu8on

Symbolic evaluators: no free lunch Does my program s8ll 👸 work a:er the file system crashes? [ASPLOS’16] Interpreter for file system opera8ons Symbolic evaluator Sketch, RoseWe, … Angelic Verifica8on Synthesis for free! Execu8on

Symbolic evaluators: no free lunch Does my program s8ll 👸 How do you make work a:er the file system these tools scale? crashes? [ASPLOS’16] Interpreter for file system opera8ons Symbolic evaluator Sketch, RoseWe, … Angelic Verifica8on Synthesis for free! Execu8on

Symbolic evaluators: no free lunch Does my program s8ll 👸 How do you make work a:er the file system these tools scale? crashes? [ASPLOS’16] Searching all paths Searching all paths Interpreter for file system through the interpreter through the interpreter opera8ons Symbolic evaluator Sketch, RoseWe, … Angelic Verifica8on Synthesis for free! Execu8on

Symbolic profiling iden<fies performance issues in symbolic evalua<on

Symbolic profiling iden<fies performance issues in symbolic evalua<on Symbolic profiling Data structures and analyses

Symbolic profiling iden<fies performance issues in symbolic evalua<on Symbolic profiling Data structures and analyses 20 Symbolic evalua8on an8-paWerns 10 Common issues and source-level repairs 0 0 1500

Symbolic profiling iden<fies performance issues in symbolic evalua<on Symbolic profiling Data structures and analyses 20 Symbolic evalua8on an8-paWerns 10 Common issues and source-level repairs 0 0 1500 Empirical results 300× speedup on real-world tools

Symbolic profiling iden<fies performance issues in symbolic evalua<on if (…) { … } Symbolic evalua8on All-paths execu8on of programs ∀ x. φ(…, x) Symbolic profiling Data structures and analyses 20 Symbolic evalua8on an8-paWerns 10 Common issues and source-level repairs 0 0 1500 Empirical results 300× speedup on real-world tools

Symbolic evalua<on All-paths execu8on of programs

Symbolic evalua<on executes all paths through a program #lang rosette ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k))

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k))

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) (filter even? ‘(x 0 x 1 ))

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 )

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 )

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 )

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 ) k=0 k=0 k=1 k=0 k=1 k=0 k=2 k=1 ‘() ‘() ‘(x 1 ) ‘() ‘(x 0 ) ‘() ‘(x 0 ) ‘(x 0 x 1 )

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) take runs 2 2 8mes ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 ) k=0 k=0 k=1 k=0 k=1 k=0 k=2 k=1 ‘() ‘() ‘(x 1 ) ‘() ‘(x 0 ) ‘() ‘(x 0 ) ‘(x 0 x 1 )

Symbolic evalua<on executes all paths through a program Inputs are unknown #lang rosette (trying to find values that violate spec) ( define (first-k-even lst k) ( define xs ( filter even? lst)) ( take xs k)) because filter ran (filter even? ‘(x 0 x 1 )) on a list of size 2 ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) take runs 2 2 8mes ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 ) k=0 k=0 k=1 k=0 k=1 k=0 k=2 k=1 ‘() ‘() ‘(x 1 ) ‘() ‘(x 0 ) ‘() ‘(x 0 ) ‘(x 0 x 1 )

Blaming filter even though it’s not the slowest

Symbolic profiling Data structures and metrics

Two data structures to summarize symbolic evalua<on ‘() ∧ ∧ ∧ ∧ ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬ ¬ ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) (even? x 0 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 ) Symbolic evalua<on graph Symbolic heap Reflects the evaluator’s strategy 
 Shape of all symbolic values 
 for all-paths execu8on of the program created by the program Any symbolic evalua<on technique can be summarized by these two data structures

The symbolic evalua4on graph summarizes branching and merging (filter even? ‘(x 0 x 1 )) Symbolic evalua<on graph ¬(even? x 0 ) (even? x 0 ) • Nodes are program states • Edges are transi8ons ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) between states ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 ) k=0 k=0 k=1 k=0 k=1 k=0 k=2 k=1 ‘() ‘() ‘(x 1 ) ‘() ‘(x 0 x 1 ) ‘(x 0 ) ‘() ‘(x 0 )

The symbolic evalua4on graph summarizes branching and merging (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 )

The symbolic evalua4on graph summarizes branching and merging Symbolic execu8on (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 )

The symbolic evalua4on graph summarizes branching and merging Symbolic execu8on Bounded model checking (filter even? ‘(x 0 x 1 )) (filter even? ‘(x 0 x 1 )) ¬(even? x 0 ) (even? x 0 ) ¬(even? x 0 ) (even? x 0 ) ‘() ‘(x 0 ) ‘() ‘(x 0 ) ¬(even? x 1 ) (even? x 1 ) ¬(even? x 1 ) (even? x 1 ) ‘() ‘(x 1 ) ‘(x 0 ) ‘(x 0 x 1 )