Aditya V. Nori, Sriram K. Rajamani Programming Languages and Tools - PowerPoint PPT Presentation

Aditya V. Nori, Sriram K. Rajamani Programming Languages and Tools Microsoft Research India

 An industrial strength program verifier  Philosophy: Synergize verification and testing  Synergy [ FSE ’06 ], Dash [ ISSTA ‘08 ], Smash [ POPL ‘10 ], Bolt [submitted] algorithms to perform scalable analysis  Engineered a number of optimizations for scalability  Integrated with Microsoft’s Static Driver Verifier ( SDV ) toolkit and used internally

Questi tion on Does the assertion hold for all possible inputs? void f(int *p, int *q) { 0: *p = 4; 1: *q = 5; 2: assert ( ¬𝜒 𝑓𝑠𝑠𝑝𝑠 ) Must analysis: finds bugs, but can’t prove their } absence May analysis: can prove the absence of bugs, but can result in false errors More generally, we are interested in the query ? 𝑔 𝜒 𝑓𝑠𝑠𝑝𝑠 〉 〈𝜒 𝑞𝑠𝑓

? = 𝑧𝑓𝑡 test 𝑈 𝑔 ∗ 𝑞 ≠ 4 = (𝑞 = 𝑟) void f(int *p, int*q) { 0: *p = 4; 1: *q = 5; } ⊆ ∗ 𝑞 ≠ 4 Captures facts that are guaranteed to hold on particular • executions of the program ( under-approximation ) Error condition is reachable by any input that satisfies (𝑞 = 𝑟) •

? = 𝑜𝑝 proof 𝑞 ≠ 𝑟 𝑔 ∗ 𝑞 ≠ 4 (𝑞 ≠ 𝑟) void f(int *p, int*q) 0 { 0: *p = 4; (𝑞 ≠ 𝑟) (𝑞 = 𝑟) 1 1 1: *q = 5; } ( ∗ 𝑞 ≠ 4) 2 Captures facts that are true for all executions of the • program ( over-approximation ) Proof can be obtained by keeping track of the predicates • (𝑞 = 𝑟) and ( ∗ 𝑞 ≠ 4)

 Algorithm uses only test case generation operations  Maintains two data structures: ▪ A forest of reachable concrete states (tests) ▪ Under-approximates executions of the program ▪ A region graph (an abstraction) ▪ Over-approximates all executions of the program  Our goal: bug finding and proving ▪ If a test reaches an error, we have found bug ▪ If we refine the abstraction so that there is *no* path from the initial region to error region, we have a proof  Key ideas ▪ Frontier 𝛽 uses only aliases α that are present along ▪ 𝑋𝑄 concrete tests that are executed

0 Step 1: Try to generate a test that crosses the frontier 1 Perform symbolic 𝜒 1  2 simulation on the path until the frontier and 3 generate a constraint 𝜒 1 frontier Conjoin with the condition  4 𝜒 2 needed to cross 5 𝜒 2 frontier 7 Is 𝜒 1 ∧ 𝜒 2 satisfiable?  8 6 10 10 9

0 Step 1: Try to generate a test that crosses the frontier 1 Perform symbolic  2 simulation on the path until the frontier and 3 generate a constraint 𝜒 1 frontier Conjoin with the condition  4 𝜒 2 needed to cross 5 frontier 7 Is 𝜒 1 ∧ 𝜒 2 satisfiable? [YES]  8 6 Step 2: run the test and 10 10 9 extend the frontier

Step 1: Try to generate a test 0 that crosses the frontier 1 Perform symbolic  simulation on the path 2 until the frontier and frontier generate a constraint 𝜒 1 3 Conjoin with the condition  𝜒 2 needed to cross 4 4 frontier 5 7 Is 𝜒 1 ∧ 𝜒 2 satisfiable? [NO]  8 Step 2: use 𝑋𝑄 𝛽 to refine so 6 that the frontier moves 10 10 9 back!

× 𝑧 = 1 0 1 Input: ut: Program m 𝑸 Property rty 𝝎 void f(int y) { 2 Cons nstr truc uct t initi tial al abstra tracti tion 0: int lock, x; Cons nstruc uct rand ndom m tests 1: do { 2: lock = 1; 3 3: x = y; 4: if (*) { yes Test t 5: lock = 0; Bug ug! succeeded? 6: y = y+1; 4 no no } yes 7: } while (x != y) Abstrac tracti tion Proof! succeeded? 8: if (lock != 1) 9: error(); 5 7 no no 10: τ = error path in abstraction } f = fronti ntier of error path th 6 8 Can exte tend test t beyond yes fronti tier? 𝜐 = (0,1,2,3,4,7,8,9 ) 9 no no Symbolic execution + Refine ne abstra tracti tion Theorem proving frontier 10

0 1 8: ρ 8:¬ ρ 8 refine 2 9 9 3 𝜍 = (𝑚𝑝𝑑𝑙. 𝑡𝑢𝑏𝑢𝑓 ! = 𝑀) 4 5 7 6 8 9 10

0 1 8:p 8 8:¬p refine 2 9 9 3 p = (𝑚𝑝𝑑𝑙. 𝑡𝑢𝑏𝑢𝑓 ! = 𝑀) 4 5 7 6 8: ¬𝑞 8:p 9 10

× 0 1 Input: ut: Program m 𝑸 Property rty 𝝎 void f(int y) { 2 Cons nstr truc uct t initi tial al abstra tracti tion 0: int lock, x; Cons nstruc uct rand ndom m tests 1: do { 2: lock = 1; 3 3: x = y; 4: if (*) { yes Test t 5: lock = 0; Bug ug! succeeded? 6: y = y+1; 4 no no } yes 7: } while (x != y) Abstrac tracti tion Proof! succeeded? 8: if (lock != 1) 9: error(); 5 7 no no 10: τ = error path in abstraction } f = fronti ntier of error path th 6 8: ¬𝑞 8:p Can exte tend test t beyond yes fronti tier? 9 no no 𝜐 = (0,1,2,3,4,7, < 8, 𝑞 >, 9) Refine ne abstra tracti tion frontier 10

1 2 0 void f(int y) { 0: int lock, x; 1: do { 3 2: lock = 1; 3: x = y; 4⋀s 4⋀¬s 4: if (*) { 5: lock = 0; 6: y = y+1; } 5⋀s 5⋀¬s 7: } while (x != y) 8: if (lock != 1) 9: error(); 6⋀¬r 6⋀r 10: } 7⋀q 7⋀¬q 8⋀p 8⋀¬p 9 10

S k-2 T Key idea Perform a recursive Dash query on the called procedure and use S k-1 the result to either generate a frontier 𝛽 test or compute 𝑋𝑄 𝐷𝐵𝑀𝑀(𝑔𝑝𝑝(𝑗, 𝑘)) S k

S k-2 T  1 ? S k-1 Dash 〈𝜒 1 𝑔𝑝𝑝 𝜒 2 〉 - pass: perform refinement 𝐷𝐵𝑀𝑀(𝑔𝑝𝑝(𝑗, 𝑘)) - fail: generate test  2 S k

must summary A must summary for a procedure 𝒬 𝑗 is of • 𝜒 1 𝑛𝑣𝑡𝑢 the form 𝜒 1 , 𝜒 2 ∈ 𝒬 𝑗 𝒬 𝑗 ∀𝑢 ∈ 𝜒 2 . ∃𝑡 ∈ 𝜒 1 . 𝑢 can be obtained by • 𝜒 2 executing 𝒬 𝑗 from an initial state 𝑡 ¬𝑛𝑏𝑧 𝑡𝑣𝑛𝑛𝑏𝑠𝑧 A ¬𝑛𝑏𝑧 𝑡𝑣𝑛𝑛𝑏𝑠𝑧 for a procedure 𝒬 𝑗 is of • ¬𝑛𝑏𝑧 the form 𝜒 1 , 𝜒 2 ∈ 𝒬 𝑗 𝜒 1 ∀𝑡 ∈ 𝜒 1 ∀𝑢 ∈ 𝜒 2 . 𝑢 cannot be obtained by • 𝒬 𝑗 executing 𝒬 𝑗 starting in state 𝑡 𝜒 2

𝜒 1 ∈ Π 𝑜 1 𝜒 2 ∈ Π 𝑜 2 𝜒 1 ∩ Ω 𝑜 1 ≠ ∅ 𝜒 2 ∩ Ω 𝑜 2 = ∅ 𝑓 = (𝑜 1 , 𝑜 2 ) ∈ 𝐹 𝒬 𝑗 𝑗𝑡 𝑏 𝑑𝑏𝑚𝑚 𝑢𝑝 𝑞𝑠𝑝𝑑𝑓𝑒𝑣𝑠𝑓 𝒬 𝑘 𝑛𝑣𝑡𝑢 𝜒 1 , 𝜒 2 ∈ 𝒬 𝑘 Ω 𝑜 1 ⊇ 𝜒 1 𝜄 ⊆ 𝜒 2 𝜒 2 ∩ 𝜄 ≠ ∅ [MUST − POST − USESUM] Ω 𝑜 2 ≔ Ω 𝑜 2 ∪ 𝜄 𝑈 0 procedure 𝒬 𝑗 must summary 𝑈 1 frontier 1 ⊆ Ω 𝑜1 𝜒 𝑘 𝒬 𝜒 1 Ω 𝑜1 2 2 ⊇ 𝜄) ∧ (𝜒 2 ∩ 𝜄 ≠ ∅) (𝜒 𝑘 Γ 𝑓 = 𝑑𝑏𝑚𝑚 𝒬 𝜒 2 𝑈 4 3 Check if frontier (𝑜 1 , 𝑜 2 ) can be extended by a • must summary 𝜒 2 1 , 𝜒 𝑈 5 6 𝑈 If yes, grow Ω 𝑜 2 with 𝜄 ⊆ 𝜒 2 • 2 𝜒 7

𝜒 1 ∈ Π 𝑜 1 𝜒 2 ∈ Π 𝑜 2 𝜒 1 ∩ Ω 𝑜 1 ≠ ∅ 𝜒 2 ∩ Ω 𝑜 2 = ∅ 𝑓 = (𝑜 1 , 𝑜 2 ) ∈ 𝐹 𝒬 𝑗 𝑗𝑡 𝑏 𝑑𝑏𝑚𝑚 𝑢𝑝 𝑞𝑠𝑝𝑑𝑓𝑒𝑣𝑠𝑓 𝒬 𝑘 𝑛𝑣𝑡𝑢 𝜒 1 , 𝜒 2 ∈ 𝒬 𝑘 Ω 𝑜 1 ⊇ 𝜒 1 𝜄 ⊆ 𝜒 2 𝜒 2 ∩ 𝜄 ≠ ∅ [MUST − POST − USESUM] Ω 𝑜 2 ≔ Ω 𝑜 2 ∪ 𝜄 𝑈 0 procedure 𝒬 𝑗 must summary 𝑈 1 frontier 1 ⊆ Ω 𝑜1 𝜒 𝑘 𝒬 𝜒 1 Ω 𝑜1 2 2 ⊇ 𝜄) ∧ (𝜒 2 ∩ 𝜄 ≠ ∅) (𝜒 𝑘 Γ 𝑓 = 𝑑𝑏𝑚𝑚 𝒬 𝜒 2 𝜄 𝑈 4 3 Check if frontier (𝑜 1 , 𝑜 2 ) can be extended by a • must summary 𝜒 2 1 , 𝜒 𝑈 5 6 𝑈 If yes, grow Ω 𝑜 2 with 𝜄 ⊆ 𝜒 2 • 2 𝜒 7

𝜒 1 ∈ Π 𝑜 1 𝜒 2 ∈ Π 𝑜 2 𝜒 1 ∩ Ω 𝑜 1 ≠ ∅ 𝜒 2 ∩ Ω 𝑜 2 = ∅ 𝑓 = (𝑜 1 , 𝑜 2 ) ∈ 𝐹 𝒬 𝑗 𝑗𝑡 𝑏 𝑑𝑏𝑚𝑚 𝑢𝑝 𝑞𝑠𝑝𝑑𝑓𝑒𝑣𝑠𝑓 𝒬 𝑘 ¬𝑛𝑏𝑧 𝜒 1 , 𝜒 2 ∈ 𝒬 𝑘 𝜒 2 ⊆ 𝜒 2 𝜄 ⊆ 𝜒 1 ¬𝜄 ∩ Ω 𝑜1 = ∅ ∪ 𝜒 1 ∩ 𝜄, 𝜒 1 ∩ ¬𝜄 𝑂 𝑓 ≔ 𝑂 𝑓 ∪ { 𝜒 1 ∩ 𝜄, 𝜒 2 } [NMAY − PRE − USESUM] Π 𝑜 1 ≔ Π 𝑜 1 ∖ 𝜒 1 𝑈 0 ¬𝑛𝑏𝑧 𝑡𝑣𝑛𝑛𝑏𝑠𝑧 procedure 𝒬 𝑗 𝑈 1 ⊇ 𝜄) ∧ (¬𝜄 ∩ Ω 𝑜1 = ∅) (𝜒 1 frontier 𝑘 𝒬 𝜒 1 ∩ 𝜄 𝜒 1 ∩ ¬𝜄 2 2 𝑂 𝑓 2 ⊇ 𝜒 2 𝜒 𝑘 Γ 𝑓 = 𝑑𝑏𝑚𝑚 𝒬 𝑘 Γ 𝑓 = 𝑑𝑏𝑚𝑚 𝒬 𝜒 2 𝑈 4 3 Check if frontier (𝑜 1 , 𝑜 2 ) can be refined by a • ¬𝑛𝑏𝑧 𝑡𝑣𝑛𝑛𝑏𝑠𝑧 𝜒 2 1 , 𝜒 𝑈 5 6 𝑈 If yes, use 𝜄 ⊆ 𝜒 1 to refine the abstraction • If both must and ¬𝑛𝑏𝑧 summaries are not • available, analyze procedure 𝒬 𝑘 2 𝜒 7 𝑧𝑓𝑡 𝑛𝑣𝑡𝑢 𝑡𝑣𝑛𝑛𝑏𝑠𝑧 for 𝒬 𝑘 • 𝑜𝑝 ¬𝑛𝑏𝑧 𝑡𝑣𝑛𝑛𝑏𝑠𝑧 for 𝒬 𝑘 •

 Engineering for making Yogi robust, scalable and industrial strength  Several of the implemented optimizations are folklore  Very difficult to design tools that are bug free evaluating optimizations is hard!  Our empirical evaluation gives tool builders information about what gains can be realistically expected from optimizations  Details in ICSE ‘10  Vanilla implementation of algorithms:  ( flpydisk , CancelSpinLock ) took 2 hours  Algorithms + engineering + optimizations:  ( flpydisk , CancelSpinLock ) took less than 1 second!