Model Checking with Maximal Causality Reduction Jeff Huang - PowerPoint PPT Presentation

Model Checking with Maximal Causality Reduction Jeff Huang Assistant Professor

A Real Bug – $12 million loss of equipment https://stackoverflow.com/questions/16159203/why-does-this-java-program-terminate-despite-that-apparently-it-shouldnt-and-d curPos = new Point(1,2); class Point { int x , y; } Thread 1: x=0 newPos = new y=0 Point( curPos.x+1, curPos.y+1 ); x=curPos.x+1 y=curPos.y+1 Thread 2: curPos = object while (newPos != null) if ( new Pos . x + 1 != new Pos . y ) ERROR

A Real Bug – $12 million loss of equipment https://stackoverflow.com/questions/16159203/why-does-this-java-program-terminate-despite-that-apparently-it-shouldnt-and-d curPos = new Point(1,2); class Point { int x , y; } Thread 1: x=0 newPos = new y=0 Point( curPos.x+1, curPos.y+1 ); x=curPos.x+1 y=curPos.y+1 Thread 2: curPos = object newPos while (newPos != null) if ( new Pos . x + 1 != statements are out of program order new Pos . y ) ERROR

Maximal Causality Reduction • Open source: https://github.com/parasol-aser/JMCR • Implementation - Java 8, multi-threading - The Z3 SMT solver • Evaluation - Takes only two runs to find the error in <1s - Orders of magnitude more effective than partial order reduction and bounded model checking - Finding new errors (data races and NPEs) in extensively studied popular benchmarks

2007 Turing Award Edmund Clarke Allen Emerson Joseph Sifakis For their role in “ developing Model-Checking into a highly effective verification technology …”

The Key Challenge: State Explosion

The Key Challenge: State Explosion

Two Classical Approaches • Partial Order Reduction [2014 CAV Award] to Godefroid, Peled, Valmari, and Wolper • Reduce the size of the state space that needs to be searched • Exploit the independence between concurrently executed transitions, which result in the same state • Bounded Model Checking Clarke, Biere, Raimi, Zhu (2001) • Limit the searched space to a certain bound

Partial Order Reduction The two sequences • s ➝ ! ➝ s 1 ➝ " ➝ r • s ➝ " ➝ s 2 ➝ ! ➝ r belong to the same equivalent class . If the specification does not distinguish between these sequences, it is beneficial to consider only one with 2 + 1 states.

Partial Order Reduction The two sequences Limited by • s ➝ ! ➝ s 1 ➝ " ➝ r Happens-Before • s ➝ " ➝ s 2 ➝ ! ➝ r belong to the same equivalent class . If the specification does not distinguish between these sequences, it is beneficial to consider only one with 2 + 1 states.

Bounded Model Checking • Restrict search to states that are reachable from initial state within fixed number k of transitions Can the given property fail in k-steps? I(V 0 ) Æ T(V 0 ,V 1 ) Æ … Æ T(V k-1 ,V k ) Æ ( : P(V 0 ) Ç … Ç: P(V k )) Property fails Initial state k-steps in some step

Bounded Model Checking • Restrict search to states that are reachable from initial state within fixed number k of transitions Incomplete i.e., limited to k Can the given property fail in k-steps? I(V 0 ) Æ T(V 0 ,V 1 ) Æ … Æ T(V k-1 ,V k ) Æ ( : P(V 0 ) Ç … Ç: P(V k )) Property fails Initial state k-steps in some step

Example initially x = y =0 T2 T3 T1 loop twice: loop twice: loop twice: 5: lock(l) if( x >1) 11: 1: lock(l) 6: x =0 if( y ==3) 12: 2: x =1 7: unlock(l) Error 13: 3: y =1 8: if( x >0) else 14: 4: unlock(l) 9: y ++ y =2 15: 10: x =2

Example initially x = y =0 T2 T3 T1 loop twice: loop twice: loop twice: 5: lock(l) if( x >1) 11: 1: lock(l) 6: x =0 if( y ==3) 12: 2: x =1 7: unlock(l) Error 13: 3: y =1 8: if( x >0) else 14: 4: unlock(l) 9: y ++ y =2 15: 10: x =2 T2T2T2 - T1T1T1T1 - T2T2T2T2 - T3T3T3 - T2T2T2T2 - T1T1 - T2T2T2T2 - T3T3

Example initially x = y =0 T2 T3 T1 loop twice: loop twice: loop twice: 5: lock(l) if( x >1) 11: 1: lock(l) 6: x =0 if( y ==3) 12: 2: x =1 7: unlock(l) Error 13: 3: y =1 8: if( x >0) else 14: 4: unlock(l) 9: y ++ y =2 15: 10: x =2 T2T2T2 - T1T1T1T1 - T2T2T2T2 - T3T3T3 - T2T2T2T2 - T1T1 - T2T2T2T2 - T3T3 7 thread context switches

Example initially x = y =0 T2 T3 T1 loop twice: loop twice: loop twice: 5: lock(l) DFS explores 3,293,931 runs if( x >1) 11: 1: lock(l) 6: x =0 if( y ==3) 12: 2: x =1 in an hour without finding the error 7: unlock(l) Error 13: 3: y =1 8: if( x >0) else 14: 4: unlock(l) 9: y ++ y =2 15: 10: x =2 T2T2T2 - T1T1T1T1 - T2T2T2T2 - T3T3T3 - T2T2T2T2 - T1T1 - T2T2T2T2 - T3T3 7 thread context switches

Example

Example Bounded 77,322 executions Model Checking 20 seconds Bounding #thread preemptions

Example Bounded 77,322 executions Model Checking 20 seconds Bounding #thread preemptions + Partial Order 3,782 executions Reduction 3 seconds Based on happens-before

Example Bounded 77,322 executions Model Checking 20 seconds Bounding #thread preemptions + Partial Order 3,782 executions Reduction 3 seconds Based on happens-before Maximal 46 executions Causality 2 seconds Reduction

Example Bounded 77,322 executions Model Checking 20 seconds Bounding #thread preemptions + Partial Order Happens-Before 3,782 executions Reduction Limitation 3 seconds Based on happens-before Maximal 46 executions Causality 2 seconds Reduction

Happens-Before Limitation Enforces dependence between conflicting reads and writes p: q: r: write x write x read x Happens-before: six non-redundant transitions p.q.r p.r.q q.p.r q.r.p r.p.q r.q.p

Happens-Before Limitation Enforces dependence between conflicting reads and writes p: q: r: write x write x read x Happens-before: six non-redundant transitions p.q.r p.r.q q.p.r q.r.p r.p.q r.q.p In fact: only four are non-redundant p.q.r == q.r.p r.q.p == r.p.q

Happens-Before Limitation Enforces dependence between conflicting reads and writes p: q: r: write x write x read x Happens-before: six non-redundant transitions p.q.r p.r.q q.p.r q.r.p r.p.q r.q.p In fact: only four are non-redundant r is the only read p.q.r == q.r.p r.q.p == r.p.q

Happens-Before Limitation Enforces dependence between conflicting reads and writes p: q: r: write x write x read x Happens-before: six non-redundant transitions p.q.r p.r.q q.p.r q.r.p r.p.q r.q.p If p and q write the same value, then only two non-redundant transitions: In fact: only four are non-redundant r is the only read p.q.r == q.p.r == q.r.p == p.r.q r.q.p == r.p.q p.q.r == q.r.p r.q.p == r.p.q

Example initially x = y =0 T2 T3 T1 loop twice: loop twice: loop twice: 5: lock(l) if( x >1) 11: 1: lock(l) 6: x =0 if( y ==3) 12: 2: x =1 loop N times 7: unlock(l) Error 13: 3: y =1 8: if( x >0) else 14: 4: unlock(l) 9: y ++ y =2 15: 10: x =2

Example

Example Bounded 520,959 executions Model Checking 183 seconds Bounding #thread preemptions

Example Bounded 520,959 executions Model Checking 183 seconds Bounding #thread preemptions + Partial Order 221,852 executions Reduction 93 seconds Based on happens-before

Example Bounded 520,959 executions Model Checking 183 seconds Bounding #thread preemptions + Partial Order 221,852 executions Reduction 93 seconds Based on happens-before Maximal 50 executions Causality 4 seconds Reduction

Maximal Causality Reduction Key idea: characterizing redundant transitions with maximal causality t : a trace of read and write Ω ( t ) a maximal set of equivalent transitions t : takes the value of reads and writes into consideration Ω (t) : contains all transitions which all programs that can generate t can also generate Serbanuta, Chen and Rosu, Maximal Causal Models for Sequentially Consistent Systems, RV’12

Maximal Causality Reduction 1. Online tracing t Program) Stateless'Model'Checker' Seed)interleaving) Scheduler ) 2. Construct Ω ( t ) New) t Trace) interleaving) Maximal'Causality'Engine' New$seed$ 3. Offline property Ω ( t ) interleavings$ …) checking with Ω ( t ) …) 4. Generate new …) seed interleavings …) Maximal)causality)envelope) with Ω ( t )

Maximal Causality Reduction 1. Online tracing t Program) Stateless'Model'Checker' Seed)interleaving) Scheduler ) 2. Construct Ω ( t ) New) Trace) t interleaving) Maximal'Causality'Engine' New$seed$ 3. Offline property Ω ( t ) interleavings$ …) checking with Ω ( t ) …) 4. Generate new …) Seed interleaving : an interleaving in Ω ( t ) with at seed interleavings …) least one read forced to see a different value Maximal)causality)envelope) with Ω ( t ) Following a seed interleaving will produce a new state

Maximal Causality Reduction N = 1, 2, ... , 10

Maximal Causality Reduction N = 1, 2, ... , 10

Maximal Causality Reduction N = 1, 2, ... , 10 MCR is almost insensitive to N when N>3 Reduced #explorations by BMC+POR by two orders of magnitude

Maximal Causality Reduction 1. Online tracing t Program) Stateless'Model'Checker' Seed)interleaving) Scheduler ) 2. Construct Ω ( t ) New) t Trace) interleaving) Maximal'Causality'Engine' New$seed$ 3. Offline property Ω ( t ) interleavings$ …) checking with Ω ( t ) …) 4. Generate new …) seed interleavings …) Maximal)causality)envelope) with Ω ( t )