Step ep-bounded bounded Bisimu imulation lation Nishanthan - PowerPoint PPT Presentation

Step ep-bounded bounded Bisimu imulation lation Nishanthan Kamaleson University of Birmingham Joint work with: David Parker, Jonathan Rowe, Mate Puljiz and Chris Good HIERATIC meeting, Jena, September 2014

Overview • Markov chains & bisimulation • Splitter based bisimulation minimisation • Signature based bisimulation minimisation • Splitter vs. Signature • Step-bounded bisimulation minimisation

Discrete-time Markov chains (DTMCs) • A DTMC D is a tuple (S,s init ,P,L) where: − S is a (finite) set of states ( “ state space ” ) − s init ∈ S is the initial state B 1 0.1 − P : S × S → [0,1] is the 0.2 1 s 0 s 1 s 2 s 3 1 transition probability matrix 0.3 where Σ s ’∈S P(s,s ’) = 1 for all s ∈ S 1 1 0.1 1 0.4 − L : S → 2 AP is a function labelling 0.9 {a} s 4 s 5 s 6 states with atomic propositions 0. 7

Bisimulation on DTMCs • Probabilistic bisimulation − preservation of stepwise behaviour (and labels) • Equivalence relation R on S is a probabilistic bisimulation on D if and only if, for all s 1 R s 2 : − L(s 1 ) = L(s 2 ) − P(s 1 , T) = P(s 2 , T) for all T ∈ S/R (i.e. for all equivalence classes of R) − where: P(s,T) = Σ s ’∈T P(s,s ’ ) for T ⊆ S • We write ~ for the coarsest possible bisimulation − we say states s 1 and s 2 are bisimilar if s 1 ~ s 2 − and DTMCs D 1 and D 2 are bisimilar if their initial states are • Probabilistic bisimulation preserves PCTL, PCTL*, LTL, …

Probabilistic temporal logics • PCTL = Probabilistic Computation Tree Logic [HJ94] − e.g. send → P ≥0.95 [ F deliver ] − “ if a message is sent, then the probability of it being delivered is at least 0.95 ” • Numerical (and step-bounded) queries − P =? [ F ≤10 deliver ] - "what is the probability of − the message being delivered within 10 steps? ” • Extensions: Probabilistic LTL and PCTL* − e.g. P ≥ 1 [ GF ready ] – “ with probability 1, the server always eventually returns to a ready-state ” − e.g. P <0.01 [ FG error ] – “ with probability at most 0.01, an irrecoverable error occurs ” − e.g. P >0.5 [ GF crit 1 ] ∧ P >0.5 [ GF crit 2 ]

Splitter Based Bisimulation minimisation • 1. Start with initial partition − say Π = { { s ∈ S | L(s)=lab } | lab ∈ 2 AP } • 2. Find a splitter T ∈ Π for some block B ∈ Π − a splitter T is a block such that probability of going to T differs for some states in block B − i.e. ∃s,s’∈ B . P (s,T) ≠ P(s ’ ,T) • 3. Split B into sub-blocks − such that P(s,T) is the same for all states in each sub-block • 4. Repeat steps 2/3 until no more splitters exist − i.e. no change to partition Π 6

Example Minimisation: B 1 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 B 2 is a splitter for B 1 s 0 s 1 s 2 s 3 1 0.3 (since e.g. P(s 1 ,B 2 )=0≠1= P(s 2 ,B 2 )) 1 1 0.1 0.3 1 0.4 0.9 {a} s 4 s 5 s 6 0.7 B 2 7

Example Minimisation: B 1 B 3 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 B 2 is a splitter for B 1 s 0 s 1 s 2 s 3 1 0.3 (since e.g. P(s 1 ,B 2 )=0≠1= P(s 2 ,B 2 )) 1 1 0.1 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } 0.3 1 0.4 B 3 is a splitter for B 1 0.9 {a} s 4 s 5 s 6 0.7 (since e.g. P(s 1 ,B 3 )=0≠0.3= P(s 0 ,B 3 )) B 2 8

Example Minimisation: B 1 B 3 B 4 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 B 2 is a splitter for B 1 s 0 s 1 s 2 s 3 1 0.3 (since e.g. P(s 1 ,B 2 )=0≠1= P(s 2 ,B 2 )) 1 1 0.1 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } 0.3 1 0.4 {a} B 3 is a splitter for B 1 0.9 s 4 s 5 s 6 0.7 (since e.g. P(s 1 ,B 3 )=0≠0.3= P(s 0 ,B 3 )) B 2 Π 2 : B 1 ={s 1 ,s 5, s 4 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 }, B 4 ={s 0 } B 1 is a splitter for B 4 (since e.g. P(s 4 ,B 1 )= 0.3≠1.0= P(s 1 ,B 1 )) 9

Example Minimisation: B 4 B 3 B 1 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 B 2 is a splitter for B 1 s 0 s 1 s 2 s 3 1 0.3 (since e.g. P(s 1 ,B 2 )=0≠1= P(s 2 ,B 2 )) 1 1 0.1 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } 0.3 1 0.4 {a} B 3 is a splitter for B 1 0.9 s 4 s 5 s 6 0.7 (since e.g. P(s 1 ,B 3 )=0≠0.3= P(s 0 ,B 3 )) B 5 B 2 Π 2 : B 1 ={s 1 ,s 5, s 4 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 }, B 4 ={s 0 } B 1 is a splitter for B 4 S/~ = { {s 1 ,s 5 }, {s 6 }, {s 2 ,s 3 }, {s 0 },{s 4 } } (since e.g. P(s 4 ,B 1 )=0.3≠1.0= P(s 1 ,B 1 )) No more splitters 10

Example S/~ = { {s 1 ,s 5 }, {s 6 }, {s 2 ,s 3 }, {s 0 },{s 4 } } 0.1 0.2 1 s 0 s 1 s 2 s 3 0.3 1 0.3 1 s 0 1 1 0.7 0.1 s 1 ,s 5 s 6 s 2 ,s 3 0.3 1 0.3 0.4 1 0.7 1 1 s 4 {a} 0.9 s 4 s 5 {a} s 6 0.7 11

Signature Based Bisimulation minimisation • 1. Start with initial partition • 2. Generate signature for each state s ∈ S − i.e. lift the distribution P(s, ⋅ ) for all states s ∈ S to Π − where signature of s is its lifted distribution • 3. Update Π based on the signature − such that signature is the same for all states in each block • 4. Repeat steps 2/3/4 until no more change in Π 12

Example Minimisation: B 1 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 • signature of s 0, s 1, s 4, s 5 = B 1 :1.0 s 0 s 1 s 2 s 3 1 0.3 • signature of s 2, s 3 = B 2 :1.0 1 1 0.1 Similarly signatures will computed for other 0.3 1 0.4 states. 0.9 {a} s 4 s 5 s 6 0.7 B 2 13

Example Minimisation: B 1 B 3 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 • signature of s 0, s 1, s 4, s 5 = B 1 :1.0 s 0 s 1 s 2 s 3 1 0.3 • signature of s 2, s 3 = B 2 :1.0 1 1 0.1 Similarly signatures will computed for all the 0.3 1 0.4 states. 0.9 {a} s 4 s 5 s 6 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } 0.7 • signature of s 0 = B 1 :0.7, B 3 :0.3 B 2 • signature of s 1, s 5, s 4 = B 1 :1.0 14

Example Minimisation: Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } B 1 B 3 B 4 0.1 • signature of s 0, s 1, s 4, s 5 = B 1 :1.0 0.2 1 • signature of s 2, s 3 = B 2 :1.0 s 0 s 1 s 2 s 3 1 0.3 Similarly signatures will computed for all the 1 1 states. 0.1 0.3 1 0.4 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } {a} 0.9 s 4 s 5 s 6 • signature of s 0 = B 1 :0.7, B 3 :0.3 0.7 • signature of s 1, s 5, s 4 = B 1 :1.0 B 2 Π 2 : B 4 ={s 1 ,s 5, s 4 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 }, B 1 ={s 0 } • signature of s 1, s 5 = B 1 :1.0 • signature of s 4 = B 1 :0.3, B 4 :0.7 15

Example Minimisation: Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } B 1 B 3 B 4 0.1 • signature of s 0, s 1, s 4, s 5 = B 1 :1.0 0.2 1 • signature of s 2, s 3 = B 2 :1.0 s 0 s 1 s 2 s 3 1 0.3 Similarly signatures will computed for all the 1 1 states. 0.1 0.3 0.4 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } {a} 0.9 s 4 s 5 s 6 • signature of s 1, s 5, s 4 = B 1 :1.0 0.7 • signature of s 0 = B 1 :0.7, B 3 :0.3 B 5 B 2 Π 2 : B 4 ={s 1 ,s 5, s 4 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 }, B 1 ={s 0 } • signature of s 1, s 5 = B 1 :1.0 S/~ = { {s 1 ,s 5 }, {s 6 }, {s 2 ,s 3 }, {s 0 },{s 4 } } • signature of s 4 = B 1 :0.3, B 4 :0.7 Further splitting is not possible 16

Example S/~ = { {s 1 ,s 5 }, {s 6 }, {s 2 ,s 3 }, {s 0 },{s 4 } } 0.1 0.2 1 s 0 s 1 s 2 s 3 0.3 1 0.3 1 s 0 1 1 0.7 0.1 s 1 ,s 5 s 6 s 2 ,s 3 0.3 1 0.3 0.4 1 0.7 1 1 s 4 {a} 0.9 s 4 s 5 {a} s 6 0.7 17

Signature vs. Splitter • Implemented signature/splitter based algorithms in PRISM − tested on standard PRISM benchmarks − splitter based algorithm uses splay trees to efficiently store block information • Splitter based algorithm has showed better performance than the signature based algorithm − signature based considers all states in every iteration − splitter based only considers the states that have outgoing transitions to the splitter 18

Signature vs. Splitter • Compare number of iterations for each algorithm • Splitter-based needs more splitting iterations and the splitting is not step-wise. • Splitting process of the signature based algorithm requires fewer number of iteration and is step-wise. 19

Step-Bounded Bisimulation • Consider a step-bounded reachability question like P =? [F <=k (“s 6 ” )] • In traditional bisimulation minimisation, the quotient model will be built when there is no further possible splitting • But, refining up to the point where the quotient model can answer the above question • Signature based algorithm is suitable for this approach because it splits step-wise • We only perform k (step-wise) splitting iterations 20

After 1 iteration Minimisation: B 1 B 3 0.1 Π 0: B 1 ={s 0 ,s 1 ,s 2 ,s 3 ,s 4 ,s 5 }, B 2 ={s 6 } 0.2 1 • signature of s 0, s 1, s 4, s 5 = B 1 :1.0 s 0 s 1 s 2 s 3 1 0.3 • signature of s 2, s 3 = B 2 :1.0 1 1 0.1 Similarly signatures will computed for all the 0.3 1 0.4 states. 0.9 {a} s 4 s 5 s 6 Π 1 : B 1 ={s 0 ,s 1 ,s 4 ,s 5 }, B 2 ={s 6 }, B 3 ={s 2 ,s 3 } 0.7 • signature of s 1, s 5, s 4 = B 1 :1.0 B 2 • signature of s 0 = B 1 :0.7, B 3 :0.3 21

After 1 iteration • Constructing the Markov chain S/~ = { {s 0, s 1 ,s 4, s 5 }, {s 6 }, {s 2 ,s 3 }} 0.1 0.2 1 1.0 s 0 s 1 s 2 s 3 1 0.3 1 1 0.1 s 0 ,s 1 ,s 4, s 5 s 6 s 2 ,s 3 1 0.3 1 0.4 1 1 0.9 {a} {a} s 4 s 5 s 6 0.7 22

How do blocks grow with iterations ? 23