Reachability in Stochastic Hybrid Systems [Ongoing Work] Patricia Bouyer 1 Thomas Brihaye 2 Mickael Randour 2,3 Cédric Rivière 2 Pierre Vandenhove 1,2,3 1 LSV, CNRS & ENS Paris-Saclay, Université Paris-Saclay, France 2 Université de Mons, Mons, Belgium 3 F.R.S.-FNRS September 12, 2019 – Reachability Problems, Brussels
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Outline • Verification of models combining: • stochastic aspects (e.g., Markov chains); • hybrid aspects (with both discrete and continuous transitions); � stochastic hybrid systems . • Properties about the reachability of states (is some set of states reached with probability 1? Can we compute the probability of reaching a set?). Goal Identify a decidability frontier for reachability in stochastic hybrid systems. Method Follow an approach that has been successful for infinite Markov chains . Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 2 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Reachability in infinite Markov chains Let M be a countable Markov chain. 1 1 2 2 1 Target: { a } 2 a c b d 1 � { a } = { d } 1 1 2 Let B ⊆ S be a subset of states, s ∈ S be an initial state. Goal Compute (or approximate) Prob M s ( ♦ B ). We set B = { s ∈ S | Prob M � s ( ♦ B ) = 0 } . Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 3 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion How to approximate the probability of reaching B ? Approximation procedure (for a given ǫ > 0) 1 We define � p Yes = Prob M s ( ♦ ≤ n B ) n s ( ♦ ≤ n � p No = Prob M B ) . n For all n , p Yes ≤ Prob M s ( ♦ B ) ≤ 1 − p No n . n We stop when (1 − p No n ) − p Yes < ǫ . n 1 Iyer and Narasimha, “Probabilistic Lossy Channel Systems”, 1997. Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 4 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Example a � p Yes = 0, p No n = 0 c, 1 = 0, 1 1 0 0 2 b b, 1 d, 1 � p Yes = 0, p No = 1 1 1 2 , n = 1 2 2 1 1 2 2 c a, 1 c, 1 � p Yes = 1 4 , p No = 1 1 2 , n = 2 2 2 2 4 4 d b, 1 d, 1 � p Yes = 1 4 , p No = 1 2 + 1 8 = 5 n = 3 8 . 1 3 3 8 8 Target: { a } · · · ⇒ � { a } = { d } . = � 1 4 ≤ Prob M c ( ♦ { a } ) ≤ 1 − 5 8 = 3 8 . � Always terminates? Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 5 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Counterexample: diverging random walk The procedure does not terminate for this infinite Markov chain: 2 2 1 M 3 3 · · · s 0 s 1 s 2 1 1 1 3 3 3 ⇒ � Initial state: s 1 , target state: B = { s 0 } = B = ∅ . For all n , s 1 ( ♦ B ) = 1 • p Yes = Prob M s 1 ( ♦ ≤ n B ) ≤ Prob M 2 . n s 1 ( ♦ ≤ n � • p No = Prob M B ) = 0. n ≥ 1 � For all n , (1 − p No n ) − p Yes 2 . . . n Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 6 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Decisiveness Let M = ( S , P ) be a countable Markov chain, B ⊆ S . Decisiveness 2 s ( ♦ B ∨ ♦ � M is decisive w.r.t. B ⊆ S if for all s ∈ S , Prob M B ) = 1. Theorem 2 If M is decisive w.r.t. B , then the approximation procedure is correct and terminates . • The diverging random walk is not decisive w.r.t. B = { s 0 } . • Decisiveness also allows for a procedure to verify almost-sure reachability . 2 Abdulla, Ben Henda, and Mayr, “Decisive Markov Chains”, 2007. Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 7 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Hybrid systems ℓ 3 ℓ 1 ℓ 2 y y y y ≤ − 1 y ≥ 1 x , y := 0 x x x , y ∈ [ − 1, 1] x • ( L , E ) is a finite graph . • A number n of continuous variables � states of the system are in L × R n � uncountable ! • For each ℓ ∈ L , γ ℓ : R n × R + → R n is a continuous dynamics . • For each edge e ∈ E , G ( e ) ⊆ R n is a guard . • For each edge e ∈ E , R ( e ) : R n → 2 R n is a reset map . Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 8 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Transitions of hybrid systems States: L × R n (discrete location × value of the continuous variables). ℓ 3 ℓ 1 ℓ 2 τ y ≤ − 1 y ≥ 1 y ≥ 1 s ′ x , y := 0 x , y ∈ [ − 1, 1] x , y ∈ [ − 1, 1] s A transition combines a continuous evolution and a discrete transition . Example: initial state is s = ( ℓ 1 , (2 , 0)); • we stay in ℓ 1 for some time τ ≥ 0; • we take an edge whose guard is satisfied; • we take a value among the possible resets , e.g. s ′ = ( ℓ 2 , ( 1 2 , 1 2 )). Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 9 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion We replace the nondeterminism of hybrid systems with probability distributions on the: • waiting time from a given state; • edge choice; • choice of a reset value. � Stochastic hybrid systems ( SHSs ) Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 10 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Undecidability Undecidability of reachability for SHSs Given an SHS H , an initial distribution µ on the states of H and a target set B ⊆ L × R n , the reachability problems • Prob H µ ( ♦ B ) = 1? • Prob H µ ( ♦ B ) = 0? • is a value ǫ -close to Prob H µ ( ♦ B )? are undecidable . � inspired from an undecidability proof for hybrid systems. 3 Goal Find a setting in which reachability is decidable. 3 Henzinger et al., “What’s Decidable about Hybrid Automata?”, 1998. Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 11 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Reachability problems in stochastic systems To deal with an uncountable number of states � “ finite abstraction ”. Abstraction of a stochastic hybrid system · · · · · · · · · α p ′ = 1 q ′ = 1 p > 0 · · · q > 0 · · · · · · · · · T 1 T 2 • Abstraction whenever p > 0 ⇔ q > 0. • Sound abstraction whenever Prob T 2 ( ♦ B ) = 1 = ⇒ Prob T 1 ( ♦ α − 1 ( B )) = 1 . Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 12 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Decidable classes for reachability Hybrid systems: existence of a finite time-abstract bisimulation • Timed automata 4 (˙ x = 1 , x := 0; region graph); • Initialized rectangular hybrid systems; 5 • O-minimal hybrid systems 6 (rich dynamics, all variables have to be reset at every discrete transition). SHSs: existence of a finite and sound abstraction • Single-clock stochastic timed automata; 7 • Reactive stochastic timed automata. 7 � Proof of soundness: finite abstraction + decisiveness . 4 Alur and Dill, “Automata For Modeling Real-Time Systems”, 1990. 5 Henzinger et al., “What’s Decidable about Hybrid Automata?”, 1998. 6 Lafferriere, Pappas, and Sastry, “O-Minimal Hybrid Systems”, 2000. 7 Bertrand et al., “When are stochastic transition systems tameable?”, 2018. Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 13 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Plan to make reachability decidable: strong resets We restrict our focus to SHSs with strong resets . 8 Strong reset = reset that does not depend on the value of the variables. Example: x follows a uniform dist. in [ x − 1 , x + 1] is not a strong reset. x follows a uniform distribution in [ − 1 , 1] is a strong reset. x x − 1 − 2 2 1 x ∼ U ( − 1, 1) 8 Lafferriere, Pappas, and Sastry, “O-Minimal Hybrid Systems”, 2000. Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 14 / 16
Outline Stochastic systems (Stochastic) hybrid systems Conclusion Consequences of strong resets Proposition If an SHS has (at least) one strong reset per cycle of the discrete graph, it • has a finite abstraction ; • is decisive w.r.t. any set of states. ⇒ finite abstraction { = sound and finite strong resets + abstraction ⇒ = decisiveness � Reachability is decidable when the abstraction is computable! Reachability in Stochastic Hybrid Systems Bouyer, Brihaye, Randour, Rivière, Vandenhove 15 / 16
Recommend
More recommend
