Probabilisti tic Model Checking and Contr troller Synth thesis - PowerPoint PPT Presentation

Probabilisti tic Model Checking 
 and Contr troller Synth thesis 
 
 Dave Parker 
 
 University of Birmingham AVACS Autumn School, October 2015

Overview • Probabilistic model checking − verification vs. strategy/controller synthesis − Markov decision processes (MDPs) − example: robot navigation • Multi-objective probabilistic model checking − examples: power management/team-formation • Stochastic (multi-player) games − example: energy management • Permissive controller synthesis 2

Motivation • Verifying probabilistic systems… − unreliable or unpredictable behaviour • failures of physical components • message loss in wireless communication • unreliable sensors/actuators − randomisation in algorithms/protocols • random back-off in communication protocols • random routing to reduce flooding or provide anonymity • We need to verify quantitative system properties − “the probability of the airbag failing to deploy 
 within 0.02 seconds of being triggered is at most 0.001” − not just correctness: reliability, timeliness, performance, … − not just verification: correctness by construction 3

Probabilistic model checking • Construction and analysis of probabilistic models − state-transition systems labelled with probabilities 
 (e.g. Markov chains, Markov decision processes) − from a description in a high-level modelling language 0.4 0.5 0.1 • Properties expressed in temporal logic, e.g. PCTL: − trigger → P ≥ 0.999 [ F ≤ 20 deploy ] − “the probability of the airbag deploying within 
 20ms of being triggered is at at least 0.999” − properties checked against models using 
 exhaustive search and numerical computation 4

Probabilistic model checking • Many types of probabilistic models supported • Wide range of quantitative properties, expressible in 
 temporal logic (probabilities, timing, costs, rewards, …) • Often focus on numerical results (probabilities etc.) − analyse trends, look for system flaws, anomalies • P ≤ 0.1 [ F fail ] – “the probability of a failure occurring is at most 0.1” • P =? [ F fail ] – “what is the probability of a failure occurring?” 6

Probabilistic model checking • Many types of probabilistic models supported • Wide range of quantitative properties, expressible in 
 temporal logic (probabilities, timing, costs, rewards, …) • Often focus on numerical results (probabilities etc.) − analyse trends, look for system flaws, anomalies • Provides "exact" numerical results/guarantees − compared to, for example, simulation • Combines numerical & exhaustive analysis − especially useful for nondeterministic models • Fully automated, tools available, widely applicable − network/communication protocols, security, biology, 
 robotics & planning, power management, … 7

Markov decision processes (MDPs) • Markov decision processes (MDPs) − widely used also in: AI, planning, optimal control, … − model nondeterministic as well as probabilistic behaviour {succ} s 2 {init} a 1 a 0.9 1 s 0 s 1 c 1 0.1 0.7 a s 3 b 0.3 {err} • Nondeterminism for: − control: decisions made by a controller or scheduler − adversarial behaviour of the environment − concurrency/scheduling: interleavings of parallel components − abstraction, or under-specification, of unknown behaviour 9

Strategies • A strategy (or “policy”, “scheduler”, “adversary”) − is a resolution of nondeterminism, based on history − is (formally) a mapping σ from finite paths to distributions − induces an (infinite-state) discrete-time Markov chain {succ} s 2 {init} a a 1 0.9 1 s 0 s 1 c 1 0.1 0.7 a s 3 b 0.3 {err} • Classes of strategies: − randomisation: deterministic or randomised − memory: memoryless, finite-memory, or infinite-memory 10

Example strategy • Strategy σ which picks b then c in s 1 {succ} − σ is finite-memory 
 s 2 {init} a a 1 0.9 and deterministic 1 s 0 s 1 c 1 0.1 0.7 a s 3 b 0.3 • Fragment of induced Markov chain: {err} s 0 s 1 s 0 s 1 s 2 0.9 1 s 0 s 1 s 0 s 0 s 1 s 0 s 1 0.7 s 0 s 1 s 0 s 1 s 3 0.1 1 s 0 s 1 s 0 1 s 0 s 1 s 1 s 2 0.9 s 0 s 1 s 1 s 2 s 2 0.3 s 0 s 1 s 1 s 0 s 1 s 1 s 3 s 0 s 1 s 1 s 3 s 3 0.1 1 11

Verification vs. Strategy synthesis {succ} • 1. Verification s 2 {init} a − quantify over all possible 
 a 1 0.9 1 strategies (i.e. best/worst-case) s 0 s 1 c 1 − P ≤ 0.1 [ F err ] : “the probability of an 
 0.1 0.7 a s 3 b error occurring is ≤ 0.1 for all strategies” 0.3 {err} − applications: randomised communication 
 protocols, randomised distributed algorithms, security, … • 2. Strategy synthesis − generation of "correct-by-construction" controllers − P ≤ 0.1 [ F err ] : "does there exist a strategy for which the probability of an error occurring is ≤ 0.1?” − applications: robotics, power management, security, … • Two dual problems; same underlying computation: − compute optimal (minimum or maximum) values 12

Running example • Example MDP − robot moving through terrain divided in to 3 x 2 grid {goal 2 } {hazard} 0.4 0.6 east s 0 s 1 s 2 east 0.1 stuck south south 0.5 0.1 0.8 0.5 0.9 north 0.1 east stuck 0.6 west s 3 s 5 s 4 west {goal 2 } 0.4 {goal 1 } 13

Example - Reachability {goal 2 } {hazard} Verify: P ≤ 0.6 [ F goal 1 ] 0.4 0.6 east or s 0 s 1 s 2 east 0.1 Synthesise for: P ≥ 0.4 [ F goal 1 ] stuck south south 0.5 0.1 ⇓ 0.8 0.5 0.9 Compute: P max=? [ F goal 1 ] north 0.1 east stuck 0.6 west s 3 s 5 s 4 Optimal strategies: 
 west memoryless and deterministic {goal 2 } 0.4 {goal 1 } Computation: 
 graph analysis + numerical soln. 
 (linear programming, value 
 iteration, policy iteration) 14

Example - Reachability {goal 2 } {hazard} Verify: P ≤ 0.6 [ F goal 1 ] 0.4 0.6 east or s 0 s 1 s 2 east 0.1 Synthesise for: P ≥ 0.4 [ F goal 1 ] stuck south south 0.5 0.1 ⇓ 0.8 0.5 0.9 Compute: P max=? [ F goal 1 ] = 0.5 north 0.1 east stuck 0.6 west s 3 s 5 s 4 Optimal strategies: 
 west memoryless and deterministic {goal 2 } 0.4 {goal 1 } Computation: 
 x 1 graph analysis + numerical soln. 
 1 (linear programming, value 
 x 0 ≥ x 1 iteration, policy iteration) (east) min x 1 ≥ 0.5 (south) x 0 0 15 2/3 0 1

Example - Reachability {goal 2 } {hazard} Verify: P ≤ 0.6 [ F goal 1 ] 0.4 0.6 east or s 0 s 1 s 2 east 0.1 Synthesise for: P ≥ 0.4 [ F goal 1 ] stuck south south 0.5 0.1 ⇓ 0.8 0.5 0.9 Compute: P max=? [ F goal 1 ] = 0.5 north 0.1 east stuck 0.6 west s 3 s 5 s 4 Optimal strategies: 
 west memoryless and deterministic {goal 2 } 0.4 {goal 1 } Computation: 
 graph analysis + numerical soln. 
 Optimal strategy: (linear programming, value 
 s 0 : east iteration, policy iteration) s 1 : south s 2 : - s 3 : - s 4 : east s 5 : - 16

Linear temporal logic (LTL) • Probabilistic LTL (multiple temporal operators) − e.g. P max=? [ (G¬hazard) ∧ (GF goal 1 ) ] – "maximum probability of avoiding hazard and visiting goal 1 infinitely often?" − e.g. P max=? [ ¬zone 3 U (zone 1 ∧ F zone 4 ) ] – "max. probability of patrolling zones 1 then 4, without passing through 3". Det. Buchi automaton A ψ • Probabilistic model checking for ψ = G¬h ∧ GF g 1 − convert LTL formula ψ to 
 deterministic automaton A ψ
 g 1 ∧ ¬h (Buchi, Rabin, finite, …) q 0 q 1 − build/solve product MDP M ⊗ A ψ ¬g 1 ∧ ¬h g 1 ∧ ¬h − reduces to reachability problem h h ¬g 1 ∧ ¬h q 2 − optimal strategies are: true • deterministic • finite-memory 17

Example: Product MDP construction M {goal 2 } A ψ ψ = G¬h ∧ GF g 1 {hazard} 0.4 0.6 east s 0 s 1 s 2 east g 1 ∧ ¬h 0.1 stuck south q 0 south q 1 0.5 0.1 0.8 0.5 0.9 ¬g 1 ∧ ¬h g 1 ∧ ¬h h north h ¬g 1 ∧ ¬h 0.1 east stuck q 2 0.6 west s 3 s 4 s 5 true west {goal 2 } 0.4 {goal 1 } M ⊗ A ψ {goal 2 } 0.4 {hazard} east 0.6 s 0 q 0 s 2 q 0 s 1 q 2 s 2 q 2 east {goal 2 } 0.1 stuck south stuck south 0.5 0.1 0.8 0.9 0.5 0.9 north north 0.1 0.1 east stuck east stuck 0.6 west 0.6 west s 4 q 0 s 3 q 0 s 5 q 1 s 3 q 0 s 4 q 2 s 5 q 2 west 18 west {goal 2 } 0.4 {goal 2 } 0.4 {goal 1 } {goal 1 }

Example: Product MDP construction M {goal 2 } A ψ ψ = G¬h ∧ GF g 1 {hazard} 0.4 0.6 east s 0 s 1 s 2 east g 1 ∧ ¬h 0.1 stuck south q 0 south q 1 0.5 0.1 0.8 0.5 0.9 ¬g 1 ∧ ¬h g 1 ∧ ¬h h north h ¬g 1 ∧ ¬h 0.1 east stuck q 2 0.6 west s 3 s 4 s 5 true west {goal 2 } 0.4 {goal 1 } M ⊗ A ψ {goal 2 } 0.4 {hazard} east 0.6 s 0 q 0 s 2 q 0 s 1 q 2 s 2 q 2 east {goal 2 } 0.1 stuck south stuck south 0.5 0.1 0.8 0.9 0.5 0.9 north north 0.1 0.1 east stuck east stuck 0.6 west 0.6 west s 4 q 0 s 3 q 0 s 5 q 1 s 3 q 2 s 4 q 2 s 5 q 2 west 19 west {goal 2 } 0.4 {goal 2 } 0.4 {goal 1 } {goal 1 }