Overview Motivation 1 Quantitative Automata Models and Model - PowerPoint PPT Presentation

Motivation Overview Motivation 1 Quantitative Automata Models and Model Checking What are discrete-time Markov chains? 2 Reachability probabilities 3 Joost-Pieter Katoen Qualitative reachability and all that 4 RWTH Aachen University Verifying ω -regular properties 5 Software Modeling and Verification Group SFM 2013 Summerschool on Dynamical Systems, Bertinoro, Italy Verifying probabilistic CTL 6 June 18, 2013 Expressiveness of probabilistic CTL 7 Probabilistic bisimulation 8 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 1/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 2/141 Motivation Motivation Probabilities help Simulating a die by a fair coin [Knuth & Yao] ◮ When analysing system performance and dependability ◮ to quantify arrivals, waiting times, time between failure, QoS, ... ◮ When modelling unreliable and unpredictable system behavior ◮ to quantify message loss, processor failure ◮ to quantify unpredictable delays, express soft deadlines, ... ◮ When building protocols for networked embedded systems ◮ randomized algorithms ◮ When problems are undecidable deterministically ◮ repeated reachability of lossy channel systems, . . . Heads = “go left”; tails = “go right”. Does this DTMC model a six-sided die? Joost-Pieter Katoen Quantitative Automata Models and Model Checking 3/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 4/141

Motivation Motivation What is probabilistic model checking? Probabilistic models Nondeterminism Nondeterminism no yes Discrete time discrete-time Markov decision Markov chain (DTMC) process (MDP) Continuous time CTMC CTMDP Some other models: probabilistic variants of (priced) timed automata Joost-Pieter Katoen Quantitative Automata Models and Model Checking 5/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 6/141 Motivation Motivation Properties Probability theory is simple, isn’t it? Logic Monitors Discrete time probabilistic deterministic automata CTL (safety and LTL) In no other branch of mathematics is it so easy to make mistakes Continuous time probabilistic deterministic as in probability theory timed CTL timed automata Henk Tijms, “Understanding Probability” (2004) Core problem: computing (timed) reachability probabilities Joost-Pieter Katoen Quantitative Automata Models and Model Checking 7/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 8/141

What are discrete-time Markov chains? What are discrete-time Markov chains? Overview Geometric distribution Geometric distribution Motivation 1 Let X be a discrete random variable, natural k > 0 and 0 < p � 1. The mass function of a geometric distribution is given by: What are discrete-time Markov chains? 2 Pr { X = k } = ( 1 − p ) k − 1 · p Reachability probabilities 3 p and Var [ X ] = 1 − p We have E [ X ] = 1 and cdf Pr { X � k } = 1 − ( 1 − p ) k . Qualitative reachability and all that 4 p 2 Verifying ω -regular properties 5 Geometric distributions and their cdf’s Verifying probabilistic CTL 6 Expressiveness of probabilistic CTL 7 Probabilistic bisimulation 8 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 9/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 10/141 What are discrete-time Markov chains? What are discrete-time Markov chains? Memoryless property Markov property The conditional probability distribution of future states of a Markov process only depends on the current state and not on its further history. Theorem 1. For any random variable X with a geometric distribution: Markov process A discrete-time stochastic process { X ( t ) | t ∈ T } over state space Pr { X = k + m | X > m } = Pr { X = k } for any m ∈ T , k � 1 { d 0 , d 1 , . . . } is a Markov process if for any t 0 < t 1 < . . . < t n < t n + 1 : This is called the memoryless property, and X is a memoryless r.v.. Pr { X ( t n + 1 ) = d n + 1 | X ( t 0 ) = d 0 , X ( t 1 ) = d 1 , . . . , X ( t n ) = d n } 2. Any discrete random variable which is memoryless is geometrically = distributed. Pr { X ( t n + 1 ) = d n + 1 | X ( t n ) = d n } The distribution of X ( t n + 1 ) , given the values X ( t 0 ) through X ( t n ) , only depends on the current state X ( t n ) . Joost-Pieter Katoen Quantitative Automata Models and Model Checking 11/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 12/141

What are discrete-time Markov chains? What are discrete-time Markov chains? Invariance to time-shifts Discrete-time Markov chain Discrete-time Markov chain Time homogeneity A discrete-time Markov chain (DTMC) is a time-homogeneous Markov Markov process { X ( t ) | t ∈ T } is time-homogeneous iff for any t ′ < t : process with discrete parameter T and discrete state space S . Pr { X ( t ) = d | X ( t ′ ) = d ′ } = Pr { X ( t − t ′ ) = d | X ( 0 ) = d ′ } . Transition probabilities The (one-step) transition probability from s ∈ S to s ′ ∈ S at epoch n ∈ N A time-homogeneous stochastic process is invariant to time shifts. is given by: p ( n ) ( s , s ′ ) = Pr { X n + 1 = s ′ | X n = s } = Pr { X 1 = s ′ | X 0 = s } Discrete-time Markov chain A discrete-time Markov chain (DTMC) is a time-homogeneous Markov where the last equality is due to time-homogeneity. process with discrete parameter T and discrete state space. Since p ( n ) ( · ) = p ( k ) ( · ) , the superscript ( n ) is omitted, and we write p ( · ) . Joost-Pieter Katoen Quantitative Automata Models and Model Checking 13/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 14/141 What are discrete-time Markov chains? What are discrete-time Markov chains? Transition probability matrix DTMCs — A transition system perspective Discrete-time Markov chain Discrete-time Markov chain A discrete-time Markov chain (DTMC) is a time-homogeneous Markov A DTMC D is a tuple ( S , P , ι init , AP , L ) with: process with discrete parameter T and discrete state space S . ◮ S is a countable nonempty set of states ◮ P : S × S → [ 0, 1 ] , transition probability function s.t. � s ′ P ( s , s ′ ) = 1 Transition probability matrix ◮ ι init : S → [ 0, 1 ] , the initial distribution with � ι init ( s ) = 1 Let P be a function with P ( s i , s j ) = p ( s i , s j ) . For finite state space S , s ∈ S function P is called the transition probability matrix of the DTMC with ◮ AP is a set of atomic propositions. state space S . ◮ L : S → 2 AP , the labeling function, assigning to state s , the set L ( s ) of atomic propositions that are valid in s . Properties 1. P is a (right) stochastic matrix, i.e., it is a square matrix, all its Initial states elements are in [ 0, 1 ] , and each row sum equals one. ◮ ι init ( s ) is the probability that DTMC D starts in state s 2. P has an eigenvalue of one, and all its eigenvalues are at most one. ◮ the set { s ∈ S | ι init ( s ) > 0 } are the possible initial states. 3. For all n ∈ N , P n is a stochastic matrix. Joost-Pieter Katoen Quantitative Automata Models and Model Checking 15/141 Joost-Pieter Katoen Quantitative Automata Models and Model Checking 16/141