Verifying Continuous-Time Markov Chains Verifying Continuous-Time Markov Chains Overview Verifying Continuous-Time Markov Chains Negative exponential distributions 1 Lecture 3+4: Continuous-Time Markov Chains What are continuous-time Markov chains? 2 Joost-Pieter Katoen Transient distribution 3 RWTH Aachen University Software Modeling and Verification Group Timed reachability probabilities 4 http://www-i2.informatik.rwth-aachen.de/i2/mvps11/ Verifying continuous stochastic CTL 5 VTSA Summerschool, Liège, Belgium September 21, 2011 Verifying linear real-time properties 6 logoRWTH Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 1/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 2/119 Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains Negative exponential distributions Overview Time in discrete-time Markov chains The advance of time in DTMCs Negative exponential distributions 1 ◮ Time in a DTMC proceeds in discrete steps ◮ Two possible interpretations: What are continuous-time Markov chains? 2 1. accurate model of (discrete) time units ◮ e.g., clock ticks in model of an embedded device 2. time-abstract Transient distribution 3 ◮ no information assumed about the time transitions take ◮ State residence time is geometrically distributed Timed reachability probabilities 4 Continuous-time Markov chains Verifying continuous stochastic CTL 5 ◮ dense model of time Verifying linear real-time properties 6 ◮ transitions can occur at any (real-valued) time instant ◮ state residence time is (negative) exponentially distributed Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 3/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 4/119
Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains Negative exponential distributions Continuous random variables Negative exponential distribution Density of exponential distribution The density of an exponentially distributed r.v. Y with rate λ ∈ R > 0 is: ◮ X is a random variable (r.v., for short) ◮ on a sample space with probability measure Pr f Y ( x ) = λ · e − λ · x for x > 0 and f Y ( x ) = 0 otherwise ◮ assume the set of possible values that X may take is dense ◮ X is continuously distributed if there exists a function f ( x ) such that: The cumulative distribution of r.v. Y with rate λ ∈ R > 0 is: � d � d λ · e − λ · x dx = [ − e − λ · x ] d 0 = 1 − e − λ · d . F x ( d ) = Pr { X � d } = f ( x ) dx for each real number d F Y ( d ) = −∞ 0 � ∞ The rate λ ∈ R > 0 uniquely determines an exponential distribution. where f satisfies: f ( x ) � 0 for all x and f ( x ) dx = 1 −∞ Variance and expectation ◮ F X ( d ) is the (cumulative) probability distribution function ◮ f ( x ) is the probability density function Let r.v. Y be exponentially distributed with rate λ ∈ R > 0 . Then: � ∞ 0 x · λ · e − λ · x dx = 1 ◮ Expectation E [ Y ] = λ � ∞ 0 ( x − E [ X ]) 2 λ · e − λ · x dx = 1 ◮ Variance Var [ Y ] = λ 2 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 5/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 6/119 Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains Negative exponential distributions Exponential pdf and cdf Why exponential distributions? ◮ Are adequate for many real-life phenomena ◮ the time until a radioactive particle decays ◮ the time between successive car accidents ◮ inter-arrival times of jobs, telephone calls in a fixed interval ◮ Are the continuous counterpart of the geometric distribution ◮ Heavily used in physics, performance, and reliability analysis ◮ Can approximate general distributions arbitrarily closely The higher λ , the faster the cdf approaches 1. ◮ Yield a maximal entropy if only the mean is known Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 7/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 8/119
Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains Negative exponential distributions Memoryless property Closure under minimum Theorem 1. For any exponentially distributed random variable X : Pr { X > t + d | X > t } = Pr { X > d } for any t , d ∈ R � 0 . Minimum closure theorem For independent, exponentially distributed random variables X and Y with rates λ , µ ∈ R > 0 , the r.v. min ( X , Y ) is exponentially distributed with rate 2. Any cdf which is memoryless is a negative exponential one. λ + µ , i.e.,: Proof: Pr { min ( X , Y ) � t } = 1 − e − ( λ + µ ) · t for all t ∈ R � 0 . Proof of 1. : Let λ be the rate of X ’s distribution. Then we derive: Pr { X > t + d | X > t } = Pr { X > t + d ∩ X > t } = Pr { X > t + d } Pr { X > t } Pr { X > t } = e − λ · ( t + d ) = e − λ · d = Pr { X > d } . e − λ · t Proof of 2. : By contraposition, using the total law of probability. Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 9/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 10/119 Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains Negative exponential distributions Proof Closure under minimum Let λ ( µ ) be the rate of X ’s ( Y ’s) distribution. Then we derive: Minimum closure theorem for several exponentially distributed r.v. ’s Pr { min ( X , Y ) � t } = Pr X , Y { ( x , y ) ∈ R 2 � 0 | min ( x , y ) � t } For independent, exponentially distributed random variables X 1 , X 2 , . . . , X n � ∞ �� ∞ � I min ( x , y ) � t ( x , y ) · λ e − λ x · µ e − µ y dy with rates λ 1 , λ 2 , . . . , λ n ∈ R > 0 the r.v. min ( X 1 , X 2 , . . . , X n ) is = dx exponentially distributed with rate � 0 < i � n λ i , i.e.,: 0 0 � t � t � ∞ � ∞ λ e − λ x · µ e − µ y dy dx + λ e − λ x · µ e − µ y dx dy = Pr { min ( X 1 , X 2 , . . . , X n ) � t } = 1 − e − � 0 < i � n λ i · t for all t ∈ R � 0 . 0 x 0 y � t � t λ e − λ x · e − µ x dx + e − λ y · µ e − µ y dy = 0 0 � t � t Proof: λ e − ( λ + µ ) x dx + µ e − ( λ + µ ) y dy = Generalization of the proof for the case of two exponential distributions. 0 0 � t ( λ + µ ) · e − ( λ + µ ) z dz = 1 − e − ( λ + µ ) t = 0 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 11/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 12/119
Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains Negative exponential distributions Winning the race with two competitors Proof Let λ ( µ ) be the rate of X ’s ( Y ’s) distribution. Then we derive: Pr { X � Y } = Pr X , Y { ( x , y ) ∈ R 2 � 0 | x � y } �� y � ∞ � The minimum of two exponential distributions λ e − λ x dx µ e − µ y = dy For independent, exponentially distributed random variables X and Y with 0 0 � ∞ rates λ , µ ∈ R > 0 , it holds: µ e − µ y � 1 − e − λ y � = dy 0 λ � ∞ � ∞ Pr { X � Y } = λ + µ. µ e − µ y · e − λ y dy = 1 − µ e − ( µ + λ ) y dy = 1 − 0 0 � ∞ µ ( µ + λ ) e − ( µ + λ ) y dy = 1 − µ + λ · 0 � �� � = 1 µ λ = 1 − µ + λ = µ + λ Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 13/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 14/119 Verifying Continuous-Time Markov Chains Negative exponential distributions Verifying Continuous-Time Markov Chains What are continuous-time Markov chains? Winning the race with many competitors Overview Negative exponential distributions 1 The minimum of several exponentially distributed r.v. ’s For independent, exponentially distributed random variables X 1 , X 2 , . . . , X n What are continuous-time Markov chains? 2 with rates λ 1 , λ 2 , . . . , λ n ∈ R > 0 it holds: Transient distribution 3 λ i Pr { X i = min ( X 1 , . . . , X n ) } = . � n j = 1 λ j Timed reachability probabilities 4 Verifying continuous stochastic CTL 5 Proof: Generalization of the proof for the case of two exponential distributions. Verifying linear real-time properties 6 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 15/119 Joost-Pieter Katoen Verifying Continuous-Time Markov Chains 16/119
Recommend
More recommend
