P OPULATION CTMC A population model is thus given by a tuple X ( N ) - PowerPoint PPT Presentation

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 21 / 123 E XAMPLE : P2P NETWORK EPIDEMICS patch_low suscept. patched loss patch_high ext_inf infect patch_low activate infected infected infect inactive active deactivate r ( N ) patch_s : R patch _ s = { s → p } , patch _ s = k low X s ; r ( N ) patch_d : R patch _ d = { d → p } , patch _ d = k low X d ; r ( N ) patch_i : R patch _ i = { i → p } , patch _ i = k high X i ; r ( N ) loss : R loss = { p → s } , loss = k l X p ;

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 22 / 123 P OPULATION CTMC A population model is thus given by a tuple X ( N ) = ( X ( N ) , T ( N ) , x ( N ) ) , where 0 X ( N ) are the collective variables; T ( N ) are the collective transitions; x ( N ) is the initial state. 0 S TATE S PACE S ( N ) = { x ∈ N n | � x i = N } CTMC INFINITESIMAL GENERATOR Q = ( q x , x ′ ) { r τ ( x ) | τ ∈ T , x ′ = x + v τ } . � q x , x ′ =

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 23 / 123 E XAMPLE : CLIENT SERVER INTERACTION S rq C rq request ready recover logging request think S l S p C rc C t timeout request recover think log process timeout reply reply process timeout wait reply C w S rp SERVER CLIENT

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 24 / 123 E XAMPLE : CLIENT SERVER INTERACTION V ARIABLES 4 variables for the client states: C rq , C w , C rc , C t . 4 variables for the server states: S rq , S p , S rp , S l . T RANSITIONS There are 7 transition in totals. request: C rq → C w , S rq → S p ; kr · min ( C rq , S rq ) reply: C w → C t , S rp → S l ; min ( k w C w , k rp S rp ) timeout: C w → C rc ; k to C w . . .

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 26 / 123 F LUID A PPROXIMATION It applies to population CTMC models with large population size N (studies the limit as N → ∞ ) It applies to population densities (normalisation step), under suitable scaling of rate functions. It is a functional version of the law of large numbers: in any finite time horizon, the trajectories of the PCTMC converge to a deterministic trajectory, solution of the fluid ODE.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 27 / 123 A N INTUITION As population increases, we observe more events each having a smaller impact on the population density vector. X time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 28 / 123 A N INTUITION As population increases, we observe more events each having a smaller impact on the population density vector. X time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 29 / 123 N ORMALIZATION X ( N ) = (ˆ x ( N ) The normalized model ˆ X , ˆ T ( N ) , ˆ ) associated with 0 X ( N ) = ( X , T ( N ) , x ( N ) ) is defined by: 0 Variables: ˆ X = X N x ( N ) x ( N ) Initial conditions: ˆ = 0 0 N r ( N ) (ˆ τ = ( R τ , ˆ X )) from τ ∈ T ( N ) : Normalized transition ˆ τ � X r ( N ) = r ( N ) rate ˆ � ( X ) . τ τ N update vector 1 N v τ . We assume to have a sequence of (normalised) models ˆ X ( N ) , N > 0, that differ only in the total population size. E XAMPLE We will consider the normalised P2P network epidemics model, for an increasing number of netwkr nodes.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 30 / 123 S CALING ASSUMPTIONS E ⊂ R n is a open (or compact) set containing the state space of each ˆ X ( N ) ( t ) for each N . As here the population remains constant, it can be taken as the unit simplex in R n : { x ∈ [ 0 , 1 ] n | � i x i = 1 } . r ( N ) 1 N ˆ is required to converge uniformly to a locally Lipschitz τ continuous and locally bounded function f τ : � � 1 r ( N ) � N ˆ � sup ( x ) − f τ ( x ) � → 0 . � τ � x ∈ E � r ( N ) If 1 N ˆ = f τ does not depend on N , the rate satisfies the density τ dependence condition. The following theorem works also under less restrictive assumptions (e.g. random increments with bounded variance and average).

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 31 / 123 D RIFT AND L IMIT V ECTOR F IELD D RIFT The drift or mean increment at level N is v τ F ( N ) ( x ) = � r ( N ) N ˆ ( x ) τ τ ∈T By the scaling assumptions, F ( N ) converges uniformly to F , the limit vector field (locally bounded and Lipschitz continuous): � F ( x ) = v τ f τ ( x ) . τ ∈T T HE FLUID ODE IS d x ( t ) = F ( x ( t )) dt

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 32 / 123 C ONVERGENCE TO THE F LUID ODE T HEOREM (K URTZ 1970) x ( N ) If ˆ → ˆ x 0 ∈ E in probability, then for any finite time horizon 0 T < ∞ , it holds that: � � || ˆ X ( N ) ( t ) − x ( t ) || > ε sup → 0 . P 0 ≤ t ≤ T T HE MOMENT CLOSURE POINT OF VIEW Alternatively, the fluid ODE can be seen as a (first order) approximation of the ODE for the average of the PCTMC.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 33 / 123 A LOOK AT K. THEOREM PROOF FOR DENSITY DEPENDENT RATES ODE SOLUTION , INTEGRAL FORM � t x ( t ) = x ( 0 ) + F ( x ( s )) ds 0 P ERTURBED ODE REPRESENTATION OF A CTMC � t X ( N ) ( t ) = ˆ ˆ X ( N ) ( 0 ) + F (ˆ X ( N ) ( s )) ds + M ( N ) ( t ) 0 M ( N ) ( t ) is a stochastic process, in particular a martingale, and by applying some martingale inequality (e.g. Doob’s), one has that � M ( N ) ( s ) � → 0 as N → ∞ ε N = sup s ≤ t The theorem then follows as for proving uniqueness of solutions for Lipschitz vector fields (Grönwall inequality).

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 34 / 123 E XAMPLE : P2P NETWORK EPIDEMICS N ORMALISED MODEL patch_low suscept. patched loss ext_inf patch_high infect patch_low activate infected infected infect inactive active deactivate r ( N ) N = Nk ext ˆ v ext _ inf = 1 X s ˆ ext_inf : N ( − 1 , 1 , 0 , 0 ) , ext _ inf = Nk ext X s ; r ( N ) N = Nk inf ˆ X s ˆ v infect = 1 X s X i ˆ infect : N ( − 1 , 1 , 0 , 0 ) , infect = Nk inf X i ; N r ( N ) act = Nk act ˆ v act = 1 ˆ activate : N ( 0 , − 1 , 1 , 0 ) , X d ;

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 35 / 123 P2P NETWORK EPIDEMICS : FLUID EQUATIONS  dx s ( t ) = − k ext x s − k inf x s x i − k low x s + k loss x p   dt     dx d ( t )   = k ext x s + k inf x s x i − k act x d − k low x d + k deact x i    dt dx i ( t )  = k act x d − k deact x i − k high x i   dt     dx p ( t )   = k low x s + k low x d + k high x i − k loss x p   dt

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 36 / 123 P2P NETWORK EPIDEMICS : FLUID AT WORK CTMC N=100 1.0 ODE 0.8 probability s 0.6 ● d ● i p 0.4 ● ● ● 0.2 ● ● ● ● ● ● ● ● ● ● ● 0.0 ● 0 20 40 60 80 100 120 time N = 100

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 37 / 123 P2P NETWORK EPIDEMICS : FLUID AT WORK CTMC N=1000 1.0 ODE 0.8 probability s 0.6 d ● i ● ● p 0.4 ● ● 0.2 ● ● ● ● ● ● ● ● ● ● ● 0.0 ● 0 20 40 60 80 100 120 time N = 1000

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 38 / 123 S TEADY STATE BEHAVIOUR Kurtz theorem in general cannot be extended to convergence of the steady state. The problem is for instance with multi-stable fluid ODEs (more than one attracting equilibrium): in this case, in the long run the CTMC will always keep jumping between these different equilibria, although it will spend a long time in each attractor. Kurtz theorem holds also for steady state distributions only if the fluid ODE has a unique globally attracting steady state. L. Bortolussi, J. Hillston, D. Latella, M. Massink. Continuous Approximation of Collective Systems Behaviour: a Tutorial. Performance Evaluation, 2013.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 39 / 123 S INGLE A GENT A SYMPTOTIC B EHAVIOUR Focus on single individuals Y ( N ) . h Fix h and let Z ( N ) = Y ( N ) be the single-agent stochastic h process with state space S (not necessarily Markov). Let Q ( N ) ( x ) be defined by P { Y ( N ) ( t + dt ) = j | Y ( N ) ( t ) = i , ˆ X ( N ) ( t ) = x } = q ( N ) i , j ( x ) dt , h h with Q ( N ) ( x ) → Q ( x ) . Let z ( t ) be the time inhomogeneous-CTMC on S with infinitesimal generator Q ( t ) = Q ( x ( t )) , x ( t ) fluid limit. T HEOREM (F AST SIMULATION THEOREM ) For any T < ∞ , P { Z ( N ) ( t ) � = z ( t ) , t ≤ T } → 0 . R. Darling, J. Norris. Differential equation approximations for Markov chains. Probability Surveys , 2008.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 40 / 123 P2P N ETWORK EPIDEMICS S INGLE NODE Y ( N ) ∈ { s , d , i , p } R ATES OF Z ( N ) r ( N ) k ext X ( N ) 1 ext _ inf ( X ( N ) ) = 1 ext_inf : = k ext s X ( N ) X ( N ) s s ( N ) r ( N ) N k inf X ( N ) = k inf ˆ 1 infect ( X ( N ) ) = 1 infect : X i X ( N ) i s R ATES OF z ext_inf : k ext infect : k inf x i

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 41 / 123 P2P N ETWORK E PIDEMICS The single agent infinitesimal generator is then Q ( N ) ( x ) = Q ( x ) , giving the following time dependent Q -matrix Q ( x ( t )) , where x ( t ) is the solution of the fluid equations.  − k ext − k inf x i ( t ) − k low k ext + k inf x i ( t ) 0 k low  0 − k act − k low k act k low     0 k deact − k deact − k high k high   k loss 0 0 − k loss Transient probabilities for the fluid approximation of the single agent can be computed by solving the forward Kolmogorov equations d Π( 0 , t ) = Π( 0 , t ) Q ( t ) . dt

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 42 / 123 P2P NETWORK EPIDEMICS : TRANSIENT PROBABILITIES CTMC N=100 1.0 ODE 0.8 probability s 0.6 d ● ● i p 0.4 ● ● ● 0.2 ● ● ● ● ● ● ● ● ● ● ● 0.0 ● 0 20 40 60 80 100 120 time N = 100

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 43 / 123 P2P NETWORK EPIDEMICS : TRANSIENT PROBABILITIES CTMC N=1000 1.0 ODE 0.8 probability s 0.6 d ● i ● ● p 0.4 ● ● 0.2 ● ● ● ● ● ● ● ● ● ● ● 0.0 ● 0 20 40 60 80 100 120 time N = 1000

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 44 / 123 C LIENT S ERVER EXAMPLE S INGLE CLIENT Y ( N ) ∈ { rq , w , t , rc } R ATES OF Z ( N ) rq k r min ( C ( N ) rq , S ( N ) 1 request: rq ) C ( N ) min ( k w C ( N ) w , k rp S ( N ) 1 reply: rp ) C ( N ) w timeout: k to ; recover: k rc R ATES OF z request: k r min ( 1 , s rq ( t ) c rq ( t ) ) s rp ( t ) reply: min ( k w , k rp c w ( t ) ) timeout: k to ; recover: k rc

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 45 / 123 C LIENT -S ERVER : TRANSIENT PROBABILITIES Transient probability RQ Transient probability T 0.30 CTMC N = 15 (10000 runs) 1.0 CTMC N = 150 (10000 runs) fluid CTMC 0.8 0.20 probability probability 0.6 0.4 0.10 0.2 CTMC N = 15 (10000 runs) CTMC N = 150 (10000 runs) 0.0 fluid CTMC 0.00 0 100 200 300 400 500 0 100 200 300 400 500 time time request think Transient probability RC Transient probability W 0.006 0.6 CTMC N = 15 (10000 runs) CTMC N = 15 (10000 runs) CTMC N = 150 (10000 runs) CTMC N = 150 (10000 runs) 0.5 fluid CTMC fluid CTMC 0.004 0.4 probability probability 0.3 0.002 0.2 0.1 0.000 0.0 0 100 200 300 400 500 0 100 200 300 400 500 time time recover wait

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 47 / 123 I NDIVIDUAL PROPERTIES We are interested in the behaviour of a (random) individual. We will specify such a behaviour in Continuous Stochastic Logic (CSL). Other possibilities include DFA, DTA, LTL, MiTL. P2P N ETWORK E PIDEMICS E XAMPLE What is the probability of a node being infected within T units of time? Is the probability of a single node remaining infected for T units of time smaller than p 1 ? Is the probability of a node being patched before getting infected larger than p 2 ? What is the probability of being patched within time T 1 , and then remaining uninfected with probability at least p 3 for T 2 units of time?

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 48 / 123 C OLLECTIVE PROPERTIES We will concentrate on collective properties of the form: ”What is the probability that a given fraction of individuals satisfies the local property φ (by time T )”? P2P N ETWORK E PIDEMICS E XAMPLE What is the probability of at most one tenth of nodes being infected within T units of time? Is the probability of at least one third of nodes remaining infected for T units of time smaller than p 1 ? Is the probability of at least half of nodes being patched before getting infected larger than p 2 ?

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 49 / 123 (T IME -B OUNDED ) C ONTINUOUS S TOCHASTIC L OGIC S YNTAX ⊳ p ( X [ T 1 , T 2 ] φ ) | P ⊲ ⊳ p ( φ 1 U [ T 1 , T 2 ] φ 2 ) φ = a | φ 1 ∧ φ 2 | ¬ φ | P ⊲ a is an atomic proposition; φ 1 ∧ φ 2 and ¬ φ are the usual boolean connectives; ⊳ p ( X [ T 1 , T 2 ] φ ) is the next state temporal modality. P ⊲ ⊳ p ( φ 1 U [ T 1 , T 2 ] φ 2 ) is the until temporal modality. P ⊲ D ERIVED MODALITIES E VENTUALLY : F [ 0 , T ] φ ≡ true U [ 0 , T ] φ A LWAYS : G [ 0 , T ] φ ≡ ¬ F [ 0 , T ] ¬ φ

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 50 / 123 CSL - RESTRICTIONS S YNTAX ⊳ p ( X [ T 1 , T 2 ] φ ) | P ⊲ ⊳ p ( φ 1 U [ T 1 , T 2 ] φ 2 ) φ = a | φ 1 ∧ φ 2 | ¬ φ | P ⊲ We do not consider timed-unbounded operators: 0 ≤ T 1 , T 2 < ∞ ; We do not consider steady state probabilities; We do not consider rewards. Rewards can be easily added. Time unbounded and steady state properties are more problematic: Kurtz theorem works only for time-bounded horizons.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 51 / 123 CSL - NOTATION We will interpret CSL formulae on a generic stochastic process Z ( t ) on S , such that all relevant sets of paths (i.e. those satisfying until or next formulae) are measurable. P ATHS A path σ of Z ( t ) is a sequence t 0 t 1 σ = s 0 → s 1 → . . . , with non null probability of jumping from s i to s i + 1 , for each i ; N OTATION σ @ t is the state of σ at time t ; σ [ i ] is the i-th state of σ ; t σ [ i ] is the time of the i -th jump in σ ;

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 52 / 123 CSL- SEMANTICS S TATE FORMULAE s , t 0 | = a if and only if a ∈ L ( s ) ; s , t 0 | = ¬ φ if and only if s , t 0 �| = φ ; s , t 0 | = φ 1 ∧ φ 2 if and only if s , t 0 | = φ 1 and s , t 0 | = φ 2 ; s , t 0 | = P ⊲ ⊳ p ( ψ ) if and only if P { σ | σ, t 0 | = ψ } ⊲ ⊳ p . P ATH FORMULAE = X [ T 1 , T 2 ] φ if and only if t σ [ 1 ] ∈ [ T 1 , T 2 ] and σ, t 0 | σ [ 1 ] , t 0 + t σ [ 1 ] | = φ . = φ 1 U [ T 1 , T 2 ] φ 2 if and only if ∃ ¯ σ, t 0 | t ∈ [ t 0 + T 1 , t 0 + T 2 ] s.t. σ @¯ t , ¯ = φ 2 and ∀ t 0 ≤ t < ¯ t | t , σ @ t , t | = φ 1 .

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 53 / 123 E XAMPLE : P2P N ETWORK INFECTION ψ 1 = F [ 0 , T ] a infected (a node is infected within T units of time); φ 1 = P < p 1 ( G [ 0 , T ] a infected ) (the probability of a single node remaining infected for T units of time is smaller than p 1 ); φ 2 = P > p 2 ( ¬ a infected U [ 0 , T ] a patched ) (the probability of a node being patched before getting infected is larger than p 2 ); ψ 2 = F [ 0 , T 1 ] ( a patched ∧ P ≥ p 3 ( G [ 0 , T 2 ] ¬ a infected )) (a node is patched within time T 1 , and then remains not infected with probability at least p 3 for T 2 units of time).

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 54 / 123 T HE IDEA Approximate the behaviour of an agent Z in the system using the time-inhomogeneous Markov chain z . Model check temporal logic formulae on z . O UTLINE OF FOLLOWING TOPICS A model checking algorithm for CSL on time-inhomogeneous CTMC (ICTMC). Investigation of its decidability. Convergence results (asymptotic correctness for large N ).

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 56 / 123 CSL MODEL CHECKING : BASIC IDEAS The model checking algorithm works by processing bottom up the parse tree of a formula. The intuition is that each state formula determines the set of states satisfying it. Once this set has been computed, one can treat the state formula as an atomic proposition. Dealing with atomic propositions and boolean connectives is easy: we just need to explain how to compute the satisfaction probability of path formulae.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 57 / 123 CSL MODEL CHECKING : NEXT STATE OPERATOR P ATH PROBABILITY X [ T 1 , T 2 ] φ We just need to evaluate the probability that, being in a state s , we jump within time [ T 1 , T 2 ] to a state that satisfies φ . We know the set { s ′ | s ′ | = φ } by (inductive) hypothesis. We consider time-homogeneous CTMCs. s ′ ∈S , s ′ � = s q ( s , s ′ ) . The exit rate in state s is q ( s ) = � The rate at which we jump to a φ -state is = φ, s ′ � = s q ( s , s ′ ) . q φ ( s ) = � s ′ | P ROBABILITY DENSITY OF X φ q φ ( s ) q ( s ) q ( s ) exp ( − q ( s ) t ) = q φ ( s ) exp ( − q ( s ) t )

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 58 / 123 CSL MODEL CHECKING : NEXT STATE OPERATOR P ROBABILITY DENSITY OF X φ q φ ( s ) q ( s ) q ( s ) exp ( − q ( s ) t ) = q φ ( s ) exp ( − q ( s ) t ) P ROBABILITY OF X [ T 1 , T 2 ] φ � T 2 P ( s , X [ T 1 , T 2 ] φ ) = q φ ( s ) exp ( − q ( s ) t ) dt T 1 q φ ( s ) = q ( s ) ( exp ( − q ( s ) T 1 ) − exp ( − q ( s ) T 2 )) We then need to solve the inequality P ( s , X [ T 1 , T 2 ] φ ) ⊲ ⊳ p to ⊳ p ( X [ T 1 , T 2 ] φ ) . decide if s satisfies P ⊲ This method requires the CTMC to be time-homogeneous

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 59 / 123 CSL MODEL CHECKING : UNTIL OPERATOR We start by considering the until path formula φ 1 U [ 0 , T ] φ 2 . We need to compute the probability of all paths that remain in a φ 1 -state before entering a φ 2 state before time T . The idea is that if we enter a ¬ φ 1 -state, we should discard the path, while if we enter a φ 2 -state, we are done. We can monitor these two events by “stopping” when they happen, making ¬ φ 1 and φ 2 -states absorbing (i.e. removing outgoing transitions).

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 60 / 123 E XAMPLE Consider the property notinfected U [ 0 , T ] patched . We need to make infected and patched states absorbing. patch_low suscept. patched loss patch_high ext_inf infect patch_low activate infected infected infect inactive active deactivate

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 61 / 123 E XAMPLE Consider the property notinfected U [ 0 , T ] patched . patch_low suscept. patched ext_inf infect infected infected inactive active

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 62 / 123 CSL MODEL CHECKING : UNTIL OPERATOR Let Π be the probability matrix: Π( 0 , T )[ s , s ′ ] gives the probability of being in s ′ at time T , starting in s at time 0. M ODEL CHECKING ALGORITHM FOR φ 1 U [ 0 , T ] φ 2 Make ¬ φ 1 and φ 2 states absorbing 1 Compute the transient probability of the so modified CTMC 2 at time T (using uniformisation or solving Kolmogorov equations): Π ¬ φ 1 ∨ φ 2 ( 0 , T ) , The desired probability is 3 � = φ 1 U [ 0 , T ] φ 2 | σ [ 0 ] = s ) = Π ¬ φ 1 ∨ φ 2 [ s , s ′ ]( 0 , T ) P ( σ | s ′ | = φ 2

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 63 / 123 CSL MODEL CHECKING : φ 1 U [ T 1 , T 2 ] φ 2 We split the problem in two parts: Compute the probability of not entering a ¬ φ 1 in the first T 1 1 units of time, by making ¬ φ 1 states absorbing. Compute the probability of the until formula φ 1 U [ 0 , T 2 − T 1 ] φ 2 2 M ODEL CHECKING ALGORITHM FOR φ 1 U [ T 1 , T 2 ] φ 2 Compute Π ¬ φ 1 ( 0 , T 1 ) by transient analysis; 1 Compute Π ¬ φ 1 ∨ φ 2 ( 0 , T 2 − T 1 ) by transient analysis; 2 = φ 1 U [ T 1 , T 2 ] φ 2 | σ [ 0 ] = s ) is The desired probability P ( σ | 3 � � Π ¬ φ 1 ( 0 , T 1 )[ s , s 1 ]Π ¬ φ 1 ∨ φ 2 [ s 1 , s 2 ]( 0 , T 2 − T 1 ) s 1 | = φ 1 s 2 | = φ 2 The method works only for time-homogeneous CTMCs.

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 65 / 123 CSL MODEL CHECKING FOR ICTMC The fluid limit z of a single agent in a population model is a time-inhomogeneous CTMC. I MPLICATIONS We cannot use the same algorithms sketched before, because we cannot always start transient computations from time 0. Non-nested properties can still be dealt with similarly, the difficulties arises with nested properties.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 66 / 123 CSL MODEL CHECKING FOR ICTMC Consider a ICTMC with state space S and rates Q = Q ( t ) . Focus on a non-nested until formula of the type ⊳ p ( φ 1 U [ 0 , T ] φ 2 ) P ⊲ which can be model checked as customary by solving the following reachability problem: What is the probability of reaching a φ 2 -state within time T without entering a ¬ φ 1 -state? S OLUTION Make ¬ φ 1 ∨ φ 2 -states absorbing, and compute the probability of reaching a goal state at time T (e.g., by solving the Kolmogorov equations or by uniformisation for ICTMC).

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 67 / 123 P2P N ETWORK EPIDEMICS : THE MODEL patch_low suscept. patched loss patch_high ext_inf infect patch_low activate infected infected infect inactive active deactivate

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 68 / 123 P2P N ETWORK EPIDEMICS : F [ 0 , T ] a infected FROM STATE s stat mc N=100 (10000 runs) 1.0 stat mc N=1000 (10000 runs) ● ● ● ● ● ● ● ● fluid mc 0.8 ● ● probability 0.6 ● 0.4 ● 0.2 ● ● 0.0 ● ● ● 0 5 10 15 20 time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 69 / 123 P2P N ETWORK EPIDEMICS : ¬ a infected U [ 0 , T ] a patched FROM STATE s stat mc N=100 (10000 runs) 0.05 stat mc N=1000 (10000 runs) ● fluid mc 0.04 ● ● ● ● ● ● ● ● ● probability ● 0.03 ● ● 0.02 ● ● 0.01 ● 0.00 ● 0 5 10 15 20 time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 70 / 123 N EXT - STATE PROBABILITY P ROBABILITY OF X [ T 1 , T 2 ] φ STARTING AT TIME t 0 � t 0 + T 2 q φ ( s , t ) · e − Λ( t 0 , t )[ s ] dt P next ( t 0 )[ s ] = t 0 + T 1 � t where Λ( t 0 , t )[ s ] = t 0 − q s , s ( τ ) d τ is the cumulative rate. We can reduce the computation of the previous integral to the following initial value problem from t 0 + T 1 to t 0 + T 2 .  d dt P ( t ) = q s , S 0 ( t ) · e − L ( t )    d  dt L ( t ) = − q s , s ( t )   with P ( t 0 + T 1 ) = 0 and L ( t 0 + T 1 ) = Λ( t 0 , t 0 + T 1 ) .

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 71 / 123 P2P N ETWORK EPIDEMICS : X [ 0 , T ] a infected FROM STATE s stat mc N=100 (10000 runs) 1.0 stat mc N=1000 (10000 runs) ● ● ● ● ● ● ● ● fluid mc 0.8 ● ● probability 0.6 ● 0.4 ● 0.2 ● ● ● 0.0 ● ● 0 5 10 15 20 time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 72 / 123 C LIENT -S ERVER : THE MODEL S rq C rq request ready recover logging request think S l S p C rc C t timeout request recover think log process timeout reply reply process timeout wait reply C w S rp SERVER CLIENT

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 73 / 123 C LIENT -S ERVER : P =? ( F ≤ T a timeout ) Pr=?[F<=T timeout] −− 10 clients, 5 servers 1.0 stat mc (10000 runs) fluid mc 0.8 0.6 probability 0.4 0.2 0.0 0 500 1000 1500 2000 2500 3000 time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 74 / 123 C LIENT -S ERVER : P =? ( a request ∨ a wait U ≤ T a timeout ) Pr=?[(request or wait) U<=T timeout] −− 10 clients, 5 servers stat mc (10000 runs) fluid mc 0.10 0.08 probability 0.06 0.04 0.02 0.00 0 20 40 60 80 100 time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 75 / 123 C LIENT -S ERVER : COMPUTATIONAL COST Pr=?[F<=T timeout] −− 10 clients, 5 servers Pr=?[(request or wait) U<=T timeout] −− 10 clients, 5 servers 1.0 stat mc (10000 runs) stat mc (10000 runs) fluid mc fluid mc 0.10 0.8 0.08 0.6 probability probability 0.06 0.4 0.04 0.2 0.02 0.0 0.00 0 500 1000 1500 2000 2500 3000 0 20 40 60 80 100 time time C OMPUTATIONAL COST The cost of analysing the limit fluid system is independent of N . For the client server example (10 clients - 5 servers) it is ∼ 100 times faster than the simulation-based approach (which increases linearly with N ).

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 76 / 123 P2P N ETWORK E PIDEMICS : COMPUTATIONAL COST stat mc N=100 (10000 runs) stat mc N=100 (10000 runs) 1.0 0.05 ● stat mc N=1000 (10000 runs) ● ● ● ● stat mc N=1000 (10000 runs) ● ● ● ● fluid mc fluid mc 0.8 ● 0.04 ● ● ● ● ● ● ● ● ● ● probability probability 0.6 ● 0.03 ● ● ● 0.4 0.02 ● ● ● 0.01 0.2 ● ● ● 0.00 0.0 ● ● ● ● 0 5 10 15 20 0 5 10 15 20 time time C OMPUTATIONAL COST Checked property Fluid MC SMC ( N = 100) SMC ( N = 1000) Kolmogorov Equations ∼ 0 . 1 s ∼ 64 s ∼ 101 s X [ 0 , T ] a infected ∼ 0 . 06 s ∼ 6 s ∼ 24 s ¬ a infected U [ 0 , T ] a patched ∼ 0 . 05 s ∼ 5 s ∼ 20 s

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 78 / 123 CSL MODEL CHECKING FOR ICTMC Consider a ICTMC with state space S and rates Q = Q ( t ) . φ 1 U [ 0 , T ] φ 2 X [ T 1 , T 2 ] φ and Time-homogeneity ⇒ we can run each transient analysis/ integral computation from time t 0 = 0! This is no more true in time-inhomogeneous CTMCs, as the probability of a path formula depends on the time at which we evaluate it. Problems arise when we consider nested until formulae. The truth value of φ in a state s depends on the time t at which we evaluate it.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 79 / 123 T IME - DEPENDENT PROBABILITY OF X [ T 1 , T 2 ] φ P ROBABILITY OF X [ T 1 , T 2 ] φ STARTING AT TIME t 0 � t 0 + T 2 q φ ( s , t ) · e − Λ( t 0 , t )[ s ] dt P next ( t 0 )[ s ] = t 0 + T 1 � t where Λ( t 0 , t )[ s ] = t 0 − q s , s ( τ ) d τ is the cumulative rate. I NTUITION d Compute dt 0 P next ( t 0 )[ s ] Construct an ODE for P next ( t 0 ) and solve the i.v. problem. ⊳ p ( X [ T 1 , T 2 ] φ ) C HECKING P ⊲ Compute the path probability P next ( t 0 )[ s ] of X [ T 1 , T 2 ] φ as a function of t 0 Solve the inequality P next ( t 0 )[ s ] ⊲ ⊳ p

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 80 / 123 P2P N ETWORK E PIDEMICS : X [ 0 , 10 ] a infected 1.1 Prob(p,t0 |=X[0,10] infected) true 1.0 0.9 probability 0.8 0.7 0.6 0.5 T~ 2.26 false 0 2 4 6 8 10 time t 0 varying (Red line: P ≥ 0 . 8 ( X [ 0 , 10 ] a infected ) )

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 81 / 123 T IME -D EPENDENT REACHABILITY PROBABILITY ⊳ p ( φ 1 U [ 0 , T ] φ 2 ) . Assume that the truth of φ 1 and φ 2 Focus on P ⊲ does not depend on time. Let Π( t 1 , t 2 ) = ( π s i , s j ( t 1 , t 2 )) i , j be the probability matrix giving the probability of being in state s j at time t 2 , given that we are in state s i at time t 1 . We consider Π = Π ¬ φ 1 ∨ φ 2 , the probability matrix of the CTMC in which ¬ φ 1 ∨ φ 2 states are made absorbing.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 82 / 123 F ORWARD AND B ACKWARD K OLMOGOROV EQUATIONS The device to compute the time dependent probability of an until formula φ 1 U [ 0 , T ] φ 2 are the Kolmogorov equations for ICTMCs. F ORWARD K OLMOGOROV E QUATION d dt Π( s , t ) = Π( s , t ) Q ( t ) B ACKWARD K OLMOGOROV E QUATION d ds Π( s , t ) = − Q ( s )Π( s , t ) C OMPUTING Π( t , t + T ) , FOR FIXED T We just need to combine the two backward and forward equations by chain rule.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 83 / 123 T IME -D EPENDENT REACHABILITY PROBABILITY 1. C OMPUTE Π( t , t + T ) , FOR t ∈ [ 0 , T f ] Π( t , t + T ) , as a function of t , with initial conditions Π( 0 , T ) , satisfies: d Π( t , t + T ) = Π( t , t + T ) Q ( t + T ) − Q ( t )Π( t , t + T ) d t 2. A DD PROBABILITY FOR GOAL STATES = φ 2 Π ¬ φ 1 ∨ φ 2 ( t , t + T )[ s , s ′ ] . P φ 1 U [ 0 , T ] φ 2 ( s , t ) is equal to � s ′ | 3. C OMPARE WITH THRESHOLD p The truth value T ( φ, s , t ) of formula φ in state s at time t is obtained by solving the inequality P φ 1 U [ 0 , T ] φ 2 ( s , t ) ⊲ ⊳ p . We need to find the zeros of the function P φ 1 U [ 0 , T ] φ 2 ( s , t ) − p .

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 84 / 123 P2P N ETWORK E PIDEMICS : G [ 0 , 10 ] ¬ a infected 0.98 Prob(p,t0 |= G[0,10] not_infected true p,t0 |= P>0.97(G[0,10] not_infected) 0.97 probability 0.96 T~81.8 0.95 false 0 50 100 150 time from state p (patched)

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 85 / 123 C LIENT S ERVER : P =? F ≤ 50 a timeout AS A FUNCTION OF t 0 Pr=?[F<=50 timeout] −− t0 varying −− 10 clients, 5 servers 0.20 ● ● ● ● ● ● ● ● ● ● 0.15 ● ● probability 0.10 0.05 stat mc (10000 runs) 0.00 fluid mc 0 5 10 15 20 25 initial time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 86 / 123 C LIENT -S ERVER : P < 0 . 167 ( F ≤ 50 timeout ) Pr=?[F<=50 timeout] −− t0 varying true 0.20 0.167 0.15 probability 0.10 t ~ 2.1 false 0.05 rq 0.00 truth−value 0 20 40 60 80 100 initial time P < 0 . 167 ( F ≤ 50 timeout ) from state rq of client.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 87 / 123 C OMPUTING THE TIME - DEPENDENT TRUTH IN PRACTICE The equation d Π( t , t + T ) = Π( t , t + T ) Q ( t + T ) − Q ( t )Π( t , t + T ) d t is utterly stiff. Its integration error blows up even for the most accurate Matlab/Octave solvers. 100.000 95.000 90.000 85.000 80.000 75.000 70.000 65.000 60.000 55.000 values 50.000 45.000 40.000 35.000 30.000 25.000 20.000 15.000 10.000 5.000 0 0 5 10 15 20 25 30 35 40 time

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 88 / 123 C OMPUTING THE TIME - DEPENDENT TRUTH IN PRACTICE The equation d Π( t , t + T ) = Π( t , t + T ) Q ( t + T ) − Q ( t )Π( t , t + T ) d t is utterly stiff. Its integration error blows up even for the most accurate Matlab/Octave solvers. time · · · T 0 = 0 T 1 = 1 · T T 2 = 2 · T T k = k · T Practically, we can exploit the semigroup property Π( t , t + T ) = Π( t , T j )Π( T j , t + T ) and solve backward and forward equations separately, looping over j .

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 90 / 123 T IME - DEPENDENT TRUTH When computing the truth value of an until formula, we obtain a time dependent value T ( φ, s , t ) in each state. When we consider nested temporal operators, we need to take this into account. The problem is that in this case the TOPOLOGY OF GOAL AND UNSAFE STATES in the CTMC can CHANGE IN TIME .

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 91 / 123 T IME DEPENDENT TRUTH : F ≤ T φ T ( φ, s , t ) true false t 0 T d At discontinuity times, changes in topology introduce discontinuities in the probability values. B UT ... Discontinuities happen at specific and FIXED time instants. We can solve Kolmogorov equations piecewise!

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 92 / 123 k DISCONTINUITIES T 1 , . . . , T k IN [ t , t + T ] time · · · T 1 T 2 T k T k + 1 t t + T T HE GENERIC CK EQUATION Π( t , t + T ) = Π 1 ( t , T 1 ) ζ ( T 1 )Π 2 ( T 1 , T 2 ) ζ ( T 2 ) · · · ζ ( T k )Π k + 1 ( T k , t + T ) . ζ ( T j ) apply the proper bookkeeping operations to deal with changes in the topology of absorbing states. We can compute Π( t , t + T ) by an ODE obtained by derivation and application of chain rule. In advancing time, when we hit a discontinuity point (from below or above), the structure of the previous equation changes: integration has to be stopped and restarted.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 93 / 123 T HE A LGORITHM ( SKETCHED ) Proceed bottom-up on the parse tree of a formula. ⊳ p ( φ 1 U [ 0 , T ] φ 2 ) , t ) : Case T ( P ⊲ Compute T ( φ 1 , t ) and T ( φ 2 , t ) Let T 1 , . . . , T m be all the discontinuity points of T ( φ 1 , t ) and T ( φ 2 , t ) up to a final time T f . Compute Π( T i , T i + 1 ) for each i Compute Π( 0 , T ) using generalized CK equations Integrate d dt Π( t , t + T ) up to T f . ⊳ p ( φ 1 U [ 0 , T ] φ 2 ) , t ) = Π( t , t + T ) ⊲ Return T ( P ⊲ ⊳ p . The use of Kolmogorov equations is feasible if the state space is small (few dozens of states). This is usually the case for single agent mean field models.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 94 / 123 P2P NETWORK EPIDEMICS : F [ 0 , T ] ( a patched ∧ P ≥ 0 . 97 ( G [ 0 , 10 ] ¬ a infected )) Prob(p,t0 |= F[0,T] (patched AND P>0.97(G[0,10] not_infected)) 1.0 0.8 probability 0.6 0.4 0.2 0.0 T~81.8 0 50 100 150 time from state p (patched)

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 95 / 123 C LIENT -S ERVER : F ≤ T ( P < 0 . 167 ( F ≤ 50 timeout )) F<=t(Pr<0.167[F<=50 timeout]) 1.0 0.8 0.6 probability 0.4 0.2 R(0)=1 W(0)=1 T(0)=1 0.0 A(0)=1 0 10 20 30 40 time

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 97 / 123 D ECIDABILITY D ECIDABILITY We use algorithms to solve ODEs with error guarantee (interval analysis). We need to find zeros of function P ( s , t ) − p (root finding), and guarantee their number to be finite (restrict to piecewise-real analytic functions). To answer the CSL query for main until formulae, we need to know if P ( s , 0 ) ⊲ ⊳ p (zero test). It is not known if root finding and zero test are decidable. p

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 98 / 123 D ECIDABILITY D ECIDABILITY We use algorithms to solve ODEs with error guarantee (interval analysis). We need to find zeros of function P ( s , t ) − p (root finding), and guarantee their number to be finite (restrict to piecewise-real analytic functions). To answer the CSL query for main until formulae, we need to know if P ( s , 0 ) ⊲ ⊳ p (zero test). It is not known if root finding and zero test are decidable. T HEOREM (Q UASI - DECIDABILITY ) Let φ = φ ( p ) be a CSL formula, with constants p = ( p 1 , . . . , p k ) ∈ [ 0 , 1 ] k appearing in until formulae. The CSL model checking for ICTMC problem is decidable for p ∈ E, where E is an open subset of [ 0 , 1 ] k , of measure 1.

I NTRODUCTION F LUID A PPROXIMATION B EHAVIOUR SPECIFICATION MC ICTMC L OCAL 2G LOBAL 99 / 123 C ONVERGENCE OF CSL TRUTH We considered also convergence of CSL properties: are properties that are true in z ( t ) ultimately true in Z ( N ) ( t ) ? Convergence suffers from similar issues as decidability (e.g., non-simple zeros , P ( s , 0 ) = p ). T HEOREM (A SYMPTOTIC CORRECTNESS ) Let φ = φ ( p ) be a CSL formula, with constants p = ( p 1 , . . . , p k ) ∈ [ 0 , 1 ] k appearing in until formulae. Then, for p ∈ E, an open subset of [ 0 , 1 ] k of measure 1, there exists N 0 such that ∀ N ≥ N 0 s , 0 | = Z ( N ) φ ⇔ s , 0 | = z φ.

O UTLINE 1 I NTRODUCTION 2 F LUID A PPROXIMATION Markov population models Fluid approximation theorems 3 B EHAVIOUR SPECIFICATION Individual Properties CSL model checking for time-homogeneous CTMC 4 M ODEL C HECKING CSL FOR ICTMC Model checking non-nested properties Time-dependent probabilities Nested CSL-formulae Theoretical results 5 F ROM I NDIVIDUAL TO C OLLECTIVE B EHAVIOUR From local properties to global properties Central Limit Approximation Examples Conclusions