Mean Field Approximation of Uncertain Stochastic Models Luca - PowerPoint PPT Presentation

Mean Field Approximation of Uncertain Stochastic Models Luca Bortolussi 1 , Nicolas Gast 2 DSN conference 2016, Toulouse, France 1 CNR-Univ Trieste, Italy 2 Inria, France Nicolas Gast – 1 / 30

Why do we need models? Design, prediction, optimization, correctness, etc. Nicolas Gast – 2 / 30

Uncertainties in models: stochastic v.s. non-determinism Stochastic Non-determinism ex: Markov chains ex: ODE, timed-automata Nicolas Gast – 3 / 30

Uncertainties in models: stochastic v.s. non-determinism Stochastic Non-determinism ex: Markov chains ex: ODE, timed-automata Quantitative analysis. Worst case / correctness + can be simulated − Symbolic computation − How to choose parameters? + No problem of parameters Combination of the two Uncertain Markov chains Nicolas Gast – 3 / 30

Building a continuous time Markov chain agents engage in actions at some rate. ( α, θ ) . P d action component dt P ( t ) = P ( t ) . Q type / derivative activity rate (parameter of an exponential distribution) Nicolas Gast – 4 / 30

Building an uncertain continuous time Markov chain agents engage in actions at some rate. ( α, θ ) . P d d action component dt P ( t ) = P ( t ) . Q dt P ( t ) = P ( t ) . Q type / derivative activity rate (parameter of an exponential distribution) d � dt P ( t ) ∈ P ( t ) Q ( θ ) unknown θ ∈ [ θ min , θ max ] θ ∈ [ θ min ,θ max ] Nicolas Gast – 4 / 30

Main problem: the state space grows exponentially We need to keep track P ( X 1 ( t ) = i 1 , . . . , X n ( t ) = i n ) and solve the differential inclusion: d � dt P ( t ) ∈ P ( t ) Q ( θ ) θ ∈ [ θ min ,θ max ] 3 13 ≈ 10 6 states. Is there any hope? Nicolas Gast – 5 / 30

Contributions (and Outline) 1 Some systems simplify when the population grows. ◮ Mean-field approach 2 We can add non-determinism to these models 3 We can build and use numerical algorithms. Nicolas Gast – 6 / 30

Outline Population Processes and Classical Mean Field Methods 1 Uncertain and Imprecise Population Processes 2 Numerical Algorithms and Comparisons 3 Numerical algorithms (transient regime) Steady-state General processor sharing example Conclusion 4 Nicolas Gast – 7 / 30

Outline Population Processes and Classical Mean Field Methods 1 Uncertain and Imprecise Population Processes 2 Numerical Algorithms and Comparisons 3 Numerical algorithms (transient regime) Steady-state General processor sharing example Conclusion 4 Nicolas Gast – 8 / 30

Mean field methods have been used in a multiple contexts ex: model-checking, performance of SSD, load balancing, MAC protocol,... SPAA 98 Analyses of Load Stealing Models Based on Differential Equations by Mitzenmacher JSAC 2000 Performance Analysis of the IEEE 802.11 Distributed Coordination Function by Bianchi FOCS 2002 Load balancing with memory by Mitzenmacher et al. DSN 2013 A logic for model-checking mean-field models by Kolesnichenko et al DSN 2013 Lumpability of fluid models with heterogeneous agent types by Iacobelli and Tribastone SIGMETRICS 2013 A mean field model for a class of garbage collection algorithms in flash-based solid state drives by Van Houdt . . . . . . Nicolas Gast – 9 / 30

These models correspond to distributed systems Each object interacts with the mass Nicolas Gast – 10 / 30

These models correspond to distributed systems Each object interacts with the mass We view the population of objects more abstractly, assuming that individuals are indistinguishable. Nicolas Gast – 10 / 30

These models correspond to distributed systems Each object interacts with the mass We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state. Nicolas Gast – 10 / 30

These models correspond to distributed systems Each object interacts with the mass We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state. Nicolas Gast – 10 / 30

These models correspond to distributed systems Each object interacts with the mass We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state. Nicolas Gast – 10 / 30

Population CTMC We consider a sequence X N , indexed by the population size N , with state spaces E N ⊂ E ⊂ R d . The transitions are: X �→ X + ℓ at rate N β ℓ ( X ) . N for a finite number of ℓ ∈ L . � The drift is f ( x ) = ℓβ ℓ ( x ). ℓ Example : The state is ( X S , X I , X R ) and the transitions are ℓ 1 = ( − 1 , +1 , 0) at rate NX S X I ℓ 2 = (0 , − 1 , +1) at rate NX I ℓ 3 = (+1 , 0 , − 1) at rate NX R ℓ 4 = ( − 1 , 0 , +1) at rate NX S Nicolas Gast – 11 / 30

Kurtz’ convergence theorem Theorem: Let X be a population model. If X N (0) converges (in probability) to a point x , then the stochastic process X N converges (in probability) to the solutions of the differential equation ˙ x = f ( x ), where f is the drift. 0.6 X S (one simulation) 0.5 [ X S ] (ave. over 10000 simu) x s (mean-field approximation) 0.4 0.3 X S 0.2 0.1 0.0 0 1 2 3 4 5 6 7 8 time N = 10 Nicolas Gast – 12 / 30

Kurtz’ convergence theorem Theorem: Let X be a population model. If X N (0) converges (in probability) to a point x , then the stochastic process X N converges (in probability) to the solutions of the differential equation ˙ x = f ( x ), where f is the drift. 0.6 X S (one simulation) 0.5 [ X S ] (ave. over 10000 simu) x s (mean-field approximation) 0.4 0.3 X S 0.2 0.1 0.0 0 1 2 3 4 5 6 7 8 time N = 100 Nicolas Gast – 12 / 30

Kurtz’ convergence theorem Theorem: Let X be a population model. If X N (0) converges (in probability) to a point x , then the stochastic process X N converges (in probability) to the solutions of the differential equation ˙ x = f ( x ), where f is the drift. 0.6 X S (one simulation) 0.5 [ X S ] (ave. over 10000 simu) x s (mean-field approximation) 0.4 0.3 X S 0.2 0.1 0.0 0 1 2 3 4 5 6 7 8 time N = 1000 Nicolas Gast – 12 / 30

Outline Population Processes and Classical Mean Field Methods 1 Uncertain and Imprecise Population Processes 2 Numerical Algorithms and Comparisons 3 Numerical algorithms (transient regime) Steady-state General processor sharing example Conclusion 4 Nicolas Gast – 13 / 30

Uncertain and imprecise models Instead of β ℓ ( x ), the rates can depend on a parameter ϑ : β ℓ ( x , ϑ ). We distinguish two kinds of uncertainties: Uncertain Imprecise ϑ = ϑ ( t ) ∈ Θ can vary (measur- ϑ ∈ Θ is constant but its value is ably) as a function of time not known precisely. human behavior, Uncertainties in the model environment, adversary,... Nicolas Gast – 14 / 30

Uncertain and imprecise population models We consider a sequence X N of Imprecise or Uncertain population processes, indexed by the size N , with state spaces E N ⊂ E ⊂ R d . The transitions are (for ℓ ∈ L ): X �→ X + ℓ at rate N β ℓ ( X , ϑ ) N � The drifts corresponding to parameter ϑ is f ( x , ϑ ) = ℓβ ℓ ( x , ϑ ). ℓ ∈L Theorem (Bortolussi, G. 2016) if X N (0) converges (in probability) to a point x, then the uncertain (or imprecise) stochastic process X N converges in probability to: Uncertain Imprecise � A solution of ˙ x = f ( x , ϑ ) (for a given ϑ ) A solution of ˙ x ∈ f ( x , ϑ ) ϑ ∈ Θ Nicolas Gast – 15 / 30

Example of the SIR model The state is ( X S , X I , X R ) and the transitions are ℓ 1 = ( − 1 , +1 , 0) at rate N ( aX S + ϑ X S X I ) ℓ 2 = (0 , − 1 , +1) at rate NbX I ℓ 3 = (+1 , 0 , − 1) at rate NcX R 0 . 30 x max (uncertain) I 0 . 25 x min (uncertain) I x max (imprecise) proportion of infected I 0 . 20 x min (imprecise) I 0 . 15 0 . 10 0 . 05 0 . 00 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5 3 . 0 3 . 5 4 . 0 Nicolas Gast – 16 / 30 time

Consequence on the stochastic system Uncertain Imprecise 0 . 35 0 . 40 x max x max (uncertain) (imprecise) I I 0 . 35 0 . 30 x min x min (uncertain) (imprecise) I I 0 . 30 proportion of infected 0 . 25 proportion of infected 0 . 25 0 . 20 0 . 20 0 . 15 0 . 15 0 . 10 0 . 10 0 . 05 0 . 05 0 . 00 0 . 00 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5 3 . 0 3 . 5 4 . 0 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5 3 . 0 3 . 5 4 . 0 time time N = 1000 Remark : the proportion of infected is non-monotone on the infection rate. Nicolas Gast – 17 / 30

Consequence on the stochastic system Uncertain Imprecise 0 . 35 0 . 35 x max x max (uncertain) (imprecise) I I 0 . 30 0 . 30 x min x min (uncertain) (imprecise) I I proportion of infected 0 . 25 proportion of infected 0 . 25 0 . 20 0 . 20 0 . 15 0 . 15 0 . 10 0 . 10 0 . 05 0 . 05 0 . 00 0 . 00 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5 3 . 0 3 . 5 4 . 0 0 . 0 0 . 5 1 . 0 1 . 5 2 . 0 2 . 5 3 . 0 3 . 5 4 . 0 time time N = 10000 Remark : the proportion of infected is non-monotone on the infection rate. Nicolas Gast – 17 / 30