Mean-field methods: what can go wrong? with some applications to - PowerPoint PPT Presentation

Mean-field methods: what can go wrong? with some applications to bike-sharing systems and caching Nicolas Gast (Inria) Inria, Grenoble, France Grenoble’s RO Summer School, July 2016 Nicolas Gast (Inria) – 1 / 47

In this talk, we will study dynamical systems Nicolas Gast (Inria) – 2 / 47

14 1-7 mai 12 nombres de place disponibles 10 8 6 4 2 0 0 1 2 3 4 5 6 7 temps (en jour) From Wikipedia: In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in a geometrical space. system’s state time Nicolas Gast (Inria) – 3 / 47

From Wikipedia: In mathematics, a dynamical system is a system in which a function describes the time dependence of a point in a geometrical space. 14 1-7 mai system’s 12 state nombres de place disponibles 10 8 6 4 2 0 0 1 2 3 4 5 6 7 temps (en jour) time Nicolas Gast (Inria) – 3 / 47

Continuous time Markov chains: Three possible definitions S   − 1 1 0 Q = 0 − 2 2   1 0 . 1 0 . 1 0 − 0 . 1 2 I R Transition graph Infinitesimal generator P ( Z ( t + dt ) = j | Z ( t ) = i ∧ the past) = P ( Z ( t + dt ) = j | Z ( t ) = i ) = Q ij dt + o ( dt ) if i � = j Markov property Nicolas Gast (Inria) – 4 / 47

Transient and steady-state analysis   − 1 1 0 Q = 0 − 2 2   0 . 1 0 − 0 . 1 Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis   − 1 1 0 Q = 0 − 2 2   0 . 1 0 − 0 . 1 Transient analysis: the master equation If X is a CTMC (continuous time Markov chain) with generator Q : d � dt P i ( t ) = P j ( t ) Q ji , j ∈ S where P i ( t ) = P ( X ( t ) = i ). Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis   − 1 1 0 Q = 0 − 2 2   0 . 1 0 − 0 . 1 Transient analysis: the master equation If X is a CTMC (continuous time Markov chain) with generator Q : d dt P ( t ) = P ( t ) Q , where P i ( t ) = P ( X ( t ) = i ). Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis   − 1 1 0 Q = 0 − 2 2   0 . 1 0 − 0 . 1 Steady-state analysis Nicolas Gast (Inria) – 5 / 47

Transient and steady-state analysis   − 1 1 0 Q = 0 − 2 2   0 . 1 0 − 0 . 1 Steady-state analysis If the chain is irreducible, The equation π Q = 0 has a unique solution such that � i π i = 1. lim i →∞ P i ( t ) = π i Nicolas Gast (Inria) – 5 / 47

State space explosion and decoupling method We need to keep track of S N states P ( Z 1 ( t ) = i 1 , . . . , Z n ( t ) = i n ) The generator Q has S N entries. 3 13 ≈ 10 6 states. Nicolas Gast (Inria) – 6 / 47

State space explosion and decoupling method We need to keep track of S N states P ( Z 1 ( t ) = i 1 , . . . , Z n ( t ) = i n ) The generator Q has S N entries. 3 13 ≈ 10 6 states. The decoupling assumption is P ( Z 1 ( t ) = i 1 , . . . , Z n ( t ) = i n ) ≈ P ( Z 1 ( t ) = i 1 ) . . . P ( Z n ( t ) = i n ) � �� � � �� � S N variables N × S variables Question: when is this (not) valid? Nicolas Gast (Inria) – 6 / 47

Outline Population models and mean-field 1 The decoupling method: finite and infinite time horizon 2 Finite time horizon: some theory Steady-state regime Rate of convergence Case-studies 3 Bike-sharing systems Cache replacement policy Conclusion and recap 4 Nicolas Gast (Inria) – 7 / 47

Outline Population models and mean-field 1 The decoupling method: finite and infinite time horizon 2 Finite time horizon: some theory Steady-state regime Rate of convergence Case-studies 3 Bike-sharing systems Cache replacement policy Conclusion and recap 4 Nicolas Gast (Inria) – 8 / 47

Mean field methods have been used in a multiple contexts ex: model-checking, performance of SSD, load balancing, MAC protocol,... JAP 90 On an index policy for restless bandits by Weber and Weiss SPAA 98 Analyses of Load Stealing Models Based on Differential Equations by Mitzenmacher JSAC 2000 Performance Analysis of the IEEE 802.11 Distributed Coordination Function by Bianchi FOCS 2002 Load balancing with memory by Mitzenmacher et al. Ramaiyan et al Fixed point analys is of single cell IEEE 802.11e WLANs: Uniqueness, multistability by ToN 2008 SIGMETRICS 2013 A mean field model for a class of garbage collection algorithms in flash-based solid state drives by Van Houdt EJTL 2014 Incentives and redistribution in homogeneous bike-sharing systems with stations of finite capacities by Fricker and G. SIGMETRICS 2015 Transient and Steady-state Regime of a Family of List-based Cache Replacement Algorithms by G. and Van Houdt . . . . . . Nicolas Gast (Inria) – 9 / 47

These models correspond to distributed systems Each object interacts with the mass Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems Each object interacts with the mass We view the population of objects more abstractly, assuming that individuals are indistinguishable. Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems Each object interacts with the mass X ( t ) We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state. Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems Each object interacts with the mass X ( t ) We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state. Nicolas Gast (Inria) – 10 / 47

These models correspond to distributed systems Each object interacts with the mass X ( t ) We view the population of objects more abstractly, assuming that individuals are indistinguishable. An occupancy measure records the proportion of agents that are currently exhibiting each possible state. Nicolas Gast (Inria) – 10 / 47

Population CTMC A population process is a sequence of CTMC X N , indexed by the population size N , with state spaces E N ⊂ E ⊂ R d such that the transitions are (for ℓ ∈ L ): X �→ X + ℓ at rate N β ℓ ( X ) . N The drift is f ( x ) = � ℓ ℓβ ℓ ( x ). Nicolas Gast (Inria) – 11 / 47

Population CTMC A population process is a sequence of CTMC X N , indexed by the population size N , with state spaces E N ⊂ E ⊂ R d such that the transitions are (for ℓ ∈ L ): X �→ X + ℓ at rate N β ℓ ( X ) . N The drift is f ( x ) = � ℓ ℓβ ℓ ( x ). Example : SIRS model The state is ( x S , x I , x R ). The transitions are S ℓ β ℓ ( x ) vacc Infection ( − 1 , +1 , 0) x S + x S x I infection Recovery (0 , − 1 , +1) x I susc. Susceptible (+1 , 0 , − 1) x R recovery I R Vaccination ( − 1 , 0 , +1) x S Nicolas Gast (Inria) – 11 / 47

Kurtz’ convergence theorem Theorem: Let X be a population process and assume that its drift f is Lipschitz-continuous and that sup ℓ ∈L | ℓ | < ∞ . If X N (0) converges (in probability) to a point x , then the stochastic process X N converges (in probability) to the solutions of the differential equation ˙ x = f ( x ), where f is the drift. 0.6 X S (one simulation) 0.5 [ X S ] (ave. over 10000 simu) x s (mean-field approximation) 0.4 0.3 X S 0.2 0.1 0.0 0 1 2 3 4 5 6 7 8 time N = 10 Nicolas Gast (Inria) – 12 / 47

Kurtz’ convergence theorem Theorem: Let X be a population process and assume that its drift f is Lipschitz-continuous and that sup ℓ ∈L | ℓ | < ∞ . If X N (0) converges (in probability) to a point x , then the stochastic process X N converges (in probability) to the solutions of the differential equation ˙ x = f ( x ), where f is the drift. 0.6 X S (one simulation) 0.5 [ X S ] (ave. over 10000 simu) x s (mean-field approximation) 0.4 0.3 X S 0.2 0.1 0.0 0 1 2 3 4 5 6 7 8 time N = 100 Nicolas Gast (Inria) – 12 / 47

Kurtz’ convergence theorem Theorem: Let X be a population process and assume that its drift f is Lipschitz-continuous and that sup ℓ ∈L | ℓ | < ∞ . If X N (0) converges (in probability) to a point x , then the stochastic process X N converges (in probability) to the solutions of the differential equation ˙ x = f ( x ), where f is the drift. 0.6 X S (one simulation) 0.5 [ X S ] (ave. over 10000 simu) x s (mean-field approximation) 0.4 0.3 X S 0.2 0.1 0.0 0 1 2 3 4 5 6 7 8 time N = 1000 Nicolas Gast (Inria) – 12 / 47

Outline Population models and mean-field 1 The decoupling method: finite and infinite time horizon 2 Finite time horizon: some theory Steady-state regime Rate of convergence Case-studies 3 Bike-sharing systems Cache replacement policy Conclusion and recap 4 Nicolas Gast (Inria) – 13 / 47

Outline Population models and mean-field 1 The decoupling method: finite and infinite time horizon 2 Finite time horizon: some theory Steady-state regime Rate of convergence Case-studies 3 Bike-sharing systems Cache replacement policy Conclusion and recap 4 Nicolas Gast (Inria) – 14 / 47