A Generic Mean Field Model for Optimization in Large-scale Stochastic Systems and Applications in Scheduling Nicolas Gast Bruno Gaujal Grenoble University Knoxville, May 13th-15th 2009 N. Gast (LIG) Mean Field Optimization Knoxville 2009 1 / 20
Introduction Mean field has been introduced by physicists to study systems of interacting objects. For example, the movement of particles in the air: First solution: the microscopic description The system is represented by the states of each particle. Many equations for each possible collision: impossible to solve exactly. Second solution (better!): macroscopic equations We are interested by the average behavior of the system: The system is described by its temperature. Deterministic equation. The transition from microscopic description to macroscopic equations is called the mean field approximation. N. Gast (LIG) Mean Field Optimization Knoxville 2009 2 / 20
Mean Field in Computer Science More recently, Mean field has been used to analyze performance of communication systems. The objects are the users in the system. For example: Performance of TCP [Baccelli, McDonald, Reynier [02]] Reputation Systems [Le Boudec et al. [07]] 802.11 [Bordenave, McDonald, Prouti` ere [05]] . . . In many example, it can be shown that when the number of users grows, the average behavior of the system becomes deterministic. N. Gast (LIG) Mean Field Optimization Knoxville 2009 3 / 20
Mean Field in Computer Science More recently, Mean field has been used to analyze performance of communication systems. The objects are the users in the system. For example: Performance of TCP [Baccelli, McDonald, Reynier [02]] Reputation Systems [Le Boudec et al. [07]] 802.11 [Bordenave, McDonald, Prouti` ere [05]] . . . In many example, it can be shown that when the number of users grows, the average behavior of the system becomes deterministic. Aim of this talk Show that mean field can also be used for optimization problem. Study a general framwork for wich we can prove the results. N. Gast (LIG) Mean Field Optimization Knoxville 2009 3 / 20
Example of mean field model Example – Consider the following brokering problem: µ 1 . a 1 . P 1 processors E 1 . t µ 1 . . . . Broker . . tasks µ d . a d . S on/off P d processors E d . t µ d sources Stochastic system Objects are sources+Processors: There are S + P 1 + · · · + P d objects The state of an object is active or inactive (random) Evolution of state is markovian N. Gast (LIG) Mean Field Optimization Knoxville 2009 4 / 20
Example of mean field model Example – Consider the following brokering problem: µ 1 . a 1 . P 1 processors E 1 . t µ 1 . . . . Broker . . tasks µ d . a d . S on/off P d processors E d . t µ d sources Mean field limit Stochastic system We scale S and P i by N . Objects are sources+Processors: We are interested in: There are S + P 1 + · · · + P d objects (number of tasks sent) / N . The state of an object is active or inactive (random) (available processors in cluster i ) / N . Evolution of state is markovian N. Gast (LIG) Mean Field Optimization Knoxville 2009 4 / 20
Main result Stochastic system N objects. Compute optimal policy (hard) Optimal system Remark: the purpose of this talk is not to solve the previous example but to study a general framework for optimization in stochastic systems. N. Gast (LIG) Mean Field Optimization Knoxville 2009 5 / 20
Main result Stochastic system N N → ∞ Mean Field Limit objects. Compute optimal policy (hard) Optimal system Compute the mean field limit (constructive definition). 1 N. Gast (LIG) Mean Field Optimization Knoxville 2009 5 / 20
Main result Stochastic system N N → ∞ Mean Field Limit objects. Deterministic Compute optimal optimization policy (hard) (Easier) Optimal system Optimal mean field Compute the mean field limit (constructive definition). 1 Solve the deterministic problem. 2 N. Gast (LIG) Mean Field Optimization Knoxville 2009 5 / 20
Main result Stochastic system N N → ∞ Mean Field Limit objects. Deterministic Compute optimal optimization policy (hard) (Easier) We can exchange the Optimal system Optimal mean field limits Our results The optimal stochastic system also converges. More precisely, when N grows: The optimal reward converges. 1 The optimal policy also converges. 2 √ The speed of convergence is O ( N ) (CLT theorem). 3 N. Gast (LIG) Mean Field Optimization Knoxville 2009 5 / 20
Theoretical Results 1 A (simple) example 2 Conclusion 3 N. Gast (LIG) Mean Field Optimization Knoxville 2009 6 / 20
An optimization problem N objects evolving in a finite state space. Environment E ( t ) at time t ( E ( t ) ∈ R d ) random( T − 1 ) random( 0 ) random( 1 ) X 1 (0) X 1 (1) X 1 ( T ) . . . . . . . . . X N (0) X N (1) X N ( T ) E (0) E (1) E ( T ) N. Gast (LIG) Mean Field Optimization Knoxville 2009 7 / 20
An optimization problem N objects evolving in a finite state space. Environment E ( t ) at time t ( E ( t ) ∈ R d ) random( T − 1 ) random( 0 ) random( 1 ) X 1 (0) X 1 (1) X 1 ( T ) . . . . . . . . . X N (0) X N (1) X N ( T ) E (0) E (1) E ( T ) Mean field assumption We define the Population mix M ( t ) – The i th component ( M ( t )) i is the proportion of objects in state i at time t . E ( t + 1) only depends on the population mix M ( t ). 1 The evolution of an object depends on E ( t ) but is independent of the 2 other objects. N. Gast (LIG) Mean Field Optimization Knoxville 2009 7 / 20
An optimization problem Controller, chooses an action a ∈ A a 0 a 1 a T − 1 random( a T − 1 ) random( a 0 ) random( a 1 ) X 1 (0) X 1 (1) X 1 ( T ) . . . . . . . . . X N (0) X N (1) X N ( T ) E (0) E (1) E ( T ) The controller can change the dynamics of the system. N. Gast (LIG) Mean Field Optimization Knoxville 2009 7 / 20
An optimization problem Controller, chooses an action a ∈ A a 0 a 1 a T − 1 random( a T − 1 ) random( a 0 ) random( a 1 ) X 1 (0) X 1 (1) X 1 ( T ) . . . . . . . . . X N (0) X N (1) X N ( T ) E (0) E (1) E ( T ) . . . Cost(X,E) + + Cost(X,E) The controller can change the dynamics of the system. Goal Technical assumption: Find a policy to maximize: Mean field evolution finite-time expected cost or Action set compact expected discounted cost Continuous parameters N. Gast (LIG) Mean Field Optimization Knoxville 2009 7 / 20
Optimal cost convergence V ∗ N – optimal cost for the system of size N . v ∗ – optimal cost for the deterministic limit. a ∗ 0 a ∗ 1 a ∗ 2 . . . – optimal actions for the deterministic limit. Theorem (Convergence of the optimal cost) Under technical assumptions, for both discounted and finite-time cost: lim N →∞ V ∗ N = v ∗ ( a . s . ) lim N →∞ V ∗ N V N = a ∗ 0 a ∗ 1 a ∗ 2 ... In particular, this shows that: Optimal cost converges Static policy a ∗ is asymptotically optimal N. Gast (LIG) Mean Field Optimization Knoxville 2009 8 / 20
A central limit-like theorem √ The convergence speed is in O (1 / N ): Theorem (CLT for the evolution of objects) Under technical assumptions, if the actions taken by the controller are fixed, then there exists a Gaussian variable G t s.t: √ � Law M N � N t − m t − − → G t The covariance of G t can be effectively computed. N. Gast (LIG) Mean Field Optimization Knoxville 2009 9 / 20
A central limit-like theorem √ The convergence speed is in O (1 / N ): Theorem (CLT for the evolution of objects) Under technical assumptions, if the actions taken by the controller are fixed, then there exists a Gaussian variable G t s.t: √ � Law M N � N t − m t − − → G t The covariance of G t can be effectively computed. Theorem (CLT for cost) Under technical assumptions, when N goes to infinity: √ � V ∗ N � − V N � N ≤ st β + γ � G 0 � ∞ √ � T a ∗ β ′ + γ ′ � G 0 � ∞ � V ∗ N � � − v ∗ ≤ st N � T T N. Gast (LIG) Mean Field Optimization Knoxville 2009 9 / 20
Theoretical Results 1 A (simple) example 2 Conclusion 3 N. Gast (LIG) Mean Field Optimization Knoxville 2009 10 / 20
A simple resource allocation problem Aim of the example Illustrate the framework by a concrete example When does N (= S + P 1 + · · · + P d ) becomes large enough for the approximation to apply? N. Gast (LIG) Mean Field Optimization Knoxville 2009 11 / 20
A simple resource allocation problem Aim of the example Illustrate the framework by a concrete example When does N (= S + P 1 + · · · + P d ) becomes large enough for the approximation to apply? µ 1 . a 1 . P 1 processors E 1 . t µ 1 . . . . . Broker . tasks µ d . . S on/off a d P d processors E d . t µ d sources Stochastic Stochastic availability: failure,. . . arrivals Optimize the total completion time = � T � d i =1 E i ( t ). t =0 N. Gast (LIG) Mean Field Optimization Knoxville 2009 11 / 20
Recommend
More recommend
