From the master equation to mean field game asymptotics Daniel - PowerPoint PPT Presentation

From the master equation to mean field game asymptotics From the master equation to mean field game asymptotics Daniel Lacker Division of Applied Mathematics, Brown University 8th Western Conference in Mathematical Finance, March 24, 2017 Joint work with Francois Delarue and Kavita Ramanan

From the master equation to mean field game asymptotics Overview Overview A mean field game (MFG) is a game with a continuum of players. In various contexts, we know rigorously that the MFG arises as the limit of n -player games as n → ∞ . But how close of an approximation is an MFG for the n -player game? This talk: Refined MFG asymptotics in the form of a central limit theorem and large deviation principle, as well as non-asymptotic concentration bounds.

From the master equation to mean field game asymptotics Interacting diffusion models Interacting diffusions Suppose particles i = 1 , . . . , n interact through their empirical measure according to n � t = 1 dX i t = b ( X i ν n t ) dt + dW i ν n t , ¯ t , ¯ δ X k t , n k =1 where W 1 , . . . , W n are independent Brownian motions. ν n Under “nice” assumptions on b , we have ¯ t → ν t , where ν t solves the McKean-Vlasov equation, dX t = b ( X t , ν t ) dt + dW t , ν t = Law ( X t ) .

From the master equation to mean field game asymptotics Interacting diffusion models Empirical measure limit theory ν n There is a rich literature on asymptotics of ¯ t : ν n → ν , where ν solves a McKean-Vlasov equation. 1. LLN: ¯ (Oelschl¨ ager ’84, G¨ artner ’88, Sznitman ’91, etc.) 2. Fluctuations: √ n (¯ ν n t − ν t ) converges to a distribution-valued process driven by space-time Brownian motion. (Tanaka ’84, Sznitman ’85, Kurtz-Xiong ’04, etc.) ν n has an explicit LDP. 3. Large deviations: ¯ (Dawson-G¨ artner ’87, Budhiraja-Dupius-Fischer ’12) 4. Concentration: Finite- n bounds are available for ν n , ν ) > ǫ ), for various metrics d . P ( d (¯ (Bolley-Guillin-Villani ’07, etc.) The idea: The McKean-Vlasov system is often more amenable to analysis than the more physical n -particle system.

From the master equation to mean field game asymptotics Interacting diffusion models From particle systems to mean field games Interacting diffusion systems are zero-intelligence models. Mean field games are often more suitable in financial/economic applications, replacing particles with decision-makers. The dynamics of X i become controlled, and the n -particle system becomes a game. The idea: Approximate the realistic n -player game equilibrium using the more tractable MFG limit ( n → ∞ ). This talk: Quantitatively relate the n -player equilibrium to an interacting diffusion system, then bootstrap existing results for the latter.

From the master equation to mean field game asymptotics Mean field games A class of mean field games Agents i = 1 , . . . , n have state process dynamics dX i t = α i t dt + dW i t , with W 1 , . . . , W n independent Brownian, ( X 1 0 , . . . , X n 0 ) i.i.d. Agent i chooses α i to minimize �� T � � � t ) + 1 J n i ( α 1 , . . . , α n ) = E f ( X i µ n 2 | α i t | 2 dt + g ( X i µ n t , ¯ T , ¯ T ) , 0 n � t = 1 µ n ¯ δ X k t . n k =1 Say ( α 1 , . . . , α n ) form an ǫ -Nash equilibrium if J n i ( α 1 , . . . , α n ) ≤ ǫ + inf β J n i ( . . . , α i − 1 , β, α i +1 , . . . ) , ∀ i = 1 , . . . , n

From the master equation to mean field game asymptotics Mean field games The n -player HJB system The value function v n i ( t , ① ), for ① = ( x 1 , . . . , x n ), for agent i in the n -player game solves n � i ( t , ① ) + 1 i ( t , ① ) + 1 i ( t , ① ) | 2 ∂ t v n ∆ x k v n 2 |∇ x i v n 2 k =1 � � n � � x i , 1 ∇ x k v n k ( t , ① ) · ∇ x k v n + i ( t , ① ) = f δ x k . n k � = i k =1 A Nash equilibrium is given by α i t = ∇ x i v n i ( t , X 1 t , . . . , X n t ) . But v n i is generally hard to find, especially for large n .

From the master equation to mean field game asymptotics Mean field games Mean field limit n → ∞ ? The problem Given a Nash equilibrium ( α n , 1 , . . . , α n , n ) for each n , can we µ n describe the limit(s) of ¯ t ? Previous results Lasry/ Lions ’06, Feleqi ’13, Fischer ’14, Lacker ’15, Cardaliaguet-Delarue-Lasry-Lions ’15, Cardaliaguet ’16... A related, better-understood problem Find a mean field game solution directly, and use it to construct an ǫ n -Nash equilibrium for the n -player game, where ǫ n → 0. See Huang/Malham´ e/Caines ’06 & many others.

From the master equation to mean field game asymptotics Mean field games Proposed mean field game limit A deterministic measure flow ( µ t ) t ∈ [0 , T ] ∈ C ([0 , T ]; P ( R d )) is a mean field equilibrium (MFE) if:  �� T � � 2 | α t | 2 �  t , µ t ) + 1 α ∗ f ( X α dt + g ( X α ∈ arg min α E T , µ T ) ,   0 dX α = α t dt + dW t ,  t   = Law( X α ∗ µ t t ) . Theorem (Law of large numbers) Under very strong assumptions, there exists a unique MFE µ , and µ n → µ in probability in C ([0 , T ]; P ( R d )) . ¯

From the master equation to mean field game asymptotics The master equation MFG value function The MFE is completely described by the master equation, when it is solvable. 1. Fix t ∈ [0 , T ) and m ∈ P ( R d ). 2. Solve the MFG starting from ( t , m ), i.e., find ( α ∗ , µ ) s.t.  �� T � � 2 | α s | 2 �  α ∗ f ( X α s , µ s ) + 1 ds + g ( X α  ∈ arg min α E T , µ T ) ,  t dX α = α s ds + dW s , s ∈ ( t , T )  s   = Law( X α ∗ µ s s ) , µ t = m 3. Define the value function, for x ∈ R d , by U ( t , x , m ) �� T � � � � � s , µ s ) + 1 f ( X α ∗ 2 | α ∗ s | 2 ds + g ( X α ∗ � � X α ∗ = E T , µ T ) = x t t

From the master equation to mean field game asymptotics The master equation Derivatives There is a dynamic programming principle for U if the MFE is unique. To derive a PDE, we need to differentiate in m : Definition Say u : P ( R d ) → R is C 1 if ∃ δ u δ m : P ( R d ) × R d → R continuous m ∈ P ( R d ), such that, for m , � � u ( m + t ( � m − m )) − u ( m ) δ u lim = δ m ( m , y ) d ( � m − m )( y ) . t h ↓ 0 R d Define also (when it exists) � δ u � D m u ( m , y ) = ∇ y δ m ( m , y ) .

From the master equation to mean field game asymptotics The master equation Key tool: The master equation Heuristically, using the DPP along with an Itˆ o formula for functions of measures, one derives the master equation for the value function: � ∂ t U ( t , x , m ) − R d ∇ x U ( t , y , m ) · D m U ( t , x , m , y ) m ( dy ) + f ( x , m ) − 1 2 |∇ x U ( t , x , m ) | 2 + 1 2∆ x U ( t , x , m ) � + 1 R d div y D m U ( t , x , m , y ) m ( dy ) = 0 , 2 Refer to Cardaliaguet-Delarue-Lasry-Lions ’15, Chassagneux-Crisan-Delarue ’14, Carmona-Delarue ’14, Bensoussan-Frehse-Yam ’15

From the master equation to mean field game asymptotics The master equation Key tool: The master equation Heuristically, using the DPP along with an Itˆ o formula for functions of measures, one derives the master equation for the value function: � ∂ t U ( t , x , m ) − R d ∇ x U ( t , y , m ) · D m U ( t , x , m , y ) m ( dy ) + f ( x , m ) − 1 2 |∇ x U ( t , x , m ) | 2 + 1 2∆ x U ( t , x , m ) � + 1 R d div y D m U ( t , x , m , y ) m ( dy ) = 0 , 2 Assume henceforth that there is a smooth classical solution!

From the master equation to mean field game asymptotics The master equation A first n -particle approximation The MFE µ is the unique solution of the McKean-Vlasov equation dX t = ∇ x U ( t , X t , µ t ) dt + dW t , µ t = Law( X t ) . � �� � α ∗ t Old idea: Consider the system of n independent processes, dX i t = ∇ x U ( t , X i dt + dW i t , µ t ) t . � �� � α i t These controls α i t can be proven to form an ǫ n -equilibrium for the n -player game, where ǫ n → 0. Note X i t are i.i.d. ∼ µ t , so their empirical measure tends to µ t .

From the master equation to mean field game asymptotics The master equation A better n -particle approximation Key idea of Cardaliaguet et al.: Consider the McKean-Vlasov system n � t = 1 dY i t = ∇ x U ( t , Y i ν n dt + dW i ν n t , ¯ t ) t , ¯ δ Y k t . � �� � n k =1 α i t ν n → ν , where ν solves the Classical theory says that ¯ McKean-Vlasov equation, dY t = ∇ x U ( t , Y t , ν t ) dt + dW t , ν t = Law( Y t ) . We had the same equation for the MFE µ , so uniqueness implies µ ≡ ν. µ n → µ , it suffices to show ¯ µ n and ¯ ν n are close . So to prove ¯

From the master equation to mean field game asymptotics The master equation A better n -particle approximation Theorem (Cardaliaguet et al. ’15) µ n Recalling that ¯ t denotes the n-player Nash equilibrium empirical µ n and ¯ ν n are very close. measure, ¯ Proof idea: Show that   � 1 u n  t , x i ,  i ( t , x 1 , . . . , x n ) = U δ x k n − 1 k � = i nearly solves the n -player HJB system. Note: This requires smoothness assumptions on the master equation U , but not on the n -player HJB system!