Mean-Field optimization problems and non-anticipative optimal - PowerPoint PPT Presentation

Hidden/no dynamics State dynamics Conclusions Mean-Field optimization problems and non-anticipative optimal transport Beatrice Acciaio London School of Economics based on ongoing projects with J. Backhoff, R. Carmona and P . Wang Thera Stochastics A Mathematics Conference in Honor of Ioannis Karatzas Thera, Santorini, May 31 - June 2, 2017

Hidden/no dynamics State dynamics Conclusions The story in a nutshell Given a (finite or infinite) set of agents who need to choose their actions/strategies and face a cost depending on their own type, action, and on the symmetric interaction with each other: � � cost ( i ) = fct type ( i ) , action ( i ) , (empirical) distrib. actions Aim to → find/characterize equilibria → through connections with non-anticipative optimal transport

Hidden/no dynamics State dynamics Conclusions Outline First setting: hidden/no dynamics 1 Problem formulation Connection with non-anticipative optimal transport Existence and uniqueness results Second setting: state dynamics 2 Problem formulation Connection with non-anticipative optimal transport First results 3 Conclusions

Hidden/no dynamics State dynamics Conclusions Setting time set: T = { 0 , ..., T } , or T = [ 0 , T ] X : agents types X ⊆ X | T | : agents types evolutions Y : agents’ actions Y ⊆ Y | T | : agents’ actions evolutions e.g. X = Y = R , and X = Y = R T + 1 or X = Y = C ([ 0 , T ]; R ) η ∈ P ( X ) : known a priori distribution over types → cost function: ˜ c ( x , y , ν ) (for each agent) ր ↑ տ type action actions’ distribution x ∈ X y ∈ Y ν ∈ P ( Y )

Hidden/no dynamics State dynamics Conclusions Cost function Separable structure: ˜ c ( x , y , ν ) = c ( x , y ) + V [ ν ]( y ) ր տ idiosyncratic mean-field part interaction with c : X × Y → R + l.s.c., V : P ( Y ) → B ( Y ; R + ) � � y , d ν congestion effect : V c [ ν ]( y ) = f dm ( y ) , with m ∈ P ( Y ) reference meas. w.r.t. which congestion measured, f ( y , . ) ր � attractive effect : V a [ ν ]( y ) = Y φ ( y , z ) ν ( dz ) , with φ symmetric, convex, minimal on the diagonal Static case: Blanchet-Carlier 2015

Hidden/no dynamics State dynamics Conclusions Pure adapted strategies pure strategy: all players of type x ∈ X choose the same strategy y = A ( x ) = ( A t ( x )) t ∈ T adapted strategy: A t ( x ) = T t ( x 0 : t ) for some measurable T t Denote by A the set of pure adapted strategies A : X → Y type distribution: η ∈ P ( X ) (known) T = ( T t ) t ∈ T strategy distribution: ν = A # η = T # η ∈ P ( Y ) , (will be determined in equilibrium)

Hidden/no dynamics State dynamics Conclusions Pure equilibrium Social planner perspective : minimize average cost For every ν ∈ P ( Y ) , denote � � � P ( ν ) := inf c ( x , A ( x )) + V [ ν ]( A ( x )) η ( dx ) A ∈A Definition An element A ∈ A is called a pure equilibrium if A attains P ( ν ) , where ν = A # η .

Hidden/no dynamics State dynamics Conclusions Cournot-Nash equilibrium Remark. Let T = { 0 , 1 , ..., T } (analogous in continuous time). Let c ( x , y ) = � T t = 0 c t ( x 0 : t , y t ) and V [ ν ]( y ) = � T t = 0 V t [ ν t ]( y t ) , then pure equilibrium for social planner = Cournot-Nash equilibrium ( η -a.s. each agent acts as best response to other agents’ actions) � � A ν : A ν # η = ν The equilibria are described by the set , where � � A ν t ( x ) = T ν t ( x 0 : t ) := arg min z c t ( x 0 : t , z ) + V t [ ν t ]( z ) . This is clearly a specific situation Anyway, pure equilibria rarely exists, so we shall consider the natural generalization to mixed-strategy equilibria.

Hidden/no dynamics State dynamics Conclusions From pure to mixed-strategy equilibrium A x A(x) type action adapted pure strategy = adapted Monge transport

Hidden/no dynamics State dynamics Conclusions From pure to mixed-strategy equilibrium x type actions non-anticipative mixed strategy = causal Kantorovich transport

Hidden/no dynamics State dynamics Conclusions Mixed non-anticipative strategy mixed-strategy: players of same type can choose different actions non-anticipative: A t ( x ) = fct ( x 0 : t ) + sth indep. of x ↓ Non-anticipative (causal) transport : π ∈ P ( X × Y ) s.t. p 1 # π = η , and for all t and D ∈F Y t , the map X∋ x �→ π x ( D ) is F X t -measurable t ) canonical filtr. in X , Y , and π x reg. cond. kernel) (where ( F X t ) , ( F Y Denote by Π c ( η, ν ) the set of causal transports between η and ν , and let Π c ( η, . ) := � ν ∈P ( Y ) Π c ( η, ν ) Note that π = ( id , T ) # η ∈ Π c ( η, . ) are the pure adapted strategies.

Hidden/no dynamics State dynamics Conclusions Mixed-strategy equilibrium For every ν ∈ P ( Y ) , denote π ∈ Π c ( η,. ) E π � � M ( ν ) := c ( x , y ) + V [ ν ]( y ) inf Definition An element π ∈ Π c ( η, . ) is called a mixed-strategy equilibrium if π attains M ( ν ) , where ν = p 2 # π , i.e., π ∈ Π c ( η, ν ) . Remark. Mixed-strategy equilibria are solutions to causal transport problems: if π ∗ m-s equilibrium, with p 2 # π ∗ = ν ∗ , then it attains π ∈ Π c ( η,ν ∗ ) E π [ c ( x , y )] . inf Analogously, pure equilibria=solutions to CTpbs over Monge maps

Hidden/no dynamics State dynamics Conclusions Potential games From the remark, we always have equilibrium = ⇒ optimal transport For potential games , we will have “ ⇐⇒ ” in some sense Assumption There exists E : P ( Y ) → R such that V is the first variation of E : � E ( ν + ǫ ( µ − ν )) − E ( ν ) lim = V [ ν ]( y )( µ − ν )( dy ) , ∀ ν, µ ∈ P ( Y ) ǫ ǫ → 0 + Y E.g. V = V c + V a (repulsive+attractive effect) is the first variation of � � � � y , d ν m ( dy ) + 1 E ( ν ) = dm ( y ) φ ( y , z ) ν ( dz ) ν ( dy ) , F 2 Y Y×Y � u where F ( y , u ) = 0 f ( y , s ) ds .

Hidden/no dynamics State dynamics Conclusions Potential games Consider the variational problem        π ∈ Π c ( η,ν ) E π [ c ( x , y )]  ( VP ) inf inf + E [ ν ]     ν ∈P ( Y )   � ������������������� �� ������������������� � CT ( η, ν ) Theorem Let E be convex, then the following are equivalent : (i) π ∗ is a mixed-strategy equilibrium, with p 2 # π ∗ = ν ∗ ; (ii) ν ∗ solves (VP), and π ∗ solves CT ( η, ν ∗ ) . Remarks. 1. Convexity only needed for “ ( i ) ⇒ ( ii ) ” 2. Convexity satisfied in the congestion case ( V = V c ) 3. Alternatively: displacement convexity can be used

Hidden/no dynamics State dynamics Conclusions Potential games Corollary (uniqueness) If E strictly convex ⇒ all m-s equilibria have same second marginal ν ∗ , i.e., unique optimal distribution of actions. Indeed, ν �→ CT ( η, ν ) convex, hence E strictly convex implies unique solution ν ∗ for (VP). Then apply theorem. Corollary (existence) For V = V c and growth condition on f ⇒ ∃ m-s equilibrium. Indeed, the growth condition ensures existence of a solution ν ∗ for (VP), and CT ( η, ν ∗ ) admits a solution π ∗ since c is bounded below and l.s.c. Then apply theorem.

Hidden/no dynamics State dynamics Conclusions Example Let T = { 0 , 1 , ..., T } , and X = Y = R T + 1 . If η has independent increments, and c ( x , y ) = c 0 ( x 0 , y 0 ) + � T t = 1 c t ( x t − x t − 1 , y t − y t − 1 ) , with c t ( u , v ) = k t ( u − v ) and k t convex, Then: • m-s equilibria (if ∃ ) are determined by the second marginal • m-s equilibria are the Knothe-Rosenblatt rearrangements • if moreover η has a density, all m-s equilibria are in fact pure

Hidden/no dynamics State dynamics Conclusions The Knothe-Rosenblatt map X 1 T 1 (x 1 )

Hidden/no dynamics State dynamics Conclusions The Knothe-Rosenblatt map x 2 T 2 (x 2 |x 1 ) X 1 T 1 (x 1 )

Hidden/no dynamics State dynamics Conclusions Actions as controls on dynamics • The previous result describes a specific situation where optimal actions are increasing with the type. • When these conditions not satisfied, which form of CT/equilibria? Example. Let actions = controls on dynamics: X t = ( k 1 t X t − 1 + k 2 t α t ) + ǫ t , t = 1 , ..., T , X 0 = x 0 , with associated cost f t ( X t , α t , ν t ) at time t . As X t = fct ( ǫ i , α i , i ≤ t ) , f t ( X t , α t , ν t ) = c t ( ǫ 0 : t , α 0 : t , ν t ) , hence total cost = E [ � T t = 0 c t ( ǫ 0 : t , α 0 : t , ν t )] . ֒ → Fits into previous framework, by reading “noises as types”.

Hidden/no dynamics State dynamics Conclusions McKean-Vlasov control problem • With the above example in mind, we will consider McKean-Vlasov control problem: �� T �� � X t , α t , P ◦ ( X t , α t ) − 1 � � ˜ X T , P ◦ X − 1 α E P dt + ˜ inf f t g T 0 subject to � � X t , α t , P ◦ X − 1 dX t = b t dt + dW t t • Let us fist mention connections to large systems of interacting controlled state processes