Ergodic BSDEs and Ergodic Optimal Control Ying Hu (IRMAR - Universit - PowerPoint PPT Presentation

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs and Ergodic Optimal Control Ying Hu (IRMAR - Universit´ e Rennes 1) joint work with Arnaud Debussche (IRMAR, ENS Cachan) Marco Fuhrman (Politecnico Milano) Gianmario Tessitore (Bicocca Milano) Tamerza, October 2010

Ergodic BSDEs and Ergodic Optimal Control Setting of the control problem and some references Ergodic Control Problem We address the following optimal control problem with State equation dX x , u = ( AX x , u + F ( X x , u X x , u )) dt + GdW t + GR ( u t ) , = x t t t 0 Cost functional � T 1 L ( X x , u J ( x , u ) = lim sup , u s ) ds . T E s T →∞ 0 Main features ergodic cost functional infinite dimensional equation (Banach space valued) possibly degenerate G

Ergodic BSDEs and Ergodic Optimal Control Setting of the control problem and some references Very incomplete list of references BSDEs and infinite horizon stochastic control P. Briand and Y. Hu, J. Funct. Anal. (1998) (Finite dimensions - all positive discounts) M. Fuhrman and G. Tessitore, Ann. Probab. (2004) (Infinite dimensions - only large discounts) F. Masiero, A.M.O. (2007), (Banach spaces)

Ergodic BSDEs and Ergodic Optimal Control Setting of the control problem and some references Ergodic stochastic control A. Bensoussan and J. Frehse, J. Reine Angew. Math. (1992) (Finite dimensions, classical solutions of HJB) M. Arisawa, P. L. Lions, Comm. Partial Differential Equations (1998) (Finite dimensions, viscosity solutions of HJB) B. Goldys and B. Maslowski, J. Math. Anal. Appl. , (1999) (Infinite dimensions, mild solutions of HJB, smoothing of Kolmogorov semigroup)

Ergodic BSDEs and Ergodic Optimal Control Forward Equation Forward (state) equation � dX t = AX t dt + F ( X t ) dt + GdW t , t ≥ 0 , X 0 = x ∈ E . E Banach, E ⊂ H Hilbert space H . A generates a C 0 semigroup in E that has an extension to H . W is a cylindrical Wiener process in the Hilbert space Ξ F : E → E is continuous and has polynomial growth. A + F is strictly dissipative (with constant η ). G is bdd. Ξ → H . The stochastic convolution � t W A t = S ( t − s ) GdW s , t ≥ 0 , 0 has an E -continuous version with sup t E | W A t | 2 E < ∞ .

Ergodic BSDEs and Ergodic Optimal Control Forward Equation Results on the forward (state) equation � dX x t = AX x t dt + F ( X x t ) dt + GdW t , t ≥ 0 , X x 0 = x ∈ E . ∀ x ∈ E there exists a unique E continuous mild solution X x . t | ≤ e − η t | x 1 − x 2 | , t ≥ 0 , x 1 , x 2 ∈ E . Moreover | X x 1 − X x 2 t Finally sup t E | X x t | E ≤ C (1 + | x | ).

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Ergodic BSDEs (EBSDEs) � T � T Y x t = Y x [ ψ ( X x σ , Z x Z x T + σ ) − λ ] d σ − σ dW σ , 0 ≤ t ≤ T < ∞ , t t or equivalently − dY x t = [ ψ ( X x t , Z x t ) − λ ] dt − Z x t dW t A solution is a triple ( Y , Z , λ ). λ is a real number. Y is a real continuous prog. meas. process such that E sup t ∈ [0 , T ] Y 2 s < ∞ , ∀ T > 0 Z is a prog. meas. process with values in Ξ ∗ such that � T 0 | Z s | 2 Ξ ∗ < ∞ , ∀ T > 0 . E

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Main Result Main Result On the function ψ : E × Ξ ∗ → R we assume: | ψ ( x , z ) − ψ ( x ′ , z ′ ) | ≤ K x | x − x ′ | + K z | z − z ′ | , x , x ′ ∈ E , z , z ′ ∈ Ξ ∗ . ψ ( · , 0) is bounded. Theorem (Existence of solutions for EBSDEs) ∃ λ ∈ R ; ∃ v : E → R Lipschitz (v (0) = 0 ); ∃ ζ : E → Ξ ∗ measurable such that if we set ¯ t ) , ¯ Y x t := v ( X x Z x t := ζ ( X x t ) then ( ¯ Y x , ¯ Z x , λ ) is a solution of the EBSDE.

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Proof of main result Sketch of the proof Considering with strictly monotonic drift α > 0: � T � T Y x ,α = Y x ,α ( ψ ( X x σ , Z x ,α ) − α Y x ,α Z x ,α + ) d σ − dW σ . t σ σ σ T t t Lemma (Briand-Hu 1998, Royer 2004) ∃ ! solution ( Y x ,α , Z x ,α ) Y x ,α bounded cont., Z x ,α ∈ L 2 P , loc . Moreover | Y x ,α | ≤ M /α , P -a.s. for all t ≥ 0 . t Define v α ( x ) = Y α, x . Clearly, | v α ( x ) | ≤ M /α and Y α, x = v α ( X x t ) t 0

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Proof of main result x , x ′ ∈ E . Claim | v α ( x ) − v α ( x ′ ) | ≤ K x η | x − x ′ | , Proof of claim Set Y = Y α, x − Y α, x ′ , ˜ Z = Z α, x − Z α, x ′ , ˜ � � ∗ t , Z α, x ′ β t = ψ ( X x ′ ) − ψ ( X x ′ t , Z α, x ) − Z α, x ′ Z α, x t t , notice β bdd. t t | Z α, x − Z α, x ′ | 2 t t Ξ ∗ t , Z x ,α t , Z x ,α ) − ψ ( X x ′ f t = ψ ( X x ) . t t � t ∃ ˜ P under which ˜ W t = 0 β s ds + W t is a Wiener process. � T � T � T ⇒ ˜ Y t = ˜ ˜ Z σ d ˜ ˜ = Y T − α Y σ d σ + t f σ d σ − W σ . t t E F t � T | ˜ Y t | ≤ e − α ( T − t ) ˜ E F t | ˜ Y T | + ˜ t e − α ( s − t ) | f s | ds = ⇒ Since ˜ Y is bdd and | f t | ≤ K x e − η t | x − x ′ | (by dissip.of forw. equat.) if T → ∞ we get | ˜ Y t | ≤ K x ( η + α ) − 1 e α t | x − x ′ | . �

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Proof of main result Proof of main result Set v α ( x ) = v α ( x ) − v α (0) , We know | v α ( x ) | ≤ K x η − 1 | x | ; α | v α (0) | ≤ M ; { v α } unif. Lip. ∃ α n ց 0 such that v α n ( x ) → v ( x ) , ∀ x and α n v α n (0) → λ . = ⇒ x = v ( X x ), then x ,α = Y x ,α − v α (0) = v α ( X x Define Y t ) and Y t t � T x ,α n x ,α n T | 2 → 0 x x t | 2 dt → 0 E | Y − Y and E | Y − Y t T 0 x ∈ L 2 P , loc (Ω; L 2 (0 , ∞ ; Ξ)) s. t. By standard BSDE arguments ∃ Z � T | Z x ,α n x t | 2 E − Z Ξ ∗ dt → 0 t 0

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Proof of main result x ,α verifies Finally we remark that Y � T � T x ,α x ,α x ,α ( ψ ( X x σ , Z x ,α σ − α v α (0)) d σ − Z x ,α Y = Y T + ) − α Y dW σ . t σ σ t t Now we can pass to the limit as n → ∞ to obtain � T � T x x x x ( ψ ( X x Y t = Y T + σ , Z σ ) − λ ) d σ − Z σ dW σ . t t The construction of ζ : E → Ξ ∗ such that Z x t = ζ ( X x t ), x ,α . � exploits the fact that the same holds for Z

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Remarks on uniqueness Uniqueness of λ x , Z x , λ ) we have constructed verifies The solution ( Y x t | ≤ c | X x | Y t | . If we require similar conditions then we immediately obtain uniqueness of λ . Theorem Suppose that, for some x ∈ E, ( Y ′ , Z ′ , λ ′ ) is a solution of (EBSDE) and verifies | Y ′ t | ≤ c x ( | X x t | + 1) , for all t ≥ 0 . Then λ ′ = λ .

Ergodic BSDEs and Ergodic Optimal Control Ergodic BSDEs Remarks on uniqueness Lack of uniqueness of EBSDEs Clearly if ( Y , Z , λ ) is a solution then ( Y + c , Z , λ ) is a solution. Even if we ask Y 0 0 = 0 the solution to EBSDE is, not unique. If we do not require Y t = v ( X x t ), Z t = ζ ( X x t ) then can construct several solutions of the above EBSDE (with Y and Z bounded). If we require Y t = v ( X x t ), Z t = ζ ( X x t ) with v and ζ continuous and X x to be recursive (see [Seidler 1997]) then v can be characterized (as in [Goldys-Maslowski 1999]) by: � τ T r [ ψ ( X x , u , u ( X x , u v ( x ) = inf u lim sup lim sup )) − λ ] ds . E s s r → 0 T →∞ 0 r = inf { s ∈ [0 , T ] : | X u , x where τ T | < r } . s

Ergodic BSDEs and Ergodic Optimal Control Optimal Ergodic Control Optimal Ergodic Control problem Let X x be the solution to equation t = ( AX x , u + F ( X x , u X x , u dX x )) dt + GdW t , = x t t 0 An admissible control u is a progressively measurable process with values in a Borel subset U of a complete metric space. The ergodic cost corresponding to u and the starting point x ∈ E is � T 1 T E u , T L ( X x J ( x , u ) = lim sup s , u s ) ds , T →∞ 0 where �� T � � T 0 R ( u s ) dW s − 1 0 | R ( u s ) | 2 ρ u P u T = ρ u T = exp Ξ ∗ ds , T P . 2 Where R : U → R , L : U × E → R with R , L bdd in u ; L Lip. in x .

Ergodic BSDEs and Ergodic Optimal Control Optimal Ergodic Control Ergodic control and EBSDEs We first define the Hamiltonian in the usual way x ∈ E , z ∈ Ξ ∗ . ψ ( x , z ) = inf u ∈ U { L ( x , u ) + zR ( u ) } , Under the present assumptions ψ is a Lipschitz function and ψ ( · , 0) is bounded thus the EBSDE − dY x t = [ ψ ( X x t , Z x t ) − λ ] dt − Z x t dW t has at least a solution ( Y x , Z x , λ )

Ergodic BSDEs and Ergodic Optimal Control Optimal Ergodic Control Synthesis of Optimal control Theorem Suppose that, for some x ∈ E, a triple ( Y , Z , λ ) verifies EBSDE and | Y x t | ≤ c x ( | X x t | + 1) , for all t ≥ 0 . Then the following holds: (i) For arbitrary control u we have J ( x , u ) ≥ λ and the equality holds if and only if L ( X x t , u t ) + Z t R ( u t ) = ψ ( X x t , Z t ) . (ii) If the infimum in the definition of ψ is attained at u = γ ( x , z ) u t = γ ( X x then the control ¯ t , Z t ) verifies J ( x , ¯ u ) = λ. Recall that λ is univocally determined.