Stochastic optimal control problems in Banach spaces Federica - PowerPoint PPT Presentation

Stochastic optimal control problems in Banach spaces Federica Masiero Universit` a Milano Bicocca La Londe 9-14 September 2007

PLAN 1. SDEs in Banach spaces; 2. The forward-backward system; 3. Identification of Z; 4. The optimal control problem; 5. The Hamilton Jacobi Bellman equation; 6. The case of arbitrarly growing coefficents; 7. Bibliographycal comments; 8. Application to nonlinear stochastic heat equations; 9. Application to stochastic delay equations. 1

SDEs in Banach spaces Our framework SDE with values in E ⊂ H , E Banach, H Hilbert � dX τ = [ AX τ + F ( X τ )] dτ + GdW τ , τ ∈ [ t, T ] X t = x, 0 ≤ t ≤ T. Theorem 1: If Hypothesis 1 is satisfied, there exists a unique mild solution X ( τ, t, x ), that is an adapted and continuous E -valued process satisfying P -a.s. � τ � τ X τ = e ( τ − t ) A x + e ( τ − s ) A F ( X s ) ds + e ( τ − s ) A GdW s , τ ∈ [ t, T ] . t t Proof: See e.g. Da Prato and Zabczyk (1992, 1996). 2

SDEs in Banach spaces Hypothesis 1 1. A generates a C 0 semigroup e tA , t ≥ 0 , in E , and there exists ω ∈ R such � � e tA � L ( E,E ) ≤ e ωt , for all t ≥ 0. e tA , t ≥ 0 extends to a C 0 semigroup that � of bounded linear operators in H . 2. F : E → E continuous and ∃ η ≥ 0 s.t. A + F − ηI is dissipative in E . � σ e sA GG ∗ e sA ∗ ds is a trace class operator in H . 3. G ∈ L (Ξ , H ) and Q σ = 0 4. W cylindrical Wiener process in Ξ and W A ( τ ) = � τ t e ( τ − s ) A GdW s admits an E -continuous version. 3

SDEs in Banach spaces Regularity with respect to the initial datum Let H p ([0 , T ] , E ) = � predictable processes: E sup τ ∈ [0 ,T ] | X τ | p E < ∞ � . • X ( τ, t, x ) is Lipschitz in x uniformly with respect to τ : � X ( τ, t, x 1 ) − X ( τ, t, x 2 ) � E ≤ e | η | T � x 1 − x 2 � E • If F is Gateaux differentiable in E , X ( τ, t, · ) is pathwise differentiable. � � 1 + � x � k • Assume that ∃ k ≥ 0 s.t. � F ( x ) � E ≤ c ⇒ E 1. ( X ( τ, t, x )) τ ∈ [0 ,T ] ∈ H p ([0 , T ] , E ) 2. the map x �→ ( X ( τ, t, x )) τ ∈ [0 ,T ] from E to H p ([0 , T ] , E ) is Gateaux differ- entiable. 4

The forward-backward system Forward-backward system  dX τ = AX τ dτ + F ( X τ ) dτ + GdW τ , τ ∈ [ t, T ]  dY τ = − ψ ( τ, X τ , Z τ ) dτ + Z τ dW τ , τ ∈ [ t, T ]  X t = x,   Y T = φ ( X T ) . Hypothesis 2: • For every σ ∈ [0 , T ], x ∈ E and z 1 , z 2 ∈ Ξ ∗ | ψ ( σ, x, z 1 ) − ψ ( σ, x, z 2 ) | ≤ L | z 1 − z 2 | Ξ ∗ . • For every σ ∈ [0 , T ] ψ ( σ, · , · ) ∈ G 1 , 1 ( E × Ξ ∗ ) and for every σ ∈ [0 , T ], x, h ∈ E and z ∈ Ξ ∗ � m � � � |∇ x ψ ( σ, x, z ) h | ≤ L � h � E 1 + � x � E 1 + | z | Ξ ∗ . • φ ∈ G 1 ( E ) and lipschitz continuous on E . � � • F ∈ G 1 ( E ) and ∃ k ≥ 0 s.t. � F ( x ) � E ≤ c 1 + � x � k . E 5

The forward-backward system Proposition Let hypotheses 1 and 2 hold true. Then the BSDE admits a unique solution ( Y, Z ) ∈ K cont ([0 , T ]) and the map ( t, x ) �→ ( Y ( · , t, x ) , Z ( · , t, x )) belongs to G 0 , 1 ([0 , T ] × E, K cont ([0 , T ])). The following estimates holds true: for every p ≥ 2, � 1 /p � � 2 � 1 + � x � ( m +1) |∇ x Y ( τ, t, x ) h | p E sup ≤ C � h � E . E τ ∈ [0 ,T ] Corollary Let hypotheses 1 and 2 hold true. Then the function v ( t, x ) := Y ( t, t, x ) belongs to G 0 , 1 ([0 , T ] × E, R ) and there exists C > 0 such that � 2 � 1 + � x � ( m +1) |∇ x v ( t, x ) h | ≤ C � h � E for all t ∈ [0 , T ], x, h ∈ E . E 6

Identification of Z Theorem 2 Let hypotheses 1 and 2 hold true and set v ( t, x ) := Y ( t, t, x ), Then, for almost every s ∈ [0 , T ], Z s ξ = ∇ v ( s, X s ) Gξ , P -almost everywhere and for every ξ ∈ Ξ 0 . Main technical result. The argument generalizes the one in Bismut, Martin- gales, the Malliavin calculus and hypoellipticity under general H¨ ormander’s conditions . Z. Wahrsch. Verw. Gebiete (1981). More in general it holds true for v ∈ G 0 , 1 ([0 , T ] × E, R ) satisfying � T � T v ( τ, X τ ) = v ( T, X T ) + ψ σ dσ − Z σ dW σ , τ ∈ [ t, T ] . τ τ 7

Identification of Z Hypothesis Assume there exists a Banach subspace Ξ 0 dense in Ξ s.t. G (Ξ 0 ) ⊂ E and G : Ξ 0 − → E is continuous. Theorem 2: Let X be solution of the SDE, Z and ψ be square integrable processes. Let v ∈ G 0 , 1 ([0 , T ] × E ) s.t. for every 0 ≤ t ≤ s ≤ T , |∇ v ( s, x ) h | ≤ � � 1 + � x � j c � h � E , for some integer j ≥ 0 and for every x, h ∈ E. If E � T � T v ( t, x ) + Z σ dW σ = v ( T, X T ) + ψ σ dσ, t t then, for almost every s ∈ [0 , T ], Z s ξ = ∇ v ( s, X s ) Gξ , P -almost everywhere and for every ξ ∈ Ξ 0 . Remark Since Ξ 0 is dense in Ξ, for every ¯ ξ ∈ Ξ there exists a sequence → ¯ ( ξ n ) n ∈ Ξ 0 such that ξ n − ξ in Ξ. For almost every s ∈ [0 , T ] and almost surely with respect to the law of X s , the operator ∇ v ( s, x ) G : Ξ 0 − → E extends to an operator defined in the whole Ξ. So Z s = ∇ v ( s, X s ) G , P -almost surely and for almost every s ∈ [0 , T ] . 8

Identification of Z Proof: Let η be a bounded and predictable process with the following form: let � kT � � kT � 2 n , ( k + 1) T 2 n , ( k + 1) T , k = 0 , .... 2 n − 1 be a partition [0 , T ]. For t ∈ 2 n 2 n � kT � 2 n , ( k + 1) T , 0 ≤ t 1 ≤ ... ≤ t l k ≤ kT 2 n , η k ∈ C ∞ η t = η k � b ( R l k , R ) . � W t 1 , ..., W t lk , t ∈ 2 n For ς ∈ Ξ 0 , set ξ t = η t ς . Notation: ξ t = ξ t ( W · ), where ( W · ) is the trajectory of W up to time t . (see Bismut). 9

Identification of Z � T � T v ( s, X ( s, t, x )) + Z σ dW σ = v ( T, X T ) + ψ σ dσ s s and ,for t ≤ s ≤ T , � s � s v ( s, X ( s, t, x )) = v ( t, x ) + Z σ dW σ − ψ σ dσ. t t � s �� s − δ � s � � � ξ ∗ ξ ∗ E v ( s, X s ) σ dW σ = − E ψ σ dσ σ dW σ s − δ t s − δ �� s � s �� s � s � � ξ ∗ ξ ∗ + E − E ψ σ dσ σ dW σ Z σ dW σ σ dW σ . s − δ s − δ t s − δ ⇓ � s � � 1 ξ ∗ E [ Z s ξ s ] = lim v ( s, X s ) δ E σ dW σ . δ → 0 s − δ 10

Identification of Z Prove that � s � � 1 ξ ∗ lim v ( s, X s ) = E [ ∇ v ( s, X s ) Gξ s ] . δ E σ dW σ δ → 0 s − δ Following Bismut, � σ W ε ξ r ( W ε σ = W σ − ε · ) dr, t W ε σ = W ε σ ( W · ). So � σ W ε ξ r ( W ε σ = W σ − ε · ( W · )) dr, 0 ≤ t ≤ σ ≤ T. t and � σ d dε | ε =0 W ε σ = ξ r ( W · ) dr. t 11

Identification of Z Let � T � T · ( W · )) dW σ − ε 2 � � dQ ε · ( W · )) | 2 dσ ξ ∗ σ ( W ε | ξ σ ( W ε d P = exp ε 2 t t By dominated convergence � s � s � s σ dW σ − ε 2 � � � � �� = d | ξ σ | 2 dσ ξ ∗ ξ ∗ v ( s, X s ) σ dW σ v ( s, X s ) exp ε E dε | ε =0 E 2 t t t = d dε | ε =0 E Q ε [ v ( s, X s )] . 12

Identification of Z Under Q ε X solves � dX τ = AX τ dτ + F ( X τ ) dτ + Gεξ τ dτ + GdW ε τ , τ ∈ [ s − δ, T ] X s − δ = X ( s − δ, t, x ) . Under P X ε solves � dX ε τ = AX ε τ dτ + F ( X ε τ ) dτ + Gεξ τ dτ + GdW τ , τ ∈ [ s − δ, T ] X ε t = X ( s − δ, t, x ) . ⇓ � � dε | ε =0 E Q ε [ v ( s, X s )] = d d · dε | ε =0 E [ v ( s, X ε s )] = E ∇ v ( s, X s ) X s 13

Identification of Z · · · � X τ = A X τ dτ + ∇ F ( X τ ) X τ dτ + Gξ τ dτ, τ ∈ [ s − δ, T ] d · X s − δ = 0 . · X τ = � τ claim: t ∇ X ( τ, σ, X ( σ, t, x )) Gξ σ dσ ⇓ � s � � 1 ξ ∗ E [ Z s ξ s ] = lim ∇ v ( s, X s ) σ dW σ δ E δ → 0 s − δ � s � � 1 = lim δ E ∇ v ( s, X s ) ∇ X ( s, σ, X ( σ, t, x )) Gξ σ dσ δ → 0 s − δ = E [ ∇ v ( s, X s ) ∇ X ( s, s, X ( s, t, x )) Gξ s ] = E [ ∇ v ( s, X s ) Gξ s ] . 14

The optimal control problem The optimal control problem: weak formulation controlled SDE � dX u τ = [ AX u τ + F ( X u τ ) + GR ( τ, X u τ , u τ )] dτ + GdW τ X u t = x ∈ E, τ ∈ [ t, T ] . u ∈ L 2 P (Ω × [0 , T ] , U ) . Cost functional and value function A = (Ω , F , F τ , P , W, u, X u ) admissible control system (a.c.s.). � T g ( s, X u s , u s ) ds + E φ ( X u J ( t, x, A ) = E T ) , t J ∗ ( t, x ) = inf A J ( t, x, A ) . 15

The optimal control problem Hypothesis 3: R : [0 , T ] × H × U − → Ξ measurable. ∀ τ ∈ [0 , T ] , | R ( τ, x, u ) | ≤ K R . φ ∈ G 1 ( E ) and lipschitz continuous on E . g : [0 , T ] × E × U − → R , continuous. ∃ K > 0 s.t. for j ≥ 0, for every x ∈ E � � 1 + � x � j | g ( τ, x, u ) | ≤ K . E Weak formulation of the optimal control problem (see e.g. Fleming-Soner � � 1993): find an a.c.s. A s.t. J t, x, A ≤ J ( t, x, A ) for every a.c.s. A . Then A is optimal. 16