Hamilton-Jacobi-Bellman equations in infinite dimensions Marco - PowerPoint PPT Presentation

Hamilton-Jacobi-Bellman equations in infinite dimensions Marco Fuhrman Politecnico di Milano Fausto Gozzi LUISS University, Rome Spring School “Stochastic Control in Finance” March 7-18 2010, Roscoff

Preliminary results and general framework. 1. A summary of stochastic integration in Hilbert spaces 2. Deterministic and stochastic evolution equations 3. Basic examples: the heat equation, delay equations 4. The optimal control problem 5. The Hamilton-Jacobi-Bellman equation and a verification theorem 1

Spaces and operators H , K denote Hilbert spaces. All Hilbert spaces are assumed to be real sepa- rable. Scalar product is denoted �· , ·� . L ( K, H ) is the space of linear bounded operators T : K → H . L ( H ):= L ( H, H ). L 2 ( K, H ) is the subspace of Hilbert-Schmidt operators, i.e. of all T ∈ L ( K, H ) such that ∞ � � T � 2 � Te i � 2 L 2 ( K,H ) = H < ∞ , i =1 where ( e i ) is an arbitrary basis of K (i.e. a complete orthonormal system). L 2 ( K, H ) is a separable Hilbert space with scalar product ∞ ∞ � � � S ∗ Te i , e i � H = Trace[ S ∗ T ] . � T, S � L 2 ( K,H ) = � Te i , Se i � H = i =1 i =1 Other notation: • L 2 (Ω; K ) is the Hilbert space of random variables X : Ω → K on a probability space (Ω , F , P ) such that � X � 2 L 2 (Ω; K ) = E � X � 2 K < ∞ . • C ( I ; K ), for I ⊂ R , is the space of continuous functions f : I → K . • etc. 2

Wiener process in Hilbert spaces On a probability space (Ω , F , P ) take a sequence of independent standard brownian motions β i = ( β i t ) t ≥ 0 , i ∈ N . Given a basis of K , a cylindrical Wiener process ( W t ) t ≥ 0 in K is defined as ∞ � β i W t = t e i , t ≥ 0 . i =1 The series is convergent (in L 2 (Ω; K 1 ) and P -a.s.) in an arbitrary Hilbert space K 1 such that K ⊂ K 1 with Hilbert-Schmidt embedding. For G ∈ L ( K ) define ∞ � β i t ≥ 0 . GW t = t Ge i , i =1 Then the series converges (in L 2 (Ω; K ) and P -a.s.) if and only if G ∈ L 2 ( K ). Suppose Ge i = √ λ i e i for a basis ( e i ) and numbers 0 ≤ λ i ≤ sup i λ i < ∞ . Then ∞ � � β i GW t = λ i e i , t ≥ 0 t i =1 and the series converges as above if and only if � i λ i < ∞ ⇐ ⇒ G ∈ L 2 ( K ). This happens in particular if λ i = 0 for all i large (finite-dimensional Wiener process). 3

Stochastic integrals in Hilbert spaces Let ( W t ) be a cylindrical Wiener process in K . One can define the stochastic integral � t I t = Φ s dW s , t ∈ [0 , T ] 0 as a process in H , under the following conditions: 1) (Φ t ) t ∈ [0 ,T ] is a stochastic process in L 2 ( K, H ) (Φ t ( ω ) ∈ L 2 ( K, H )). 2) (Φ t ) is progressive with respect to ( F t ) t ≥ 0 , the natural completed filtration of W : denoting by N the P -null sets, F 0 t = σ ( β i F t = σ ( F 0 s : s ∈ [0 , t ] , i ∈ N ) , t , N ) . 3) � T 0 � Φ s � 2 L 2 ( K,H ) ds < ∞ , P -a.s. Then ( I t ) t ∈ [0 ,T ] is a stochastic process in H with continuous paths and it is a local martingale: there exist ( F t ) stopping times τ n ↑ ∞ such that the stopped processes Φ t ∧ τ n , t ∈ [0 , T ] are martingales in H . 4

If (Φ t ) is an ( F t )-progressive process in L 2 ( K, H ) satisfying the stronger con- dition � T � Φ s � 2 L 2 ( K,H ) ds < ∞ , (1) E 0 then ( I t ) t ∈ [0 ,T ] is a mean-zero, continuous martingale in H and the Ito isometry holds: � � � T � T 2 � � � � � Φ s � 2 Φ s dW s = E E L 2 ( K,H ) ds. � � 0 0 H Three basic tools: representation theorem, Ito’s formula, Girsanov’s theorem. • Representation theorem: if (Ψ t ) is an ( F t )-martingale in H and Ψ T ∈ L 2 (Ω; H ) then � t Ψ t = Ψ 0 + Φ s dW s , t ∈ [0 , T ] , 0 for an appropriate ( F t )-progressive process (Φ t ) with values in L 2 ( K, H ) sat- isfying (1). • The Ito formula will be recalled later, when needed, in appropriate form. 5

• The Girsanov theorem: let ( u t ) t ∈ [0 ,T ] be an ( F t )-progressive process in K satisfying � T 0 � u s � 2 K ds < ∞ , P -a.s. Define �� t � � t s dW s − 1 u ∗ � u s � 2 ρ t = exp K ds , t ∈ [0 , T ] , 2 0 0 where u ∗ s ( ω ) denotes k �→ � u s ( ω ) , k � U , belonging to L 2 ( K, R ). If E ρ T = 1 then ( ρ t ) t ∈ [0 ,T ] is a martingale and the process � t ¯ W t = W t − t ∈ [0 , T ] , u s ds, 0 is a cylindrical Wiener process in K with respect to the probability Q defined on (Ω , F ) by the formula Q ( dω ) = ρ T ( ω ) P ( dω ). 6

Evolution equations The “obvious” generalization for an SDE in H would be: X 0 = x ∈ H, dX t = F ( X t ) dt + G ( X t ) dW t , for an unknown process ( X t ) t ∈ [0 ,T ] in H . W is a cylindrical Wiener process in K , the equation is understood as � t � t X t = x + F ( X s ) ds + G ( X s ) dW s , t ∈ [0 , T ] , 0 0 and F : H → H, G : H → L 2 ( K, H ) , are appropriate coefficients. However, the useful form of the equation is dX t = AX t dt + F ( X t ) dt + G ( X t ) dW t , X 0 = x ∈ H, where A is a linear unbounded operator in H : A : D ( A ) → H, D ( A ) ⊂ H. So we will first address the simpler equation dX t = AX t , X 0 = x ∈ H, i.e. the deterministic abstract Cauchy problem d dty ( t ) = Ay ( t ) , y (0) = x ∈ H, with unknown y : [0 , T ] → H . 7

Semigroups of operators d dty ( t ) = Ay ( t ) , y (0) = x ∈ H, t ∈ [0 , T ] . If A ∈ L ( H ) then the solution is given by the power series formula ∞ � t n y ( t ) = e tA x = n ! A n x. n =0 Setting S ( t ) = e tA one has, for t, s ≥ 0, x ∈ H , S (0) = I, S ( t + s ) = S ( t ) S ( s ) , S ( t ) x → x in H as t → 0 . (2) Note that, for all x ∈ H , S ( t ) x − x Ax = lim in H. (3) t t → 0 Definition. ( S ( t )) t ≥ 0 ⊂ L ( H ) is called a strongly continuous semigroup of linear bounded operators on H if (2) holds. Its infinitesimal generator is the operator A given by (3) and defined on D ( A ) = { x ∈ H : the limit (3) exists in H } . Note that in general D ( A ) � H and A is not H -continuous on D ( A ). Standing notation: from now on, A denotes the generator of a semigroup S and we use the exponential notation e tA instead of S ( t ) (even if the power series formula does not hold). 8

Deterministic evolution equations Given a generator A , we ask whether the function y ( t ) = e tA x is a solution to the homogeneous abstract Cauchy problem d y (0) = x ∈ H, t ∈ [0 , T ] . dty ( t ) = Ay ( t ) , One can prove that if x ∈ D ( A ) then y is a strict solution, i.e. y ∈ C 1 ([0 , T ]; H ), y ( t ) ∈ D ( A ) and the equation holds. The strict solution is unique. If only x ∈ H then y ∈ C ([0 , T ]; H ) and it is called mild solution. Given a generator A and f : [0 , T ] → H , we consider the nonhomogeneous abstract Cauchy problem d y (0) = x ∈ H, t ∈ [0 , T ] . dty ( t ) = Ay ( t ) + f ( t ) , One can prove that if f ∈ C 1 ([0 , T ]; H ) , x ∈ D ( A ) , then there exists a unique strict solution given by the variation of costants formula � t y ( t ) = e tA x + e ( t − s ) A f ( s ) ds, 0 i.e. y ∈ C 1 ([0 , T ]; H ), y ( t ) ∈ D ( A ) and the equation holds. If only x ∈ H and f ∈ C ([0 , T ]; H ) (or even f ∈ L 1 ([0 , T ]; H )) then y ∈ C ([0 , T ]; H ) is called mild solution. 9

Example 1: the heat semigroup We take O ⊂ R d open bounded with smooth boundary and set H = L 2 ( O ) . Thus x ∈ H is a real function x ( ξ ) , ξ ∈ O in L 2 ( O ), and y : [0 , T ] → H is a real function y ( t, ξ ) , t ∈ [0 , T ] , ξ ∈ O such that y ( t, · ) ∈ L 2 ( O ). We define an unbounded linear operator in H = L 2 ( O ) setting D ( A ) = H 2 ( O ) ∩ H 1 A = ∆ ξ , 0 ( O ) , where ∆ ξ is the Laplace operator with respect to the space variable ξ ∈ O . The equation d dty ( t ) = Ay ( t ) , y (0) = x ∈ H, t ∈ [0 , T ] is an abstract form of the heat equation with homogeneous Dirichlet boundary conditions:   ξ ∈ O , t ∈ [0 , T ] , ∂ t y ( t, ξ ) = ∆ ξ y ( t, ξ ) , ξ ∈ O , y (0 , ξ ) = x ( ξ ) ,  ξ ∈ ∂ O , t ∈ [0 , T ] . y ( t, ξ ) = 0 , A is a positive self-adjoint operator in H and generate a semigroup. There exists a basis ( e i ) of H and numbers 0 < α i ↑ ∞ such that Ae i = − α i e i , and we have e tA x = � i e − α i t � x, e i � H e i for all x ∈ H, t ≥ 0 . 10

Stochastic evolution equations: a special case Consider the linear equation with additive noise (Langevin equation) dX t = AX t dt + G dW t , X 0 = x ∈ H, t ∈ [0 , T ] , where A is a generator in H , W is a cylindrical Wiener process in K , G ∈ L ( K, H ). We define the mild solution as � t e ( t − s ) A G dW s , X t = e tA x + 0 provided the (deterministic) integrand Φ s = e ( t − s ) A G satisfies ( P -a.s.) � T � T � Φ s � 2 � e sA G � 2 L 2 ( K,H ) ds = L 2 ( K,H ) ds < ∞ . 0 0 X is called the Ornstein-Uhlenbeck process in H . One proves that if there exists γ ∈ [0 , 1 / 2) and K > 0 such that � e tA G � L 2 ( K,H ) ≤ Kt − γ , t ∈ (0 , T ] . then X has continuous paths in H , P -a.s. This condition always holds (with γ = 0) if G ∈ L 2 ( K, H ). 11