Semi-Lagrangian schemes for linear and fully non-linear - PowerPoint PPT Presentation

Introduction SL schemes for HJB Conclusion Semi-Lagrangian schemes for linear and fully non-linear Hamilton-Jacobi-Bellman equations Kristian Debrabant 1 ,Espen R. Jakobsen 2 1 Department of Mathematics and Computer Science, University of Southern Denmark 2 Department of Mathematical Sciences, Norwegian University of Science And Technology, Trondheim, Norway Padova, June 28, 2012 1

Introduction SL schemes for HJB Conclusion Outline Introduction 1 Semi-Lagrangian schemes for Hamilton-Jacobi-Bellman 2 equations Conclusion 3 2

Introduction SL schemes for HJB Conclusion Outline Introduction 1 Semi-Lagrangian schemes for Hamilton-Jacobi-Bellman 2 equations Conclusion 3 3

Introduction SL schemes for HJB Conclusion Stochastic control problem = ⇒ HJB equation M � � � � � dX ( t ) = b t , X ( t ) , α ( t ) dt + t , X ( t ) , α ( t ) dW i ( t ) , t 0 ≤ t ≤ T , σ i i = 1 R N . X ( t 0 ) = X 0 ∈ I � � T �� � � � u ( t 0 , X 0 ) = inf α ∈ A E β ( t 0 , t ) f t , X ( t ) , α ( t ) dt + β ( t 0 , T ) g X ( T ) t 0 where � t � � t 0 c s , X ( s ) ,α ( s ) ds . β ( t 0 , t ) = e Under appropriate assumptions: u solves HJB equation � � − u t ( t , x ) − inf L [ u ]( t , x , α ) + c ( t , x , α ) u ( t , x ) + f ( t , x , α ) = 0 α ∈A R N , R N , in [ 0 , T ) × I u ( T , x ) = g ( x ) in I L [ u ]( t , x , α ) = 1 2 Tr [ σ ( t , x , α ) σ ⊤ ( t , x , α ) D 2 u ( t , x )] + b ( t , x , α ) Du ( t , x ) 4

Introduction SL schemes for HJB Conclusion Stochastic control problem = ⇒ HJB equation M � � � � � dX ( t ) = b t , X ( t ) , α ( t ) dt + t , X ( t ) , α ( t ) dW i ( t ) , t 0 ≤ t ≤ T , σ i i = 1 R N . X ( t 0 ) = X 0 ∈ I � � T �� � � � u ( t 0 , X 0 ) = inf α ∈ A E β ( t 0 , t ) f t , X ( t ) , α ( t ) dt + β ( t 0 , T ) g X ( T ) t 0 where � t � � t 0 c s , X ( s ) ,α ( s ) ds . β ( t 0 , t ) = e Under appropriate assumptions: u solves HJB equation � � − u t ( t , x ) − inf L [ u ]( t , x , α ) + c ( t , x , α ) u ( t , x ) + f ( t , x , α ) = 0 α ∈A R N , R N , in [ 0 , T ) × I u ( T , x ) = g ( x ) in I L [ u ]( t , x , α ) = 1 2 Tr [ σ ( t , x , α ) σ ⊤ ( t , x , α ) D 2 u ( t , x )] + b ( t , x , α ) Du ( t , x ) 4

Introduction SL schemes for HJB Conclusion Stochastic control problem = ⇒ HJB equation M � � � � � dX ( t ) = b t , X ( t ) , α ( t ) dt + t , X ( t ) , α ( t ) dW i ( t ) , t 0 ≤ t ≤ T , σ i i = 1 R N . X ( t 0 ) = X 0 ∈ I � � T �� � � � u ( t 0 , X 0 ) = inf α ∈ A E β ( t 0 , t ) f t , X ( t ) , α ( t ) dt + β ( t 0 , T ) g X ( T ) t 0 where � t � � t 0 c s , X ( s ) ,α ( s ) ds . β ( t 0 , t ) = e Under appropriate assumptions: u solves in the viscosity sense � � − u t ( t , x ) − inf L [ u ]( t , x , α ) + c ( t , x , α ) u ( t , x ) + f ( t , x , α ) = 0 α ∈A R N , R N , in [ 0 , T ) × I u ( T , x ) = g ( x ) in I L [ u ]( t , x , α ) = 1 2 Tr [ σ ( t , x , α ) σ ⊤ ( t , x , α ) D 2 u ( t , x )] + b ( t , x , α ) Du ( t , x ) 4

Introduction SL schemes for HJB Conclusion Convergence to viscosity solution via Lax type result Barles & Souganidis 1991: The numerical approximation U obtained by the scheme S (∆ t , ∆ x , n , j , u n j , u ) = 0 converges uniformly to the viscosity solution if at least for some sequence (∆ t , ∆ x ) converging to zero it is stable: For all (∆ t , ∆ x ) there exists a solution U with a bound independent of (∆ t , ∆ x ) . consistent: S (∆ t , ∆ x , n , j , Φ n j + ξ, Φ + ξ ) t , x , Φ( t , x ) , Φ t ( t , x ) , D Φ( t , x ) , D 2 Φ( t , x ) � � lim → F ρ (∆ t , ∆ x ) ξ → 0 ∆ t , ∆ x → 0 ( n ∆ t , j ∆ x ) → ( t , x ) for some positive function ρ , any smooth function Φ and any ( x , t ) monotone: S (∆ t , ∆ x , n , j , u n j , u ) ≤ S (∆ t , ∆ x , n , j , v n j , v ) if u ≥ v , u n j = v n j , for any ∆ t , ∆ x , n , j , u and v Oberman (2006), Pooley, Forsyth, Vetzal (2003): Nonmonotone methods need not converge or even converge to a wrong function 5

Introduction SL schemes for HJB Conclusion Outline Introduction 1 Semi-Lagrangian schemes for Hamilton-Jacobi-Bellman 2 equations Conclusion 3 6

Introduction SL schemes for HJB Conclusion Proposed family of SL schemes � � PDE: u t ( t , x ) − inf L [ u ]( t , x , α ) + c ( t , x , α ) u ( t , x ) + f ( t , x , α ) = 0 α ∈A R N , R N , in ( 0 , T ] × I u ( 0 , x ) = g ( x ) in I L [ u ]( t , x , α ) = 1 2 Tr [ σ ( t , x , α ) σ ⊤ ( t , x , α ) D 2 u ( t , x )] + b ( t , x , α ) Du ( t , x ) Semi-discretization in ( 0 , T ) × X ∆ x : � � U t ( t , x ) − inf L k [ I U ]( t , x , α ) + c ( t , x , α ) U ( t , x ) + f ( t , x , α ) = 0 , α ∈A t , x + y − P � t , x + y + � � � k , i ( t , x , α ) − 2 ϕ ( t , x ) + ϕ k , i ( t , x , α ) ϕ � L k [ ϕ ]( t , x , α ) := 2 k 2 i = 1 P � [ y + k , i + y − k , i ] = 2 k 2 b + O ( k 4 ) , such that i = 1 P k , i ] = 2 k 2 σσ ⊤ + O ( k 4 ) = � [ y + k , i y + ⊤ + y − k , i y − ⊤ ⇒ k | σ | ∼ stencil length k , i i = 1 P P � [ ⊗ 3 j = 1 y + k , i + ⊗ 3 j = 1 y − k , i ] = O ( k 4 ) , � [ ⊗ 3 j = 1 y + k , i + ⊗ 3 j = 1 y − k , i ] = O ( k 4 ) i = 1 i = 1 7

Introduction SL schemes for HJB Conclusion Proposed family of SL schemes � � PDE: u t ( t , x ) − inf L [ u ]( t , x , α ) + c ( t , x , α ) u ( t , x ) + f ( t , x , α ) = 0 α ∈A R N , R N , in ( 0 , T ] × I u ( 0 , x ) = g ( x ) in I L [ u ]( t , x , α ) = 1 2 Tr [ σ ( t , x , α ) σ ⊤ ( t , x , α ) D 2 u ( t , x )] + b ( t , x , α ) Du ( t , x ) Semi-discretization in ( 0 , T ) × X ∆ x : � � U t ( t , x ) − inf L k [ I U ]( t , x , α ) + c ( t , x , α ) U ( t , x ) + f ( t , x , α ) = 0 , α ∈A t , x + y − P � t , x + y + � � � k , i ( t , x , α ) − 2 ϕ ( t , x ) + ϕ k , i ( t , x , α ) ϕ � L k [ ϕ ]( t , x , α ) := 2 k 2 i = 1 � D ϕ � ∞ + � D 2 ϕ � ∞ + � D 3 ϕ � ∞ + � D 4 ϕ � ∞ k 2 � � = ⇒ | L k [ ϕ ] − L [ ϕ ] | ≤ C Important for monotonicity: I monotone, i. e. � ( I ϕ )( x ) = ϕ ( x j ) w j ( x ) , w i ( x j ) = δ ij , and w j ( x ) ≥ 0 for all i , j ∈ N j In general: Not more than second order accurate interpolation ( I linear) 7

Introduction SL schemes for HJB Conclusion Proposed family of SL schemes � � PDE: u t ( t , x ) − inf L [ u ]( t , x , α ) + c ( t , x , α ) u ( t , x ) + f ( t , x , α ) = 0 α ∈A R N , R N , in ( 0 , T ] × I u ( 0 , x ) = g ( x ) in I L [ u ]( t , x , α ) = 1 2 Tr [ σ ( t , x , α ) σ ⊤ ( t , x , α ) D 2 u ( t , x )] + b ( t , x , α ) Du ( t , x ) Semi-discretization in ( 0 , T ) × X ∆ x : � � U t ( t , x ) − inf L k [ I U ]( t , x , α ) + c ( t , x , α ) U ( t , x ) + f ( t , x , α ) = 0 , α ∈A t , x + y − P � t , x + y + � � � k , i ( t , x , α ) − 2 ϕ ( t , x ) + ϕ k , i ( t , x , α ) ϕ � L k [ ϕ ]( t , x , α ) := 2 k 2 i = 1 � D ϕ � ∞ + � D 2 ϕ � ∞ + � D 3 ϕ � ∞ + � D 4 ϕ � ∞ k 2 � � = ⇒ | L k [ ϕ ] − L [ ϕ ] | ≤ C Important for monotonicity: I monotone, i. e. � � ( I ϕ )( x ) = ϕ ( x j ) w ϕ, j ( x ) , w ϕ, i ( x j ) = δ ij , w ϕ, j ( x ) ≥ 0 , w ϕ, j ( x ) ≡ 1 j j In general: Not more than second order accurate interpolation ( I linear) But: Higher order monotonicity preserving interpolation is possible if ϕ is known to be monotone → nonlinear in ϕ 7

Introduction SL schemes for HJB Conclusion A unifying framework - examples √ ∆ x , y ± k = k 2 b The approximation of Falcone (1987): k = 1 bD ϕ ≈ I ϕ ( x + ∆ xb ) − I ϕ ( x ) ∆ x √ k , j = ± k σ j + k 2 ∆ x , y ± Camilli-Falcone (1995): k = M b 2 1 2 Tr [ σσ ⊤ D 2 ϕ ] + bD ϕ √ √ M ∆ x σ j + ∆ x ∆ x σ j + ∆ x I ϕ ( x + M b ) − 2 I ϕ ( x ) + I ϕ ( x − M b ) � ≈ 2 ∆ x j = 1 New version (efficient for σ independent of α ): 3 1 2 Tr [ σσ ⊤ D 2 ϕ ] + bD ϕ M ≈ I ϕ ( x + k 2 b ) − I ϕ ( x ) I ϕ ( x + k σ j ) − 2 I ϕ ( x ) + I ϕ ( x − k σ j ) � + k 2 2 k 2 j = 1 8

Introduction SL schemes for HJB Conclusion Monotonicity preserving cubic Hermite interpolation (Fritsch & Carlson 1980, Eisenstat, Jackson & Lewis 1985) On [ x i , x i + 1 ] : ( I ϕ )( x ) = c 0 + c 1 ( x − x i ) + c 2 ( x − x i ) 2 + c 3 ( x − x i ) 3 with ( I ϕ )( x i ) = ϕ i , ( I ϕ )( x i + 1 ) = ϕ i + 1 , ( I ϕ ) ′ ( x i ) ≈ ϕ ′ i , ( I ϕ ) ′ ( x i + 1 ) ≈ ϕ ′ i + 1 ϕ i − 2 − 8 ϕ i − 1 + 8 ϕ i + 1 − ϕ i + 2 ϕ i + 1 − ϕ i d i + 1 β i = d i d i = , ∆ i = , ∆ i , γ i = ∆ i . 12 ∆ x ∆ x Use ( I ϕ ) ′ ( x i ) = β i ∆ i , ( I ϕ ) ′ ( x i + 1 ) = γ i ∆ i = ⇒ ( I ϕ )( x ) = ϕ i + ( ϕ i + 1 − ϕ i ) P i ( x ) where � 2 � 3 P i ( x ) = β i x − x i � x − x i � x − x i + ( 3 − γ i − 2 β i ) − ( 2 − β i − γ i ) . ∆ x ∆ x ∆ x = ⇒ w ϕ, i ( x ) = ( 1 − P i ( x )) 1 [ x i , x i + 1 ) ( x ) + P i − 1 ( x ) 1 [ x i − 1 , x i ) ( x ) γ i 4 3 Modify β i and γ i such that ( β i , γ i ) ∈ M : 2 M 1 ⇒ fourth order accurate monotonicity preserving C 0 = β i 1 2 3 4 interpolant = ⇒ New second order compact stencil schemes 9