Singular Perturbations in Stochastic Control and - PowerPoint PPT Presentation

Singular Perturbations in Stochastic Control and Hamilton-Jacobi-Bellman Equation Hicham Kouhkouh joint work with Martino Bardi Dipartimento di Matematica “Tullio Levi-Civita” Universit` a di Padova kouhkouh@math.unipd.it IPAM Workshop ”Stochastic Analysis Related to Hamilton-Jacobi PDEs” Los Angeles, May 18-22, 2020 Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 1 / 13

Problem Goal: study the limit 1 as ε → 0 , of the system √ 2 σ ε ( X t , Y t , u t ) dW t , X 0 = x ∈ R n dX t = f ( X t , Y t , u t ) dt + � (SDE( 1 ε )) dY t = 1 2 Y 0 = y ∈ R m ε b ( X t , Y t ) dt + ε̺ ( X t , Y t ) dW t , Assumptions: y · b < − α | y | when | y | ≥ R , and ̺̺ ⊤ bounded Issues:  ∗ High dimension : ∀ n , m ≥ 1      ∗ Controlled dynamics : u t     ∗ Unbounded domain : x ∈ R n , y ∈ R m = ⇒ Can we do something?  ∗ Unbounded data:      | f | , � σ � , | b | ≤ C (1 + | x | + | y | )    ∗ Possible degeneracy of σ and also ̺ Yes, but... 1 Ref.: Bardi, M., & Cesaroni, A. (2011) , and the references therein! Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 2 / 13

Plan V ε ( t , x , y ) SDE ( 1 ( HJB ) ε ε ) Viscosity control prob Ergodicity Effective Ham ??? ??? Homogenization Bellman Ham ( Selection arg . ) control prob HJ ( B ) V ( t , x ) ( ⋆ )? Viscosity Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 3 / 13

Stochastic Control Problem with Singular Perturbations (Ω , F , F t , P ) a complete filtered probability space, ( W t ) t an F t -adapted standard r -dimensional Brownian motion, √ 2 σ ε ( X t , Y t , u t ) dW t , X 0 = x ∈ R n dX t = f ( X t , Y t , u t ) dt + � (1) dY t = 1 2 Y 0 = y ∈ R m ε b ( X t , Y t ) dt + ε̺ ( X t , Y t ) dW t , Pay-off function J : [0 , T ] ∋ ( t , x , y , u ) × R n × R m × U → R , λ > 0 � � � T e λ ( t − T ) g ( X T ) + ℓ ( X s , Y s , u s ) e λ ( s − T ) ds J ( t , x , y , u ) := E x , y , t Value function V ε ( t , x , y ) := sup { J ( t , x , y , u ) , s . t . ( X · , Y · ) in (1) } (2) u ∈U U the set of F t -progressively measurable processes taking values in U . Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 4 / 13

HJB equation A fully nonlinear degenerate parabolic equation in ( 0 , T ) × R n × R m � �  D 2 yy V ε D 2 xy V ε x , y , V ε , D x V ε , D y V ε   − V ε t + F ε , D 2 xx V ε , , = 0 , √ ε ε ε   V ε ( T , x , y ) = g ( x ) , in R n The Hamiltonian F ε : R n × R m × R × R n × R m × S n × S m × M n , m → R is F ε ( x , y , s , p , q , M , N , Z ) := H ε ( x , y , p , M , Z ) − L ( x , y , q , N ) + λ s , where � � H ε ( x , y , p , M , Z ) := min − tr( σ ε σ ε ⊤ M ) − f · p − 2tr( σ ε ̺ ⊤ Z ⊤ ) − ℓ u ∈ U L ( x , y , q , N ) := b · q + tr( ̺̺ ⊤ N ) σ ε , f , b and ℓ are computed at ( x , y , u ) and ̺ = ̺ ( x , y ) Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 5 / 13

Ergodicity √ 2 ̺ ( x , Y y ( · )) dW t , Y y (0) = y ∈ R m , x fixed dY y ( · ) = b ( x , Y y ( · )) dt + It is well known 2 that an invariant measure µ x of Y y ( · ) exists, is unique, has finite moments and Lip. Cont. density 3 w.r.t. x � P Y y ( t ) ( · ) − µ x ( · ) � TV ≤ C (1 + | y | d ) (1 + t ) − (1+ k ) satisfies Moreover, we prove 4 for τ n := inf { t ≥ 0 s.t. � Y y ( t ) � ≥ n } , Lemma � � � � − τ n ≤ C n β e − n η − ∃ η > 0 , ∀ β > 0 , E exp n → + ∞ 0 , ( loc . unif . y ) − − − → n β 2Veretennikov, ”On polynomial mixing and convergence rate for stochastic difference and differential equations.” Theory of Probability & Its Applications 44.2 (2000) 3Pardoux & Veretennikov, ”On Poisson equation and diffusion approximation 2.” The Annals of Probability (2003) 4In the line of proof [Prop.1.4] in: Herrmann, Imkeller, Peithmann, ”Transition times and stochastic resonance for multidimensional diffusions with time periodic drift: A large deviations approach” , Ann. Appl. Probab.(2006) Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 6 / 13

Construct an Effective Hamiltonian � An approximation of the δ -Cell problem: � � n →∞ R m (e.g. D n ball of radius n ). Let { D n } n ⊂ R m , ∂ D n smooth, D n − − − → Consider the Dirichlet-Poisson problem, for h ( y ) := H ( x , y , p , M , 0 ) � δω ( y ) − L ω ( y ) = − h ( y ) , in D n ω ( y ) = 0 , on ∂ D n � � τ n � It has a unique solution ω δ, n ( y ) = E 0 h ( Y y ( t )) e − δ t dt − where τ n is the first exist time of Y y ( · ) from D n . Proposition � n − (4+ α ) � Let δ ( n ) = O , for some α > 0 , the one has � � �� � � � � � δ ( n ) ω δ ( n ) , n ( y ) − � lim − R m h ( y ) d µ ( y ) � = 0 , loc. unif. in y , n →∞ where µ is the unique invariant probability measure for the process Y y ( · ) . Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 7 / 13

Convergence of the value function The effective Hamiltonian is � � � H ( x , p , M ) := R m H ( x , y , p , M , 0) d µ ( y ) The effective � HJB equation is � − V t + H ( x , D x V , D 2 ( t , x ) ∈ ( 0 , T ) × R n xx V ) + λ V ( x ) = 0 , in R n V ( T , x ) = g ( x ) , Theorem The solution V ε to ( HJB ) ε converges uniformly on compact subsets of [0 , T ) × R n × R m to the unique continuous viscosity solution to the limit problem � HJB satisfying a quadratic growth condition in x, i.e. ∃ K > 0 such that | V ( t , x ) | ≤ K (1 + | x | 2 ) , ∀ ( t , x ) ∈ [0 , T ] × R n Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 8 / 13

Control representation of � HJB Proposition Under the standing assumptions, the effective Hamiltonian writes � � � � � − trace ( σσ ⊤ M ) − f · p − ℓ H ( x , p , M ) = min d µ x ( y ) ν ∈U ex ( x ) R m where σ, f and ℓ are computed at ( x , y , u ) , and U ex ( x ) is the set of progressively measurable processes taking values in the extended control set U ex ( x ) := L 2 (( R m , µ x ) , U ) . The extended controls are ν · ( · ) : t �→ ν t ( · ) ∈ L 2 (( R m , µ ˆ X t ) , U ) � � � � � R m | φ ( y ) | 2 d µ ˆ � = φ ( · ) : y �→ φ ( y ) ∈ U X t ( y ) < ∞ � � � � This is an exchange operation ” min = min ” � � Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 9 / 13

Limit Control Problem (I) A guess for the limit dynamics:  �   d ˆ R m f ( ˆ X t = X t , y , ν t ( y )) d µ ˆ X t ( y )( y ) dt      ��    √ R m σσ ⊤ ( ˆ + 2 X t , y , ν t ( y )) d µ ˆ X t ( y ) dW t , (3)           ν t ( · ) ∈ U ex ( ˆ ˆ X 0 = x ∈ R n . X t ) , and The effective optimal control problem V ( t , x ) = sup { ˆ J ( t , x , ν · ( · )) , subject to (3) } (4) where the effective pay off ˆ J ( t , x , ν · ( · )) is � � � T � e λ ( t − T ) g ( ˆ R m ℓ ( ˆ X s ( y ) e λ ( s − T ) ds E x X T ) + X s , y , ν s ( y )) d µ ˆ t Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 10 / 13

Limit Control Problem (II) Theorem The value function (4) is the unique viscosity solution to the Cauchy HJB. It is in particular, the limit of V ε defined in (2) for ( HJB ) ε . problem � V ε ( t , x , y ) SDE ( 1 ( HJB ) ε ε ) Viscosity control prob Ergodicity Effective Ham THEOREM ??? Homogenization Bellman Ham Selection arg . control prob � HJB V ( t , x ) SDI Viscosity Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 11 / 13

Convergence of Trajectories Key observation: The convergence Theorem for the value function holds independently of the choice of the cost functional, i.e. � As ε → 0, � � SDE ( 1 ε ) and SDI always produce the same value for every choice of a cost functional in the optimal control problem. So we can hope for at least a convergence of the type t ∈ [0 , T ] � φ ( X ε t ) − φ ( ˆ ε → 0 max lim X t ) � = 0 where φ is any real valued continuous function. Work in Progress Hicham Kouhkouh (Universit` a di Padova) Singular Perturbations & HJB PDE Los Angeles, May 18, 2020 12 / 13