Value Function HJB Equation State Constraints Optimal Control and Hamilton-Jacobi Equations H´ el` ene Frankowska CNRS and UNIVERSIT´ E PIERRE et MARIE CURIE Control and Optimization, Monastir, Tunisia May 15-19, 2017 H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Deterministic Control System � x ( t ) ˙ = f ( t , x ( t ) , u ( t )) , u ( t ) ∈ U a.e. in [0 , 1] ( CS ) x ( t 0 ) = x 0 U is a complete separable metric space, t denotes the time f : [0 , 1] × R n × U → R n , x 0 ∈ R n Controls are Lebesgue measurable functions u ( · ) : [0 , 1] → U A trajectory of (CS) is any absolutely continuous function x ∈ W 1 , 1 ([ t 0 , 1]; R n ) satisfying for some control u ( · ) x ( t ) = f ( t , x ( t ) , u ( t )) a.e. in [0,1] ˙ We assume that f ( t , x , · ) is continuous f ( · , x , u ) is measurable, f ( t , x , U ) are closed and ∃ γ : [0 , 1] → R + integrable such that sup u ∈ U | f ( t , x , u ) | ≤ γ ( t )(1 + | x | ) H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Deterministic Control System � x ( t ) ˙ = f ( t , x ( t ) , u ( t )) , u ( t ) ∈ U a.e. in [0 , 1] ( CS ) x ( t 0 ) = x 0 U is a complete separable metric space, t denotes the time f : [0 , 1] × R n × U → R n , x 0 ∈ R n Controls are Lebesgue measurable functions u ( · ) : [0 , 1] → U A trajectory of (CS) is any absolutely continuous function x ∈ W 1 , 1 ([ t 0 , 1]; R n ) satisfying for some control u ( · ) x ( t ) = f ( t , x ( t ) , u ( t )) a.e. in [0,1] ˙ We assume that f ( t , x , · ) is continuous f ( · , x , u ) is measurable, f ( t , x , U ) are closed and ∃ γ : [0 , 1] → R + integrable such that sup u ∈ U | f ( t , x , u ) | ≤ γ ( t )(1 + | x | ) H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Deterministic Control System � x ( t ) ˙ = f ( t , x ( t ) , u ( t )) , u ( t ) ∈ U a.e. in [0 , 1] ( CS ) x ( t 0 ) = x 0 U is a complete separable metric space, t denotes the time f : [0 , 1] × R n × U → R n , x 0 ∈ R n Controls are Lebesgue measurable functions u ( · ) : [0 , 1] → U A trajectory of (CS) is any absolutely continuous function x ∈ W 1 , 1 ([ t 0 , 1]; R n ) satisfying for some control u ( · ) x ( t ) = f ( t , x ( t ) , u ( t )) a.e. in [0,1] ˙ We assume that f ( t , x , · ) is continuous f ( · , x , u ) is measurable, f ( t , x , U ) are closed and ∃ γ : [0 , 1] → R + integrable such that sup u ∈ U | f ( t , x , u ) | ≤ γ ( t )(1 + | x | ) H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Semilinear Control System X is a separable Banach space. Consider the densely defined unbounded linear operator A - the infinitesimal generator of a strongly continuous semigroup S ( t ) : X → X , f : [0 , 1] × X × U → X , x 0 ∈ X and the semilinear control system x ( t ) = A x + f ( t , x ( t ) , u ( t )) , u ( t ) ∈ U , x ( t 0 ) = x 0 ˙ Its mild trajectory is defined by � t x ( t ) = S ( t − t 0 ) x 0 + S ( t − s ) f ( s , x ( s ) , u ( s )) ds ∀ t ∈ [ t 0 , 1] t 0 Many of the deterministic results were already adapted to the framework of semilinear control systems (controlled PDEs). Naturally this means some assumptions on semigroups: S ( · ) has to be compact to prove existence of optimal controls. H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Optimal Control The concept of optimal control can be described as the process of influencing the behavior of a dynamical system so as to achieve the desired goal: to maximize a profit, to minimize the energy, to get from one point to another one, etc. “After correctly stating the problem of optimal control and having at hand some satisfactory existence theorems, augmented by necessary conditions for optimality, we can consider that we have sufficiently substantial basis to study some special problems, as for instance Moon Flight Problem ”. From a book on Optimal Control, 1969 H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Value Function of the Mayer Problem g : R n → R ∪ { + ∞} , ξ 0 ∈ R n . Consider the Mayer’s problem : min { g ( x (1)) | x is a trajectory of (CS) , x (0) = ξ 0 } The value function associated with this problem is defined by: ∀ ( t 0 , x 0 ) ∈ [0 , 1] × R n V ( t 0 , x 0 ) = inf { g ( x (1)) | x is a trajectory of (CS) , x ( t 0 ) = x 0 } V (1 , · ) = g ( · ). In general V is nonsmooth even for smooth data. If x ( · ) is a trajectory of (CS), then for any t 0 ≤ t 1 ≤ t 2 ≤ 1, V ( t 1 , x ( t 1 )) ≤ V ( t 2 , x ( t 2 )). ¯ x ( · ) is optimal if and only if V ( t , ¯ x ( t )) ≡ g (¯ x (1)). Dynamic Programming Principle : ∀ h > 0 such that t 0 + h ≤ 1 V ( t 0 , x 0 ) = inf { V ( t + h , x ( t + h )) | x is a trajectory of ( CS ) , x ( t 0 ) = x 0 } H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Hamilton-Jacobi Equation The Hamiltonian H : [0 , 1] × R n × R n → R is defined by H ( t , x , p ) = max u ∈ U � p , f ( t , x , u ) � If V ∈ C 1 it satisfies the Hamilton-Jacobi equation − V t ( t , x ) + H ( t , x , − V x ( t , x )) = 0 , V (1 , · ) = g ( · ) Optimal (feedback) control u ( t , x ) ∈ U is chosen by : �− V x ( t , x ) , f ( t , x , u ( t , x )) � = H ( t , x , − V x ( t , x )) If u ( t , · ) is Lipschitz, then the solution ¯ x ( · ) of x ( t ) = f ( t , x ( t ) , u ( t , x )) a.e. in [0 , 1] , x (0) = ξ 0 ˙ is optimal for the Mayer problem. Even if data are smooth, V is not differentiable. H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Active Mathematical Domains Motivated by Optimal Control Nonsmooth, Set-Valued Analysis, Variational Analysis (since 1975) Solutions of HJB equations : viscosity and bilateral (since 1983) Control under state constraints (since 2000) First order necessary optimality conditions (since 1957) Second order necessary optimality conditions (since 1965) Sensitivity relations in control (since 1986) ..... H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Outline Value Function of the Mayer Problem 1 Hamilton-Jacobi-Bellman Equation 2 State Constraints 3 H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Existence of Optimal Controls Below we always assume that for a.e. t and ∀ r > 0, f ( t , · , u ) is c r ( t )-Lipschitz on B (0 , r ) ∀ u ∈ U , with integrable c r : [0 , 1] → R . For the Mayer problem let ( x i , u i ) be a minimizing sequence of trajectory-control pairs. By Gronwall’s Lemma, {� x i � ∞ } is bounded and | ˙ x i ( t ) | ≤ γ ( t )( � x i � ∞ + 1). Take a subsequence { x i j } converging uniformly to some x : [0 , 1] → R n with ˙ x i j converging weakly in L 1 ([0 , 1]; R n ) to some ¯ y : [0 , 1] → R n . Then ˙ ¯ x = ¯ ¯ y , ¯ x (0) = ξ 0 and ˙ x ( t ) ∈ conv f ( t , ¯ ¯ x ( t ) , U ) a.e. where conv denotes convex hull. If f ( t , x , U ) is convex, then, by the measurable selection theorem, u such that ˙ there exists a control ¯ ¯ x ( t ) = f ( t , ¯ x ( t ) , ¯ u ( t )) a.e. H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Existence of Optimal Controls If g is lower semicontinuous, then lim inf j →∞ g ( x i j (1)) ≥ g (¯ x (1)) and therefore ¯ u is optimal control. Theorem If f ( t , x , U ) are convex and g is lower semicontinuous, then for the Mayer problem an optimal solution does exist. For semilinear control systems only a part of this proof applies and one has either to assume, that S ( t ) is compact for all t > 0 or that { ( x , f ( t , x , u )) | u ∈ U , x ∈ X } is convex. This last assumption is very strong. For nonlinear stochastic control systems the convexity assumptions are worse even in the finite dimensional framework. H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Relaxation Theorem x ( t ) ∈ conv f ( t , x ( t ) , U ) a.e. in [ t 0 , 1] ˙ ( RS ) has more W 1 , 1 ([ t 0 , 1]; R n ) solutions ( relaxed trajectories ) than the control system (CS) for the same initial condition x ( t 0 ) = x 0 . Theorem x : [ t 0 , 1] → R n be a relaxed trajectory. Then for every ε > 0 Let ¯ there exists a trajectory x of (CS) such that � x − ¯ x � ∞ ≤ ε and x ( t 0 ) = ¯ x ( t 0 ) . Corollary If g is continuous, then the infimum in the Mayer problem is attained by a relaxed trajectory and V = V rel , where V rel ( t 0 , x 0 ) = min { g ( x (1)) | x ∈ W 1 , 1 satisfies ( RS ) , x ( t 0 ) = x 0 } H. Frankowska Optimal Control
Value Function HJB Equation State Constraints Regularity of Value Function Under our assumptions, sets of solutions of (CS) and (RS) depend on initial state in a locally Lipschitz way. Hence Theorem If ∀ x ∈ R n , f ( t , x , U ) is convex and g is lower semicontinuous, then V is lower semicontinuous. If g is continuous, then V is continuous. If g is locally Lipschitz, then V ( t , · ) is locally Lipschitz for every t ∈ [0 , 1] . Furthermore if γ is essentially bounded, then V is locally Lipschitz. If g ∈ C 1 , f ( t , · , u ) ∈ C 1 , H ( t , · , · ) ∈ C 2 on R n × ( R n \{ 0 } ) , ∇ g never vanishes and for every initial datum the optimal solution of the relaxed problem is unique, then V ∈ C 1 . H. Frankowska Optimal Control
Recommend
More recommend
