Optimal Control 4SC000 Q2 2017-2018 Duarte Antunes Recap - PowerPoint PPT Presentation

Optimal Control 4SC000 Q2 2017-2018 Duarte Antunes

Recap • Continuous-time optimal control problems Dynamic model x ( t ) = f ( x ( t ) , u ( t )) , ˙ x (0) = x 0 , t ∈ [0 , T ] Z T Cost g ( x ( t ) , u ( t )) dt + g T ( x ( T )) 0 • We have seen two simple versions of the PMP considering • problems with no terminal constraints and final cost g T ( x ( T )) • problems with terminal constraints and no final cost x ( T ) = ¯ x 1

Recap • In both problems, we have is an optimal path candidate if s.t. ( u ∗ ( t ) , x ∗ ( t )) ∃ λ ( t ) , t ∈ [0 , T ] State eq. x ∗ ( t ) = f ( x ∗ ( t ) , u ∗ ( t )) ˙ ˙ Adjoint eq. λ ( t ) = − ( ∂ ∂ x f ( x ∗ ( t ) , u ∗ ( t ))) | λ ( t ) − ( ∂ ∂ x g ( x ∗ ( t ) , u ∗ ( t ))) | ∂ u g ( x ∗ ( t ) , u ∗ ( t )) | = 0 ∂ ∂ Control eq. ∂ u f ( x ∗ ( t ) , u ∗ ( t )) | λ ( t ) + • The boundary conditions are different for the two problems ∂ (no terminal state constraints) x ∗ (0) = ¯ λ ( T ) = ∂ x g T ( x ∗ ( T )) x 0 (terminal constraints) x ∗ (0) = ¯ x ∗ ( T ) = ¯ x x 0 • It is also possible that only the terminal value of some variables are constrained, x i ( T ) = ¯ x i , i ∈ C 2

Outline • Pontryagin’s maximum principle • Examples • Hamiltonian’s principle in mechanics • Brachistochrone • Minimum energy problems • Minimal time problems for linear systems (bang-bang control)

Discussion • Last lecture we used an informal discretization approach to derive a simplified version of the PMP . • The Pontryagin’s maximum principle is more general and it allows to consider constraints and free terminal time problems. This requires a different mathematical framework in continuous-time. CT PMP CT Optimal control CT DP path and problem policy Taking the limit Discretization, step τ τ → 0 DT PMP Stage Optimal decision path and DT DP problem policy The goals for today are • State the general Pontryagin’s maximum principle. • Solve minimal time problems with the PMP for linear system with input constraints. 3

Problem formulation Control system x (0) = x 0 x ( t ) = f ( x ( t ) , u ( t )) ˙ Cost functional to be minimized R T J ( u ) = 0 g ( x ( t ) , u ( t )) dt Assumptions • , control input may be constrained x ( t ) ∈ R n u ∈ U ⊂ R m • Lipschitz property: for every bounded set , s.t. D ∈ R n × U ∃ L | f ( x 1 , u ) − f ( x 2 , u ) | ≤ L | x 1 − x 2 | for all ( x 1 , u ) , ( x 2 , u ) ∈ D • are continuous. f, g, ∂ f ∂ x , ∂ g ∂ x • Initial time and initial state are fixed. x 0 = x ( t 0 ) t 0 = 0 • Final time and final state can be free or fixed. x f := x ( T ) T 4

Target set • Depending on the control objective, the final time and final state can be free or fixed, or can belong to some set. • All the possibilities are captured by introducing a closed target set S ⊂ [0 , ∞ ) × R n and letting be the smallest time such that ( T, x f ) ∈ S T • fixed-time, fixed end state for some fixed (previous lecture) S = { T } × { x f } T, x f • fixed-time, free-end state (previous lecture) S = { T } × R n • or in general for some surface in { T } × S 1 S 1 R n • In particular only some of the state variables may be constrained. • free-time, fixed end state S = [0 , ∞ ) × { x 1 } • or in general S = [0 , ∞ ) × S 1 • free-time, free end state (not so common) S = [0 , ∞ ) × R n 5

Target set • We can always consider since it captures not only free time-fixed end S = [0 , ∞ ) × S 1 state problems ( ) but also fixed-time by considering time as a state S = [0 , ∞ ) × { x 1 }  �  �  �  � x ( t ) ˙ f ( x ( t ) , u ( t )) x (0) x 0 = = x n +1 = t x n +1 ( t ) ˙ 1 x n +1 (0) 0 Then, for this new problem S = [0 , ∞ ) × S 1 × { t 1 } • In particular, • S = [0 , ∞ ) × R n × { t 1 } fixed-time, free end state • fixed-time, fixed end state S = [0 , ∞ ) × { x 1 } × { t 1 } S 1 = { x ∈ R n : h 1 ( x ) = · · · = h n − k ( x ) = 0 } h i : R n → R • We assume that , differentiable • If the target set is never reached the cost is set to infinity. 6

Terminal cost • To consider the problem with terminal cost R T J ( u ) = 0 g ( x ( t ) , u ( t )) dt + g T ( x f ) x (0) = x 0 x ( t ) = f ( x ( t ) , u ( t )) ˙ R T d note that g T ( x f ) = g T (0) + dt g T ( x ( t )) dt 0 ∂ ∂ d where dt g T ( x ( t )) = ∂ x g T ( x ( t )) ˙ x ( t ) = ∂ x g T ( x ( t )) f ( x ( t ) , u ( t )) R T • Then J ( u ) = 0 ¯ g ( x ( t ) , u ( t )) dt + constant ∂ g ( x, u ) = g ( x, u ) + ¯ ∂ x g T ( x ) f ( x, u ) and so there is no loss of generality in our problem formulation. 7

Pontryagin’s maximum principle Consider an optimal control problem with target set . S = [0 , ∞ ) × S 1 x ∗ : [0 , T ] → R n u ∗ : [0 , T ] → U Let be an optimal control (in the global sense) and let p ∗ : [0 , T ] → R n be the corresponding optimal state trajectory. Then there exist a function and a constant satisfying for all and having the p ∗ t ∈ [0 , T ] 0 ≤ 0 ( p ∗ 0 ( t ) , p ∗ ( t )) 6 = (0 , 0) following properties: x ∗ = ∂ 1) and satisfy x ∗ p ∗ ˙ ∂ pH ( x ∗ , u ∗ , p ∗ , p ∗ 0 ) p ∗ = − ∂ ˙ ∂ xH ( x ∗ , u ∗ , p ∗ , p ∗ 0 ) with boundary conditions and , where is H : R n × U × R n × R → R x ∗ ( T ) ∈ S 1 x ∗ (0) the Hamiltonian, defined as . H ( x, u, p, p 0 ) = p | f ( x, u ) + p 0 g ( x, u ) 2) For each fixed , the function has a global maximum, i.e., u → H ( x ∗ ( t ) , u, p ∗ ( t ) , p ∗ 0 ) t H ( x ∗ ( t ) , u ∗ ( t ) , p ∗ ( t ) , p ∗ 0 ) ≥ H ( x ∗ ( t ) , u, p ∗ ( t ) , p ∗ 0 ) hold for all and all . t ∈ [0 , T ] u ∈ U 3) for all t ∈ [0 , T ] H ( x ∗ ( t ) , u ∗ ( t ) , p ∗ ( t ) , p ∗ 0 ) = 0 4) The vector is orthogonal to the tangent space to at x ∗ ( T ) p ∗ ( T ) S 1 ∀ d, i : < d, ∂ ∂ xh i ( x ∗ ( T )) | > = 0 < p ∗ ( T ) , d > = 0 8

Discussion • These are still only necessary conditions for optimality and not sufficient. However a local maximum or a local minimum cannot satisfy these conditions and thus these are necessary conditions for global optimality. • The function should be interpreted as our previous costate, but p ( t ) : [0 , T ] → R n multiplied by , . This is just a convention. p ( t ) = − λ ( t ) − 1 • The scalar is called the abnormal multiplier. We can replace except in p ∗ 0 = − 1 p 0 degenerate cases which we will not address (in which case ). p ∗ 0 = 0 • Accordingly, the Hamiltonian is defined in a slightly different manner H ( x, u, p, − 1) = p | f ( x, u ) − g ( x, u ) which is obtained by multiplying by our previous definition − 1 ˆ H ( x, u, λ ) = λ | f ( x, u ) + g ( x, u ) = − p | f ( x, u ) + g ( x, u ) = − H ( x, u, p, − 1) 9

Conditions 1 and 2 • The conditions stated in point 1) of the Theorem are nothing more that the equations for the state and the co-state that we have obtained before. In our previous notation x ∗ = ∂ x ∗ = f ( x ∗ , u ∗ , p ∗ ) ˙ ∂ pH ( x ∗ , u ∗ , p ∗ , p ∗ 0 ) ˙ λ ∗ = − ∂ ∂ xf ( x ∗ , u ∗ , λ ∗ ) | λ ∗ − ∂ ˙ p ∗ = − ∂ ∂ xL ( x ∗ , u ∗ , λ ∗ ) | ˙ ∂ xH ( x ∗ , u ∗ , p ∗ , p ∗ 0 ) (as in previous lectures) • The condition 2) states that has a (global) maximum. In our u → H ( x ∗ ( t ) , u, p ∗ ( t ) , p ∗ 0 ) previous notation this would correspond to a minimum. If there are no input constraints and are differentiable with respect to (our previous setting) then we must have f, g u ∂ ∂ uf ( x ∗ , u ∗ , p ∗ ) | λ ∗ + ∂ ∂ ∂ ug ( x ∗ , u ∗ , p ∗ ) | = 0 ∂ uH ( x ∗ , u, p ∗ , p ∗ 0 ) | u = u ∗ = 0 (as in previous lectures) Thus condition 2 is more general and shall allow us to handle input constraints 10