Optimality conditions for bang-bang controls (theory and examples) - PDF document

NIKOLAI P. OSMOLOVSKII Optimality conditions for bang-bang controls (theory and examples) Joint work with Helmut Maurer CEA-EDF-INRIA School Optimal Control: Algorithms and Application May 30 - June 1st 2007 INRIA Rocquencourt, France 1

Mathematical programming: optimality conditions R n : Consider problem in I J = f 0 ( x ) → min , f i ( x ) ≤ 0 , i = 1 , . . . , k g j ( x ) = 0 , j = 1 , . . . , m, where f 0 , f i , g j are C 2 -functions. Lagrange function has the form k m � � L ( x, α, β ) = α i f i ( x ) + β j g j ( x ) . i =0 j =1 2

Set Λ 0 , critical cone, and quadratic form At a feasible point ˆ x , define the set of normed tuples of Lagrange multipliers : � � λ = ( α, β ) | α ≥ 0 , | α | + | β | = 1 , Λ 0 = α i f i (ˆ x ) = 0 , i = 1 , . . . , k, L x (ˆ x, α, β ) = 0 , the critical cone x | f ′ f ′ g ′ K = { ¯ 0 (ˆ x )¯ x ≤ 0 , i (ˆ x )¯ x ≤ 0 , i ∈ I, j (ˆ x )¯ x = 0 , j = 1 , . . . , m } , where I = { i ∈ { 1 , . . . , k } | f i (ˆ x ) = 0 } is the set of active indices, and the form : Ω( λ, ¯ x ) = � L ′′ xx (ˆ x, α, β )¯ x, ¯ x � , which is linear in λ and quadratic in ¯ x . 3

Second order no-gap conditions of a local minimum (Levitin, Milyutin, Osmolovskii, 1974) Theorem 1 Let the point ˆ x be a local minimum in the problem. Then, at this point, (a) the set Λ 0 is nonempty and x ) ≥ 0 ∀ ¯ x ∈ K . (b) max λ ∈ Λ 0 Ω( λ, ¯ Theorem 2 Let at the feasible point ˆ x (a) the set Λ 0 be nonempty and ∀ ¯ x ∈ K \ { 0 } . (b) max λ ∈ Λ 0 Ω( λ, ¯ x ) > 0 Then ˆ x is a local minimum in the problem. 4

Two types of second order optimality conditions for bang-bang control problems Second order (necessary, sufficient) conditions for bang-controls obtained by Osmolovskii published in: Milyutin, A.A. and Osmolovskii, N.P. (1998) Calculus of Variations and Optimal Control, Translations of Mathematical Monographs, Vol. 180, American Mathematical Society, Providence Second order sufficient conditions for bang-controls obtained by Agrachev, Stefani, and Zezza published in: Agrachev, A.A, Stefani, G. and Zezza, P.L. (2002) Strong optimality for a bang-bang trajectory, SIAM J. Control and Optimization, vol. 41, pp. 991-1014. 5

Linear in control problem on a nonfixed time interval Minimize J ( t 0 , t 1 , x ( · ) , u ( · )) = J ( t 0 , x ( t 0 ) , t 1 , x ( t 1 )) under the constraints F ( t 0 , x ( t 0 ) , t 1 , x ( t 1 )) ≤ 0 , K ( t 0 , x ( t 0 ) , t 1 , x ( t 1 )) = 0 , x = f ( t, x, u ) , ˙ u ∈ U, where f ( t, x, u ) = a ( t, x ) + B ( t, x ) u, U is a convex polyhedron , J, F, K, and B are C 2 − functions , 6

� t 1 Remark : The integral functional J = t 0 ϕ ( t, x, u ) dt has the endpoint form: J = y ( t 1 ) − y ( t 0 ) , where ˙ y = ϕ ( t, x, u ) . 7

Fix ∆ := [ t 0 , t 1 ]. Abbreviations: x 0 = x ( t 0 ) , x 1 = x ( t 1 ) , p = ( x 0 , x 1 ) . Definition A trajectory T = ( x ( t ) , u ( t ) | t ∈ ∆) is admissible , if x ( · ) is absolutely continuous, u ( · ) is measurable bounded, T satisfies all constraints. 8

Bang-bang control Fix an admissible trajectory ˆ T = (ˆ u ( t ) | t ∈ ∆) such that u ( t ) is a bang-bang control , i.e., x ( t ) , ˆ (a) ˆ u ( · ) is piecewise constant , (b) ˆ u ( t ) ∈ ex U . Denote by Θ = { ˆ τ 1 , . . . , ˆ τ s } , t 0 < ˆ τ 1 < · · · < ˆ τ s < t 1 the set of discontinuity points of ˆ u ( t ). Denote by u ] k = ˆ [ˆ u (ˆ τ k +) − ˆ u (ˆ τ k − ) the jump of ˆ u ( t ) at ˆ τ k ∈ Θ . 9

u ( t ) ˆ ✻ ✛ ✲ ✛ ✲ ✲ t t t t t 0 τ 1 ˆ ˆ τ 2 t 1 t ✛ ✲ 10

Introduce • the Pontryagin function (or the Hamiltonian) H ( t, x, u, ψ ) = ψf ( t, x, u ) = ψa ( t, x ) + ψB ( t, x ) u, • the switching function σ ( t, x, ψ ) = ψB ( t, x ) , • the end-point Lagrange function l ( α 0 , α, β, p ) = α 0 J ( p ) + αF ( p ) + βK ( p ) , • a quadruple of Lagrange multipliers λ = ( α 0 , α, β, ψ ( · )) . 11

Maximum principle For ˆ T , denote by M 0 the set of λ satisfying: α 0 ≥ 0 , α ≥ 0 , αF (ˆ p ) = 0 , α 0 + � α i + � | β j | = 1 , ˙ ψ = − H x , ψ ( t 0 ) = l x 0 , ψ ( t 1 ) = − l x 1 , max u ∈ U H ( t, ˆ x ( t ) , u, ψ ( t )) = H ( t, ˆ x ( t ) , ˆ u ( t ) , ψ ( t )) , where p = (ˆ ˆ x ( t 0 ) , ˆ x ( t 1 )) l x 0 = l x 0 ( α 0 , α, β, ˆ p ) , H x = H x ( t, ˆ x ( t ) , ˆ u ( t ) , ψ ( t )) . 12

Pontryagin minimum Definition (Milyutin) ˆ T is a Pontryagin minimum if it is a minimum on the set � x ( · ) − ˆ x ( · ) � ∞ < ε, � u ( · ) − ˆ u ( · ) � 1 < ε with some ε > 0 . Theorem 3 • If ˆ T is a Pontryagin minimum, then M 0 � = ∅ . • M 0 is a finite-dimensional compact set • the projector λ �→ ( α 0 , α, β ) is injective on M 0 . 13

First type of second order conditions. Necessary conditions. Quantity D k ( H ) Proposition 1 For each λ ∈ M 0 and for each ˆ τ k ∈ Θ the derivative u ] k � � � D k ( H ) := d � σ ( t, ˆ x ( t ) , ψ ( t ))[ˆ t =ˆ dt τ k exists and, moreover, D k ( H ) ≥ 0 . 14

The space Z (Θ) Denote by P Θ C 1 (∆ , I R n ) the space of piecewise continuous functions R n , x ( · ) : ∆ → I ¯ continuously differentiable on each interval of the set ∆ \ Θ. Set z = (¯ ¯ ξ, ¯ x ) , where ¯ R s , R n ). Thus, x ∈ P Θ C 1 (∆ , I ξ ∈ I ¯ R s × P Θ C 1 (∆ , I R n ) . z ∈ Z (Θ) := I ¯ For ¯ z , set p = (¯ ¯ x ( t 0 ) , ¯ x ( t 1 )) . 15

Critical cone Denote by K the set of ¯ z ∈ Z (Θ) satisfying � J ′ (ˆ p ) , ¯ p � ≤ 0 , � F ′ i (ˆ p ) , ¯ p � ≤ 0 ∀ i ∈ I F (ˆ p ) , K ′ (ˆ p )¯ p = 0 , x ] k = [˙ x ] k ¯ ˙ x ( t ) = f ′ ¯ x ( t, ˆ x ( t ) , ˆ u ( t ))¯ x ( t ) , [¯ ξ k ∀ k, � where p ) = { i ∈ { 1 , . . . , d ( F ) } | F i (ˆ p ) = 0 } , I F (ˆ x ] k = ¯ τ k +) − ¯ τ k − ) , [¯ x (ˆ x (ˆ x ] k = ˙ [ ˙ τ k +) − ˙ ˆ x ( � x (ˆ τ k − ) � � x ( t ) and ˙ are the jumps of ¯ x ( t ), resp., at the point ˆ ˆ τ k ∈ Θ. The cone K is finite-dimensional and finite-faced! 16

Quadratic form For each λ ∈ M 0 and ¯ z ∈ K we set � � � t 1 � s D k ( H )¯ k + 2[ ˙ av ¯ ξ 2 ψ ] k ¯ x k + � l pp ¯ p � − � H xx ¯ x ( t ) � dt, Ω( λ, ¯ z ) = ξ k p, ¯ x ( t ) , ¯ k =1 t 0 ψ ] k is the jump of ˙ where [ ˙ ψ ( t ) at ˆ τ k ∈ Θ, av = 1 x k ¯ 2(¯ x (ˆ τ k − ) + ¯ x (ˆ τ k +)) , l pp = l pp (ˆ p, α 0 , α, β ) , H xx = H xx ( t, ˆ x ( t ) , ˆ u ( t ) , ψ ( t )) . 17

Quadratic necessary optimality condition Theorem 4 (Osmolovskii, 1988) If ˆ T is a Pontryagin minimum, then the following condition A hold: (a) the set M 0 is nonempty z ) ≥ 0 for all ¯ z ∈ K . (b) max λ ∈ M 0 Ω( λ, ¯ 18

Essential component of the state Definition The component x i of the state vector x is called unessential if (a) f does not depend on x i and (b) F, J, K are affine in x i ( t 0 ) and x i ( t 1 ). Example : Let � t 1 J = J = y ( t 1 ) − y ( t 0 ) , ϕ ( t, x, u ) dt � y = ϕ ( t, x, u ) . ˙ t 0 Then y is unessential component. ess ( x ) denote the vector of all essential components of the vector x . 19

Strong minimum Definition ˆ T is a strong minimum , if it is a minimum on the set � � | x ( t 0 ) − ˆ x ( t 0 ) | < ε, � ess x ( · ) − ˆ x ( · ) � ∞ < ε with some ε > 0 . 20

Strict bang-bang control Let Arg max v ∈ U σv be the set v ∈ U where the maximum of σv is attained. Definition ˆ u ( t ) is a strict bang-bang control if there exists λ ∈ M 0 such that Arg max v ∈ U σ ( t, ˆ x ( t ) , ψ ( t )) v = [ˆ u ( t − ) , ˆ u ( t +)] . If dim( u ) = 1, then the strict bang-bang property is equivalent to σ ( t, ˆ x ( t ) , ψ ( t )) � = 0 ∀ t ∈ ∆ \ Θ . 21

Quadratic sufficient optimality condition Theorem 5 (Osmolovskii, 1998) For ˆ T , let the following Condition B be fulfilled (a) the set M 0 is nonempty and ˆ u ( t ) is a strict bang-bang control; (b) there exists λ ∈ M 0 such that D k ( H ) > 0 , ∀ k ; (c) λ ∈ M 0 Ω( λ, ¯ max z ) > 0 for all ¯ z ∈ K \ { 0 } . Then ˆ T is a strict strong minimum. There is no gap between the necessary condition A and the sufficient condition B ! 22

Second type of second order conditions. Function u ( t ; τ ) For any i = 1 , . . . , s , denote by u i the vertex of the polyhedron U such that u ( t ) = u i for t ∈ (ˆ ˆ τ i − 1 , ˆ τ i ) where ˆ τ 0 = t 0 , ˆ τ s +1 = t 1 . R s close to the vector ˆ Take a vector τ = ( τ 1 , . . . , τ s ) ∈ I τ = (ˆ τ 1 , . . . , ˆ τ s ) . Define the function u ( t ; τ ) by the condition u ( t ; τ ) = u i for t ∈ ( τ i − 1 , τ i ) ∀ i, where τ 0 = t 0 , τ s +1 = t 1 . Obviously, u ( t ; ˆ τ ) = ˆ u ( t ) , t ∈ ∆ \ Θ . 23

u ( t ; τ 1 , τ 2 ) ✻ u 1 u 3 ✛ ✲ ✛ ✲ ✲ t t t t t t t 0 =: ˆ τ 0 τ 1 τ 1 ˆ ˆ τ 2 τ 2 τ 3 := t 1 ˆ t u 2 ✛ ✲ 24

Function x ( t ; x 0 , τ ) x (ˆ Take a vector x 0 close to the vector ˆ x 0 := ˆ t 0 ). Let x ( t ; x 0 , τ ) be the solution to the ”basic initial value problem” x = f ( t, x, u ( t ; τ )) , ˙ x ( t 0 ) = x 0 . x ( t ) ∀ t ∈ ∆ . Obviously, x ( t ; ˆ x 0 , ˆ τ ) = ˆ R n × I Denote ˆ R s . τ ) ∈ I ζ := (ˆ x 0 , ˆ 25