A sequential quadratic Hamiltonian scheme for solving optimal - PowerPoint PPT Presentation

A sequential quadratic Hamiltonian scheme for solving optimal control problems with non-smooth cost functionals Alfio Borzì and Tim Breitenbach Institute for Mathematics, University of Würzburg, Germany A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Motivation ◮ Apply the Pontryagin maximum principle (PMP) to optimal control problems governed by ordinary (ODE) or partial differential equations (PDE) with different cost functionals (non-smooth, non-convex, discontinuous). ◮ Formulate a PMP consistent numerical method to calculate optimal controls. ◮ Investigate PMP necessary optimality conditions. ◮ Extend applicability of optimal control theory. A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Examples of optimal control problems � 1 � ˆ 2 ( y − y d ) 2 + g ( u ) min J ( y , u ) := dz Q ∂ t y − ∆ y = f ( z , y , u ) , y ( · , 0 ) = y 0 , y = 0 on ∂ Ω s.t. u ∈ U ad U ad := { u ∈ L q ( Q ) | u ( z ) ∈ K U a . e . } , K U ⊆ R , compact , where Q = Ω × ( 0 , T ) space-time domain, q ≥ 2, z := ( x , t ) ∈ Q . We require g lower semi-continuous (l.s.c) (at least). In particular � 1 if | u | � = 0 g ( u ) = α 2 u 2 + β | u | , α, β ≥ 0 , α + β > 0 . g ( u ) = γ , γ > 0 ; 0 else Further, we consider f ( z , y , u ) = u , f ( z , y , u ) = − u y . A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

About existence of optimal solutions ◮ Control to state map S : U ad → L 2 ( Q ) , u �→ y = S ( u ) ◮ Reduced cost functional ˆ J ( u ) := J ( S ( u ) , u ) ◮ Existence of a solution ¯ u ∈ U ad with ˆ ˆ J (¯ u ) = inf J ( u ) u ∈ U ad for g bounded from below, Lipschitz continuous and convex ◮ ˆ J bounded from below ◮ Minimizing sequence ◮ U ad is weakly sequentially compact ◮ ˆ J weakly lower semi-continuous ◮ For g non-convex, non-differentiable or discontinuous: Weakly lower semi-continuity of ˆ J can be lost (e.g. “ L 0 -norm”) A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Alternative concepts for existence Existence by minimizing sequences for the non-convex case: ◮ Compact set of L q ( Q ) In general, suboptimal ǫ - solutions on U ad : ◮ Setting: ◮ g l.s.c. ◮ g bounded from below ◮ S : U ad → L 2 ( Q ) continuous ◮ Consequences: ◮ ˆ J ( u ) l.s.c, bounded from below ◮ Existence of inf u ∈ U ad ˆ J ( u ) ◮ Existence of a suboptimal solution ¯ u ∈ U ad with ˆ ˆ J (¯ u ) ≤ inf J ( u ) + ǫ, ǫ > 0 . u ∈ U ad A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Lagrange approach if g and f are differentiable ◮ Functional formulation: Lagrange functional L ( y , u , p ) := 1 ˆ 2 � y − y d � 2 L 2 ( Q ) + g ( u ( z )) dz Q ˆ p ( z ) ( f ( z , y , u ) − y ′ ( z ) + ∆ y ( z )) dz + Q ◮ Necessary optimality conditions for optimal solution ¯ u : ◮ State equation ◮ Adjoint equation − ∂ t p − ∆ p = ( y − y d ) + p ∂ p ( · , T ) = 0 . ∂ y f ( z , y , u ) , ◮ Variational inequality � ∂ g (¯ u ) + p ∂ f � ( w − ¯ u ) ≥ 0 , w ∈ U ad . ∂ ¯ u ∂ ¯ u A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Pontryagin maximum principle 1 ◮ Pointwise formulation: Hamilton-Pontryagin (HP) function H : R n × R × K U × R → R : H ( z , y , u , p ) := 1 2 ( y − y d ) 2 + g ( u ) + p f ( z , y , u ) ◮ Adjoint equation: − ∂ t p − ∆ p = ( y − y d ) + p ∂ ∂ y f ( z , y , u ) , p ( · , T ) = 0 . Theorem 1: A necessary condition for ¯ u to be an optimal control is given by H ( z , ¯ y ( z ) , ¯ u ( z ) , ¯ p ( z )) ≤ H ( z , ¯ y ( z ) , v , ¯ p ( z )) for all v ∈ K U for almost all z ∈ Q , where ¯ y is the solution to the state equation for u ← ¯ u , and ¯ p is the solution to the adjoint equation for y ← ¯ y and u ← ¯ u . 1Pontryagin, Boltyanskii, Gamkrelidze, Mishchenko. The Mathematical Theory of Optimal Processes, Wiley & Sons, 1962 A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

To prove the PMP: The needle variation 2 , 3 Needle variation of u ∗ ∈ U ad at z ∈ Q with v ∈ K U , index k ∈ N : � if z ∈ S k ( z ) ∩ Q v u k ( z ) := u ∗ ( z ) if z ∈ Q \ S k ( z ) Lemma 2 Connection of J and H : 1 | S k ( z ) | ( J ( y k , u k ) − J ( y ∗ , u ∗ )) = H ( z , y ∗ , v , p ∗ ) − H ( z , y ∗ , u ∗ , p ∗ ) lim k →∞ at almost every z ∈ Q , and y k solution to the state equation for u ← u k y ∗ solution to the state equation for u ← u ∗ p ∗ solution to adjoint equation for y ← y ∗ and u ← u ∗ 2X. Li, J. Yong. Optimal Control Theory for Infinity Dimensional Systems, Birkhäuser, 1995 3J.-P. Raymond, H. Zidani. Hamiltonian Pontryagin’s principles for control problems governed by semilinear parabolic equations, Applied Mathematics and Optimization, 1999 A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Suboptimal solutions & PMP For functions u 1 , u 2 ∈ U ad ⊆ U , define the distance d ( u 1 , u 2 ) = | { z ∈ Q | u 1 ( z ) � = u 2 ( z ) } | : U complete metric space. By Ekeland’s variational principle 4 : For any ǫ > 0 there exists ¯ u ∈ U ad : J ( w ) − ˆ ˆ J (¯ u ) > − ǫ d ( w , ¯ u ) for all w ∈ U ad \ { ¯ u } . In particular, with w = u k needle variation of ¯ u : 1 � � J ( u k ) − ˆ ˆ J (¯ u ) > − ǫ | S k ( z ) | Theorem 3 5 Existence of a suboptimal solution ¯ u such that H ( z , ¯ y ( z ) , ¯ u ( z ) , ¯ p ( z )) ≤ H ( z , ¯ y ( z ) , v , ¯ p ( z )) + ǫ for almost all z ∈ Q and all v ∈ K U , where ¯ y is the solution to the state equation for u ← ¯ u , and ¯ p is the solution to the adjoint equation for y ← ¯ y and u ← ¯ u . 4I. Ekeland. On the variational principle, Journal of Mathematical Analysis and Applications, 1974 5A. Hamel. Suboptimality theorems in optimal control, Birkhäuser, 1998 A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Calculation of (sub-)optimal solutions Minimizing H is associated to minimizing J : 1 | S k ( z ) | ( J ( y k , u k ) − J ( y ∗ , u ∗ )) = H ( z , y ∗ , v , p ∗ ) − H ( z , y ∗ , u ∗ , p ∗ ) lim k →∞ Minimize the HP function: Successive approximation method 6 ◮ Control update u k + 1 ( z ) = arg min w ∈ K U H z , y k , w , p k � � ◮ Update the state and adjoint after each control update ◮ Fast calculation, but not robust with respect to convergence Penalize the control update 7 , ǫ > 0: w − u k � 2 � z , y k , w , u k , p k � � z , y k , w , p k � � := H + ǫ K ǫ ◮ Control update u k + 1 ( z ) = arg min w ∈ K U K ǫ � z , y k , w , u k , p k � ◮ Requires strategies to update the state ◮ Robust with respect to convergence, convergence theory 6I.A. Krylov, F.L.Chernous’ko. On a method of successive approximations for the solution of problems of optimal control, USSR Computational mathematics and Mathematical Physics, 1963 7Y. Shindo, Y. Sakawa. On global convergence of an algorithm for optimal control, IEEE Transactions on Automatic Control, 1980 A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

Combine the advantages of successive approximation and penalization ◮ Augmented Hamiltonian z , y k , w , u k , p k � := H w − u k � 2 z , y k , w , p k � � � � K ǫ + ǫ ◮ Control update u ( z ) = arg min w ∈ K U K ǫ � z , y k , w , u k , p k � ◮ Penalization term for efficient and robust convergence performance ◮ The state y k valid for the entire updated control sweep ◮ Fast calculation ◮ Convergence theory 8 8Shindo & Sakawa; J. F. Bonnans. On an algorithm for optimal control using Pontryagin’s maximum principle, SIAM Journal on Control and Optimization, 24(3):579–588, 1986. A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach

The SQH method 1. Choose ǫ > 0, κ ≥ 0, σ > 1, ζ ∈ ( 0 , 1 ) , η ∈ ( 0 , ∞ ) , u 0 ∈ U ad , compute y 0 corresponding to u = u 0 , and p 0 corresponding to y = y 0 and u = u 0 ; set k ← 0 2. Update the control � z , y k , w , u k , p k � u ( z ) = arg min K ǫ w ∈ K U for all z ∈ Q (sweep) 3. Compute y corresponding to u and τ := � u − u k � 2 L 2 ( Q ) y k , u k � � 4. If J ( y , u ) − J > − ητ : Choose ǫ ← σǫ Else: Choose ǫ ← ζǫ , y k + 1 ← y , u k + 1 ← u , calculate p k + 1 by the adjoint equation for y ← y k + 1 and u ← u k + 1 , set k ← k + 1 5. If τ < κ : STOP and return u k Else go to 2. A sequential quadratic Hamiltonian scheme for solving optimal Alfio Borzì and Tim Breitenbach