TSA Part 2: The Revenge A HJB-POD approach for the control of - PowerPoint PPT Presentation

TSA Part 2: The Revenge A HJB-POD approach for the control of nonlinear PDEs on a tree structure Luca Saluzzi joint work with A. Alla and M. Falcone ICODE Workshop on Numerical Solution of HJB Equations Paris, January 9, 2020

Outline Extension to high-order 1 High-order TSA Numerical test Control of nonlinear PDEs by TSA 2 Model Order Reduction Methods HJB-POD on a tree structure Numerical tests L. Saluzzi (GSSI) A HJB-POD approach for PDEs 2 / 29

Outline Extension to high-order 1 High-order TSA Numerical test Control of nonlinear PDEs by TSA 2 Model Order Reduction Methods HJB-POD on a tree structure Numerical tests L. Saluzzi (GSSI) A HJB-POD approach for PDEs 3 / 29

Extension to high-order (Falcone, Ferretti, ’94) We introduce a convergent one-step approximation � y n + 1 = y n + ∆ t Φ( y n , U n , t n , ∆ t ) , y 0 = x , where the admissible control matrix U n ∈ U ∆ t ⊂ U × U . . . × U ∈ R M × ( q + 1 ) , with U ⊂ R M . We assume that the function Φ is consistent ∆ t → 0 Φ( x , u , t , ∆ t ) = f ( x , u , t ) , lim where u = (¯ u , . . . , u ) ∈ U for u ∈ U and Lipschitz continuous: | Φ( x , U , t , ∆ t ) − Φ( y , U , t , ∆ t ) | ≤ L Φ | x − y | . Under these assumptions the scheme is convergent. L. Saluzzi (GSSI) A HJB-POD approach for PDEs 4 / 29

Extension to high-order schemes Then, we consider the approximation of the cost functional q N − 1 � � J ∆ t w i L ( y m + τ i , u m i , t m ) + g ( y N ) , x , t n ( { U m } ) = ∆ t m = n i = 0 where τ i and w i are the nodes and weights of the quadrature formula. Finally we define the numerical value function as { U n } J ∆ t V ( t , x ) = inf x , t ( { U n } ) Proposition (Discrete DPP) � � q � w i L ( y n + τ i , u n i , t n + τ i ) + V ( t n + 1 , y n + 1 ) V ( t , x ) = inf ∆ t { U m } i = 0 L. Saluzzi (GSSI) A HJB-POD approach for PDEs 5 / 29

Pruning for high-order scheme We can again define the pruned trajectory = η n +∆ t Φ( η n , U n , t n , ∆ t )+ E ε T ( η n + ∆ t Φ( η n , U n , t n , ∆ t ) , { η n + 1 η n + 1 } i ) j i L. Saluzzi (GSSI) A HJB-POD approach for PDEs 6 / 29

Pruning for high-order scheme We can again define the pruned trajectory = η n +∆ t Φ( η n , U n , t n , ∆ t )+ E ε T ( η n + ∆ t Φ( η n , U n , t n , ∆ t ) , { η n + 1 η n + 1 } i ) j i Proposition Given a one-step approximation { y n } n and its perturbation { η n } n , then t n − t | y n − η n | ≤ ε T ∆ t e L Φ ( t n − t ) . To guarantee p-th order convergence, the tolerance must be chosen such that ε T ≤ C (∆ t ) p + 1 . L. Saluzzi (GSSI) A HJB-POD approach for PDEs 6 / 29

Test 1: Bilinear control for Advection Equation   y t + cy x = yu ( t ) ( x , t ) ∈ Ω × [ 0 , T ] ,  y ( x , t ) = 0 ( x , t ) ∈ ∂ Ω × [ 0 , T ] ,   y ( x , 0 ) = y 0 ( x ) x ∈ Ω . � T � 2 dx + 0 . 01 | u ( s ) | 2 � y ( s ) � 2 y ( T ) � 2 � y ( s ) − ˜ ds + � y ( T ) − ˜ J y 0 , t ( u ) = 2 . t Semi-discrete problem (System dimension = 10 2 ) ˙ y ( t ) = Ay ( t ) + y ( t ) u ( t ) , ∆ x = 0 . 01 , Ω = [ 0 , 3 ] and c = 1 . 5 L. Saluzzi (GSSI) A HJB-POD approach for PDEs 7 / 29

Case 1 : ˜ y = 0, U = [ − 4 , 0 ] 0.14 1 Uncontrolled Implicit Euler Implicit Euler Uncontrolled Trapezoidal Trapezoidal 0.12 Controlled Implicit Euler 0.8 Controlled Trapezoidal 0.1 0.6 0.08 0.06 0.4 0.04 0.2 0.02 0 0 0 0.2 0.4 0.6 0.8 1 0 0.5 1 1.5 2 2.5 3 Figure: Top: Uncontrolled (left) and trapezoidal rule controlled solution (right). Bottom: cost functionals (left) and solutions at final time (right). L. Saluzzi (GSSI) A HJB-POD approach for PDEs 8 / 29

Case 2: ˜ y ( x , t ) = y 0 ( x − ct ) , U = [ 0 , 1 ] 1 0.08 Uncontrolled Implicit Euler Uncontrolled Implicit Euler Uncontrolled Trapezoidal Uncontrolled Trapezoidal 0.07 Controlled Implicit Euler Controlled Implicit Euler Controlled Trapezoidal 0.8 Controlled Trapezoidal 0.06 0.05 0.6 0.04 0.4 0.03 0.02 0.2 0.01 0 0 0 0.2 0.4 0.6 0.8 1 0 0.5 1 1.5 2 2.5 3 Figure: Comparison of the cost functionals (left) and the solutions at final time (right). ∆ t Nodes CPU Error 2 Order 0.1 506 0.11s 2.8e-2 0.05 3311 0.7s 8e-3 1.84 Table: Trapezoidal rule with 2 × 2 discrete controls and ε T = ∆ t 3 L. Saluzzi (GSSI) A HJB-POD approach for PDEs . 9 / 29

How does the cardinality change? Euler method Trapezoidal method 500 100 400 Cardinality Cardinality 300 50 200 100 0 0 0 50 100 0 5 10 15 20 Time level Time level Figure: Implicit Euler: |T | = O ( N 2 ) , Trapezoidal rule: |T | = O ( N 3 ) Method ∆ t Controls Nodes CPU Error Implicit Euler 2.5e-3 2 80982 9s 9e-3 Trapezoidal 5e-2 2 × 2 3311 0.7s 8e-3 L. Saluzzi (GSSI) A HJB-POD approach for PDEs 10 / 29

Outline Extension to high-order 1 High-order TSA Numerical test Control of nonlinear PDEs by TSA 2 Model Order Reduction Methods HJB-POD on a tree structure Numerical tests L. Saluzzi (GSSI) A HJB-POD approach for PDEs 11 / 29

Problem Setting Semidiscretized PDE  M ˙ y ( t ) = Ay ( t ) + f ( t , y ( t )) , t ∈ ( 0 , T ] ,   y ( 0 ) = y 0 , Assumptions y 0 ∈ R n is a given initial data, M , A ∈ R n × n given matrices, f : [ 0 , T ] × R n → R n a continuous function in both arguments and locally Lipschitz-type with respect to the second variable WARNING: High dimensional problems are computationally expensive. L. Saluzzi (GSSI) A HJB-POD approach for PDEs 12 / 29

Proper Orthogonal Decomposition and SVD Given snapshots ( y ( t 0 ) , . . . , y ( t n )) ∈ R m i = 1 in R m with ℓ ≪ min { n , m } s.t. We look for an orthonormal basis { ψ i } ℓ � � 2 � � n ℓ d � � � � � σ 2 J ( ψ 1 , . . . , ψ ℓ ) = � y j − � y j , ψ i � ψ i = α j � � i � j = 1 i = 1 i = ℓ + 1 reaches a minimum where { α j } n j = 1 ∈ R + . min J ( ψ 1 , . . . , ψ ℓ ) s . t . � ψ i , ψ j � = δ ij Singular Value Decomposition: Y = ΨΣ V T . For ℓ ∈ { 1 , . . . , d = rank ( Y ) } , { ψ i } ℓ i = 1 are called POD basis of rank ℓ. ℓ � σ 2 i i = 1 ERROR INDICATOR: E ( ℓ ) = with σ i singular values of the SVD. � d σ 2 i i = 1 L. Saluzzi (GSSI) A HJB-POD approach for PDEs 13 / 29

Reduced Order Modelling Control Problem MOR ansatz y ( t ) ≈ Ψ y ℓ ( t ) Ψ T Ψ = I , Ψ ∈ R n × ℓ Compact Notations x ℓ := Ψ T x , y ℓ ( t ) := Ψ T y ( t ) , g ℓ ( y ℓ ( t )) := Ψ T g (Ψ y ℓ ( t )) , f ℓ ( y ℓ ( t ) , u ( t ) , t ) := Ψ T f (Ψ y ℓ ( t ) , u ( t ) , t ) , L ℓ ( y ℓ ( t ) , u ( t )) := L (Ψ y ℓ ( t ) , u ( t )) . � ˙ y ℓ ( t ) = f ℓ ( y ℓ ( t ) , u ( t )) , t ∈ [ 0 , T ] , y ℓ ( 0 ) = x ℓ ∈ R ℓ . The cost functional is: � T L ℓ ( y ℓ ( t ) , u ( t ) , t ) e − λ t dt + g ℓ ( y ℓ ( T )) J ℓ x ℓ ( u ) = 0 L. Saluzzi (GSSI) A HJB-POD approach for PDEs 14 / 29

Reduced Order Modelling Control Problem Reduced Value Function v ℓ ( x ℓ , t ) = inf J ℓ x ℓ , t ( u ) u ∈U ad Reduced HJB equation − ∂ v ℓ ( x ℓ , t ) + λ v ℓ ( x ℓ , t )+sup {−∇ x ℓ v ℓ ( x ℓ , t ) · f ℓ ( x ℓ , u , t ) − L ℓ ( x ℓ , u , t ) } = 0 ∂ t u ∈ U Feedback Control u ℓ, ∗ ( x ℓ , t ) = arg min { f ℓ ( x ℓ , u , t ) · ∇ x ℓ v ℓ ( x ℓ , t ) + L ℓ ( x ℓ , u , t ) } u ∈ U L. Saluzzi (GSSI) A HJB-POD approach for PDEs 15 / 29

HJB-POD on a tree structure Computation of the snapshots POD for optimal control problems presents a major bottleneck: the choice of the control inputs to compute the snapshots. n = 0 T n for a We store the tree in the snapshots matrix Y = T = ∪ N chosen ∆ t and discrete control set U . Computation of the basis functions We solve � � 2 � ℓ � N � � � � � min � y ( t j , u j ) − � y ( t j , u j ) , ψ i � ψ i , � ψ i , ψ j � = δ ij , � � � ψ 1 ,...,ψ ℓ ∈ R d j = 1 u j ⊂ U j i = 1 No restrictions on the choice of the number of basis ℓ , since we will solve the HJB equation on a tree structure. We choose ℓ such that E ( ℓ ) ≈ 0 . 999 , L. Saluzzi (GSSI) A HJB-POD approach for PDEs 16 / 29

HJB-POD on a tree structure Construction of the reduced tree Construction of a new (projected) tree T ℓ with a smaller ∆ t and/or a finer control space with respect to the snapshots set. The first level of the tree is contains the projection of the initial condition, i.e. T 0 ,ℓ = Ψ T x . Again we have T n ,ℓ = { ζ n − 1 ,ℓ + ∆ t f ℓ ( ζ n − 1 ,ℓ , u j , t n − 1 ) } M i = 1 , . . . , M n − 1 , j = 1 i i where the reduced nonlinear term f ℓ can be done via POD or POD-DEIM. The procedure follows the full dimensional case, but with the projected dynamics. L. Saluzzi (GSSI) A HJB-POD approach for PDEs 17 / 29