Computational Approaches to H ∞ Problems and Differential Games Université catholique de Louvain September 2008 Brian D O Anderson Australian National University National ICT Australia
Global Motivation • Some H ∞ problems give Riccati equations which cause standard solvers to break down • We give a cure • The cure is extendable to nonlinear game theory problems � This may be useful if there are numerical problems � Solution procedures are very few anyway, and numerical properties are not well understood. 16 September 2008 UCL 2
Outline • Global Motivation • Solving H ∞ Riccati equations • Nonlinear Extension • Conclusions and Future Work 16 September 2008 UCL 3
Outline • Global Motivation • Solving H ∞ Riccati equations � Detailed Motivation � Solving Riccati Equations, Kleinman and its repair � Algorithm Convergence � Game Theory Interpretation • Nonlinear Extension • Conclusions and Future Work 16 September 2008 UCL 4
Detailed Motivation • Software to solve Asymptotic Riccati Equations arising in H 2 problems is standard • The software can collapse on certain problems • The Kleinman algorithm will save the day: � Recursive � Requires Lyapunov equation solutions � Requires stabilizing gain to initialize � Computation burden is not the issue; numerical accuracy is � Converges quadratically (it is a Newton algorithm) • It does not extend to H ∞ equations (indefinite quadratic term)--and yet standard software can collapse here too. • What can we do? 16 September 2008 UCL 5
Solving Riccati Equations Direct methods The example on the next transparency shows what can Computational go wrong with an H-infinity disadvantages Riccati equation. A numerical example 16 September 2008 UCL 6
Numerical Problem Direct Huge Methods Errors Direct methods problems are an old difficulty! This is a motivation for using the Kleinman algorithm in the LQ case. 16 September 2008 UCL 7
Solving Riccati Equations Direct methods Iterative methods H 2 control problem : Definite R Traditional Computational Newton methods disadvantages A numerical example LQ problem Solve AREs Difficult to choose Kleinman with R¸0 an initial condition algorithm 16 September 2008 UCL 8
9 Kleinman Algorithm UCL 16 September 2008
Kleinman Algorithm for H ∞ The proof of convergence of the Kleinman algorithm cannot be extended to the H ∞ case! It does not work for the H ∞ case! Divergence 16 September 2008 UCL 10
Solving Riccati Equations Direct methods Iterative methods H 2 control H ∞ control problem : problem : Definite R Indefinite R Traditional Computational New Newton methods disadvantages algorithm ?? A numerical example LQ problem Solve AREs Difficult to choose Kleinman with R ¸ 0 an initial condition algorithm 16 September 2008 UCL 11
Problem setting Quadratic term is sign indefinite 16 September 2008 UCL 12
Algorithm for H ∞ ARE Indefinite Easy initialization Recursive algorithm using Nonnegative LQ regulator Riccati equations at each step, not Lyapunov equations! 16 September 2008 UCL 13
Convergence • Global convergence is guaranteed provided the H ∞ control problem is solvable � A monotone increasing matrix sequence is constructed to approximate the stabilizing solution of an H ∞ -type ARE . • Local quadratic rate of convergence • Motivation is not operation count in computations. It is accuracy. 16 September 2008 UCL 14
Game theoretic interpretation • Recall • Player u: minimize J; player w: maximize J. • P k is the monotone increasing matrix sequence in our algorithm . • Strategies for player u and w: u k+1 solves LQ, not game theory problem, when fixed w k is being used 16 September 2008 UCL 15
Comparison of results Easy Newton method Our algorithm (Kleinman) • Reduce an ARE with an • Reduce an ARE with indefinite quadratic term negative semidefinite Remark: to a series of AREs with a quadratic term to a negative quadratic term; series of Lyapunov If desired, one could use a nested equations; • Simple choice of the initial iteration, each LQ equation being • Need careful choice of condition; solved using Kleinman. initial condition; • A monotone non- • A monotone non- decreasing matrix increasing matrix sequence. sequence. For LQ H 2 problems For LQ game problems 16 September 2008 UCL 16
Periodic Equations--H 2 • Some problems are periodic, e.g. satellite control • H 2 periodic Riccati equations potentially have stabilizing periodic solution • Computational procedures are current research topic • “Kleinman” algorithm for time-varying Riccati equations over a finite interval predates Kleinman algorithm • “Kleinman” algorithm for periodic time-varying Riccati equations over infinite interval yields stabilizing periodic solution as limit of solution of periodic Lyapunov differential equations 16 September 2008 UCL 17
Periodic Equations--H ∞ • H ∞ periodic Riccati equations potentially have stabilizing periodic solution • “Kleinman” approach will not work on infinite interval. • Solution can be found by solving a sequence of H 2 periodic Riccati equations (“Kleinman” generalization will work for each one of these) • Game theory interpretation exists • No surprises in relation to the time-invariant case: � Quadratic monotone convergence. 16 September 2008 UCL 18
19 Problem Formulation UCL 16 September 2008
Periodic Equation Algorithm Indefinite H 2 periodic equations can be treated using a Kleinman-like algorithm, requiring solution of a sequence of periodic linear differential equations Semidefinite 16 September 2008 UCL 20
Outline • Global Motivation • Solving H ∞ Riccati equations • Nonlinear Extension � HJBI and HJB equations � Recursive solution of HJBI via HJB equations � Quadratic convergence and game theoretic interpretation • Conclusions and Future Work 16 September 2008 UCL 21
LQ and generalization Disturbance input HJB HJBI One player game Nonlinear H- infinity control LQ problem Nonlinear Nonlinear LQ Game optimal control Riccati equations Linear H-infinity control problem LQ problem 16 September 2008 UCL 22
Summary of nonlinear result H ∞ problem solution H 2 problem solution Iteration Linear-quadratic to Linear-quadratic to Nonlinear-nonquadratic Nonlinear-nonquadratic HJBI problem solution HJB problem solution Iteration can be found 16 September 2008 UCL 23
In addition….. H ∞ problem H 2 problem Kleinman solution solution iteration Iteration HJBI problem HJB problem Linear PDE solution solution Iteration Iteration exists Linear PDE iteration to give HJB solution stems from 1966 approx, though not carefully done for infinite time problem. 16 September 2008 UCL 24
HJB equation HJB equation may have multiple solutions. We are always interested in the stabilizing solution, which is nonnegative definite. Uniqueness properties hold. Π (x) is the optimal performance index given initial state x and the optimal control involves the gradient of Π 16 September 2008 UCL 25
HJBI equation Again, there may be multiple solutions but we seek unique stabilizing solution of HJBI equation. From it we can obtain Indefinite optimal performance, optimal control and worst case disturbance. 16 September 2008 UCL 26
Nonlinear Algorithm Simple Initial Condition HJB Eqn (get stabilizing solution) Each HJB equation in principle can be Nonnegative tackled by a sequence of linear partial DEs 16 September 2008 UCL 27
Convergence • Local convergence is guaranteed provided the H infinity control problem is locally solvable � A monotone increasing function sequence is constructed to approximate the stabilizing solution of an HJBI equation . • Local quadratic rate of convergence 16 September 2008 UCL 28
Game theoretic interpretation • Recall • Player u: minimize J; player w: maximize J. • Strategies for players u and w: u k+1 optimized for fixed w k and then w k+1 found V k: The monotone increasing function sequence in algorithm. 16 September 2008 UCL 29
Computational Results Existing methods Our algorithm • Taylor Expansion Method: • Local quadratic rate of Stability cannot always be convergence guaranteed, converges • Simple initial choice slowly • Galerkin Approximation • A natural game method: difficult to choose theoretic interpretation an initial stabilizing gain. • Method of characteristics: Converges slowly • Others exist again 16 September 2008 UCL 30
Example (van der Schaft) Figure compares exact solution, iterations from method of characteristics, and iterations from our method. 16 September 2008 UCL 31
A numerical example (continued) 16 September 2008 UCL 32
Outline • Global Motivation • Solving H ∞ Riccati equations • Nonlinear Extension • Conclusions and Future Work 16 September 2008 UCL 33
Recommend
More recommend
