On Moment Problems in Robust Control, Spectral Estimation, Image - PowerPoint PPT Presentation

On Moment Problems in Robust Control, Spectral Estimation, Image Processing and System Identification Anders Lindquist Shanghai Jiao Tong University, China, and the Royal Institute of Technology, Stockholm, Sweden University of Maryland May, 2017

Special recognition Christopher Byrnes Tryphon Georgiou + Sergei Gusev, Alexei Matveev, Alexandre Megretski, Giorgio Picci, Per Enqvist, Johan Karlsson, Ryozo Nagamune, Anders Blomqvist, Vanna Fanizza, Enrico Avventi, Axel Ringh

What is the talk about • A classical problem – the moment problem – with a decidedly non-classical twist motivated by engineering applications. • What is new are certain rationality constraints imposed by applications that alter the mathematical problem and make it nonlinear. • A global-analysis approach that studies the class of solutions as a whole. • A powerful paradigm for smoothly parameterizing, comparing, and shaping solutions to specifications.

The moment problem dµ ∈ M + space of positive measures Chebyshev Markov Lyapunov

µ x 4 Ex. 1 1 µ step function x 3 N x 2 � x 1 α k ( t j ) x j = c k , k = 0 , 1 , . . . , n a t t 3 t 1 t 4 j =1 t 2 b x       α 0 ( t 1 ) α 0 ( t 2 ) α 0 ( t N ) x 1 c 0 · · · N > n α 1 ( t 1 ) α 1 ( t 2 ) α 1 ( t N ) x 2 c 1       · · ·  = . . . . .       ... . . . . .       . . . . . infinitely many      α n ( t 1 ) α n ( t 2 ) α n ( t N ) x N c n solutions · · · z = e i θ Im dµ = Φ ( e i θ ) d θ spectral density Ex. 2 2 π Re Z π e ik θ Φ ( e i θ ) d θ θ 2 π = c k , k = 0 , 1 , . . . , n − π

Where do we find moment problems in applications? $! 2 • spectral estimation ) ' • speech synthesis ( 94:/;<.5,235=8 % • system identification ! > $ ! % > & > # • image processing ! ( > 2;5,4? ! ' 2 ! !"# $ $"# % %"# & *+,-.,/0123+45678 • optimal control $" • robust control generator coupling load %" " • model reduction ! %" 67-8,91):;19,*02<5 • model matching problems ! $" ! '" • simultaneous stabilization ! &" • optimal power transfer ! !"" ! !%" !%?@ ! A)2*):4/4?*B:C/:D-19=:E-?*)F= A)787-19:4/4?*B ! !$" " # !" !" ()*+,*-./0)1234*.5

… and why do these problems require a nonclassical approach? • Solution must be of bounded complexity (such as rational of a bounded degree) so that one can realize it by a finite-dimensional device y u w(z) • Classical theory does not provide natural para- meterizations of rational solutions of bounded degree

Prototype problem: Covariance extension c k = E { y ( t + k ) y ( t ) } , k = 0 , 1 , 2 , . . . , where y stationary stochastic process Given c 0 , c 1 , . . . , c n , find an infinite Carathéodory extension c n +1 , c n +2 , . . . such that Schur 2 c 0 + c 1 z + · · · + c n z n + c n +1 z n +1 + · · · f ( z ) = 1 · · 2 f (i) is a Carath´ eodory function · · · (ii) is rational of degree at most n r f ∈ C + Kalman

Trigonometric moment problem f MOMENT PROBLEM: Find Φ of degree at most 2n such that Φ ( z ) = P ( z ) P, Q trigonometric polynomials of degree n Q ( z ) ,

Spectral estimation by � covariance extension stationary process y u white noise with spectral density w(z) where observed data Since , we use Hence, we can only estimate ergodic estimate Remains to determine

Modeling speech y u Speech Excitation signal w(z) w(z) varies with time w(z) constant on each (30 ms) subinterval

A 30 ms frame of speech for Periodogram (FFT) of the voiced nasal phoneme [ng] 6 voiced nasal phoneme [ng] 4 Construct a rational filter with a rational w ( z ) of low degree, modeling the window of speech y u y u w ( z ) filter determined by few parameters

Linear Predictive (LPC) Filtering yields Szegö polynomial and modeling filter where

Linear Predictive (LPC) Filtering y u y u w ( z ) n = 10 Choose an excitation signal Send coe ffi cients in u from a code book with ϕ 10 ( z ) and number 1024 = 2 10 entries (10 bits) of “best” signal w ( z ) = σ ( z ) However, can we construct the general solution? a ( z )

w ( z ) = σ ( z ) a ( z ) Cellular telephone: envelope in purple

All the solutions THEOREM. The solutions of the rational covariance extension problem are completely parameterized by the zeros of the corres- ponding shaping filter, i.e., given an arbitrary monic stable polynomial σ ( z ) there is one and only one stable polynomial a ( z ) such that w ( z ) = σ ( z ) a ( z ) is a shaping filter for c 0 , c 1 , . . . , c n . The correspondence is a di ff eomorphism. • Existence proved by Georgiou 1983; conjectured uniqueness • The rest proved by Byrnes, Lindquist, Gusev, Matveev 1993 • These first proofs were nonconstructive, but there are now constructive proofs based on optimization

envelope zeros/poles zeros/poles envelope

10th degree LPC 20th degree LPC 6th degree filter with appropriate zeros

Optimization approach Given σ and c, minimize over all q such that THEOREM. There is a unique minimum. where Then

Nonlinear coordinates Normalize The space of all rational Carath´ eodory functions f so that c 0 = 1 of degree at most n is a 2 n -dimensional manifold. " " " A foliation with one leaf for each choice of σ (fast Kalman filtering) A foliation with one leaf for each choice of c = ( c 1 , c 2 , . . . , c n ) ! ! ! min J σ ( q ) q unique solution Φ = | σ | 2 Q

A global analysis approach • Find complete parameterization smooth Complete class of bijection Sets of tuning solutions satisfying parameters complexity constraint • For any choice of tuning parameters, determine the corresponding solution by convex optimization • Choose a solution that best satisfies additional design specifications (without increasing the complexity)

Other applications leading to moment problem with rationality constraints

Nevanlinna-Pick interpolation f

Tuning by moving interpolation points Covariance extension corresponds to N-P interpolation with all interpolation points at the origin ( z = 0) We moved the spectral zeros from the origin closer to the unit circle maximum entropy (LPC) Next we tune by moving the interpolation points closer to the unit circle

A tunable high resolution � spectral estimator (THREE) (1) G (z) x 1 v 1 (2) y v G (z) x 2 y 2 (n) v x n G (z) n

Estimation of spectral � lines in colored noise

Robust control

f • positive real function • f analytic for |z| ≥ 1 • Re{ f (z)} > 0 for |z| ≥ 1 f • Carathéodory function • f analytic for |z| ≤ 1 • Re{ f (z)} > 0 for |z| ≤ 1 f • Schur function • f analytic for |z| ≤ 1 • |f(z)| < 1 for |z| ≤ 1

Loop shaping in robust control d u y G(z) K(z) d y S(z) f

f The interpolants of degree at most n are parmeterized by the spectral zeros ( σ ) in a 1 − 1 fashion 4;012<-*+=*> ( ' & )*+,-./0123045 % $ # " ! !(6 !#% 89*:12301+5 76 $% 6 ! " ! ! 6 ! !6 !6 !6 !6 ?=1@/1,AB223=*0C:1A5

Example: Sensitivity shaping step r u y K(s) G(s) reference - Doyle, Francis Tannenbaum Design a strictly proper K so that the closed-loop system is • internally stable and • settling time at most 8 seconds • satisfies the specifications: • overshoot at most 10% •

(DFT) Try to achieve Internal stability requires (unstable plant poles) (nonminimum-phase plant zeros) (strictly proper controller) 120 120 Choose: 100 100 80 80 60 60 | σ | 2 40 40 20 20 0 -3 -3 -2 -2 -1 -1 0 1 2 3 DFT: deg K = 8 NPDC: deg K = 4 A. Blomqvist and R. Nagamune

• settling time at most 8 seconds DFT: deg C = 8 • overshoot at most 10% NPDC: deg C = 4 •

Multidimensional moment problems Z Z α k dµ = c k , k = 1 , 2 , . . . , n K • dµ nonnegative measure on a compact subset K of R d • α 1 , α 2 , . . . α n linearly independent basis functions defined on K Image compression

Model reduction Original system: Antoulas-Sorensen: Global-analysis approach:

A large-scale problem: � A CD player Model reduction: Antoulas-Sorensen solutions:

Global-analysis solution Antoulas-Sorensen solutions

Conclusions An enhanced theory for generalized moment problems that incorporates rationality constraints prescribed by applications. • Complete parameterizations of solutions with smooth tuning strategies. • A global analysis approach that studies the class of solutions as a whole. • Convex optimization for determining solutions.