Efficient Algorithms for Structured Matrices Vadim Olshevsky Efficient Algorithms for Structured Matrices Vadim Olshevsky www.math.uconn.edu/ ˜ olshevsky University of Connecticut Joint works with T.Bella (U Conn), Yu.Eidelman (Tel Aviv), I.Gohberg (Tel Aviv), A.Shokrollahi (EPFL). MTNS, Kyoto, Japan, July 2006 This work was supported by the NSF grants CCR 0242518 . – Typeset by Foil T EX – 1
Efficient Algorithms for Structured Matrices Vadim Olshevsky Contents • Examples and introduction. • Algorithms. – Tangential Interpolation. – Displacement structure. Fast algorithms. – Superfast algorithms. • Applications – List Decoding. • Quasiseparable matrices and eigenvalue problems – Typeset by Foil T EX – 2
Efficient Algorithms for Structured Matrices Vadim Olshevsky Part 1. Examples of structured matrices � t i − j � � � x j − 1 Toeplitz, T = Vandermonde, V = i x n − 1 x 2 t 0 · · · · · · t − n +1 1 x 1 · · · t − 1 1 1 . . . . . . . . . . t 1 t 0 . . . . . t − 1 . . ... ... ... . . . . . . . . . . . . . . . . . ... . x 2 x n − 1 . t 0 t − 1 1 x n · · · n n t n − 1 · · · · · · t 1 t 0 � � � 1 − x i x ∗ � 1 j Cauchy, C = Pick, P = x i − y j z i + z ∗ j 1 − x 1 · x 1 ∗ 1 − x 1 · x n ∗ · · · 1 1 z 1 + z 1 ∗ z 1 + z n ∗ · · · . . . . x 1 − y 1 x 1 − y n . . . . . . . . 1 − x n · x 1 ∗ 1 − x n · x n ∗ · · · 1 1 · · · z n + z 1 ∗ z n + z n ∗ x n − y 1 x n − y n • Exploiting the structure ⇛ Fast algorithms. O ( n 3 ) flops General matrices Standard algorithms Gaussian Elimination O ( n 2 ) Toeplitz Matrices Fast algorithms Levinson and Schur O ( n log 2 n ) Toeplitz Matrices Superfast algorithms Brent-Gustafson-Yan – Typeset by Foil T EX – 3
Efficient Algorithms for Structured Matrices Vadim Olshevsky Growing interest • Conferences: Santa Barbara-1996, Boulder-1999 (AMS), Cortona-2000, South Hadley-2001 (AMS), Moscow-2003, Cortona-2004. • Special sessions and minisymposia at SIAM, ILAS, MTNS, IWOTA (2002, 2003, 2004, 2005, 2006) and SPIE conferences. • Recent publishing projects: – 1999. Kailath & Sayed, Eds, SIAM Publicatiosn. – 2000. Bini, Tyrtyshnikov and Yalamov, Eds, Nova Publications. – 2001. Two-Volume set (38 papers) edited V.O. to be published by AMS. – 2002. A special issue of LAA, Dewilde, Sayed & V.O. Eds. – 2003. Fast algorithms for structured matrices, V.O., ed., AMS and SIAM. —————————————– • 2004-2006. Semiseparable matrices. Cortona-2004, FOCM-2005, Householder-2005, IWOTA-2005, Moscow-2005, ILAS-2006. – Typeset by Foil T EX – 4
Efficient Algorithms for Structured Matrices Vadim Olshevsky Examples of efficient algorithms • Full research cycle . – Extraction. Applications. Mathematical models. ∗ Estimation, Signal processing, Filtering, Circuit simulations, Systems and Control, Image Processing, Error-correcting codes. – Processing. Fundamental Mathematics. ∗ Various interpolation and Approximation problems, Orthogonal polynomials, Moment problems, RKHS, Commutant-lifting theory, etc. – Injection. Algorithm development. Software design. • A RECENT DISCOVERY : Though fast algorithms were believed to be typically inaccurate, it turns out that there are better modified algorithms that BLEND SPEED AND ACCURACY. – Typeset by Foil T EX – 5
Efficient Algorithms for Structured Matrices Vadim Olshevsky Example. Positive definite Hankel matrices (Injection) h 0 h 2 · · · h n − 1 h 1 . ... . h 2 . h 1 ... h 2 h 2 n − 3 H = > 0 . ... . . h 2 n − 3 h 2n − 21 h n − 1 · · · h 2 n − 3 h 2 n − 1 h 2n − 2 Conditioning • Tyrtyshnikov (1994): k 2 ( H ) = � H � 2 � H − 1 � 2 > 2 n − 6 – Typeset by Foil T EX – 6
Efficient Algorithms for Structured Matrices Vadim Olshevsky Backward Stable Fast Algorithms (Injection) • Slow and accurate: Gaussian elimination: H = LL T – Exact arithmetic: H ≈ � L � L T – Floating point arithmetic: – Cost : O ( n 3 ) . � H − � L � L T � 2 < O ( ǫ ) � H � 2 . – Stability: • Fast: (There are many...) – Cost : O ( n 2 ) or O ( n log 2 n ) . – Stability: ? . • Fast and Accurate [OS01a] : – Cost : O ( n 2 ) . � H − � L � L T � 2 < O ( ǫ ) � H � 2 . – Stability: – Typeset by Foil T EX – 7
Efficient Algorithms for Structured Matrices Vadim Olshevsky Part 2. The confluent tangential Nevanlinna-Pick problem The classical scalar Nevanlinna-Pick problem points z 1 , . . . , z n in the RHP: Re z k ≥ 0 • Given: values f 1 , . . . , f n in the unit disk: | f k | < 1 • Construct an interpolant f ( z k ) = f k , such that passive | f ( z ) | < 1 in the RHP (i) f ( z ) is (ii) f ( z ) is rational The Nevanlinna algorithm (1929) Its matrix interpretation computation of the The Cholesky factorization R = LL ∗ , [ L is lower triangular] for the Pick interpolant f ( z ) � 1 − f i f ∗ � j matrix R = z i + z ∗ j O ( n 2 ) O ( n 3 ) We shall show how to do faster than O ( n 2 ) – Typeset by Foil T EX – 8
Efficient Algorithms for Structured Matrices Vadim Olshevsky Applications to system and circuit theory • Many physical phenomena can be described by a LTI ✲ Output Linear Time-Invariant system Input ✲ system ✲ y ( z ) = f ( z ) · u ( z ) f ( z ) u ( z ) ✲ ∆ ∆ = | f ( x ) | ≤ 1 = Conservation of energy Passivity • Energy of input u ( z ) ≥ Energy of output y ( z ) . • Many engineering applications have been discovered, e.g., the classical Nevanlinna (1929) algorithm is known under the name “Darlington (1939) synthesis procedure” – Typeset by Foil T EX – 9
Efficient Algorithms for Structured Matrices Vadim Olshevsky MIMO systems. Tangential interpolation • MIMO systems (multi-input-multi-output) give rise to matrix rational functions � p ij ( z ) � Θ( z ) = . q ij ( z ) x N ✲ ✲ y M . . . . . . Θ( z ) ✲ ✲ x 2 y 2 x 1 ✲ ✲ y 1 • This gives rise to the tangential interpolation conditions: � � � � x 1 · · · x N · Θ( z k ) = y 1 · · · y M – Typeset by Foil T EX – 10
Efficient Algorithms for Structured Matrices Vadim Olshevsky Confluent interpolation conditions • Given: Now for each interpolation point z k we are given two chains of vectors: { x k 1 , . . . , x k,m k } { y k 1 , . . . , y k,m k } • We consider confluent interpolation conditions: F ( mk − 1) ( z k ) F ′ ( z k ) F ( z k ) · · · · · · ( m k − 1)! . ... . 0 F ( z k ) . � x k 1 � . ... ... . . . x k,m k × = . 0 0 . . ... ... . F ′ ( z k ) . 0 · · · · · · 0 F ( z k ) � y k, 1 � · · · y k,m k . – Typeset by Foil T EX – 11
Efficient Algorithms for Structured Matrices Vadim Olshevsky Example: linear pencil and Jordan chains • Let F ′ ( z ) = − I, F ′′ ( z ) = 0 . F ( z ) = A − zI, then • If ( A − z k I ) − I 0 � u k 1 � � 0 0 � = u k 2 u k 3 0 ( A − z k I ) − I 0 · . 0 0 ( A − z k I ) • Hence u k 1 is the (left) eigenvector : u k 1 ( A − z k I ) = 0 . • Further, u k 2 is the first generalized eigenvector u k 2 ( A − z k I ) = u k 1 . • Finally, u k 3 is the second generalized eigenvector : u k 3 ( A − z k I ) = u k 2 . – Typeset by Foil T EX – 12
Efficient Algorithms for Structured Matrices Vadim Olshevsky Computational cost of our interpolation problem { z k } r are given r points k =1 { m k } r are their multiplicities k =1 • Given: { x k 1 , . . . , x k,m k } r are the associated chains k =1 { y k 1 , . . . , y k,m k } r k =1 • The cost C ( n ) is [OS01b] : C ( n ) = O ( n log 2 n · [1 + � r n log 2 n n log 3 n m k n log n ≤ m k ]) ≤ � �� � � �� � k =1 Caratheodory-Fejer Nevanlinna-Pick arithmetic operations, where n = m 1 + . . . + m r HOW? VIA STRUCTURED MATRICES. We present next a matrix description of the algorithm – Typeset by Foil T EX – 13
Efficient Algorithms for Structured Matrices Vadim Olshevsky Part 3. Fast multiplication algorithms for structured matrices. Toeplitz matrices. Discrete Convolution. • Matrix-vector product: a 0 b 0 a 0 b 0 a 1 a 0 b 1 a 1 b 0 + a 0 b 1 a 2 a 1 a 0 b 2 a 2 b 0 + a 1 b 1 + a 0 b 3 = . . . ... ... ... . . . . . . · · · a 2 a 1 a 0 • Polynomials: ( a 0 + a 1 z + a 2 z 2 + . . . )( b 0 + b 1 z + b 2 z 2 + . . . ) = a 0 b 0 +( a 1 b 0 + a 0 b 1 ) z +( a 2 b 0 + a 1 b 1 + a 0 b 3 ) z 2 + . . . – Typeset by Foil T EX – 14
Recommend
More recommend
