Max-min and min-max approximation problems for normal matrices - PowerPoint PPT Presentation

Max-min and min-max approximation problems for normal matrices revisited Petr Tichý Czech Academy of Sciences, University of West Bohemia joint work with Jörg Liesen TU Berlin January 30, SNA 2014, Nymburk, Czech Republic 1

Bounding GMRES residual norm A x = b , A ∈ C n × n is nonsingular, b ∈ C n , x 0 = 0 and � b � = 1 for simplicity . GMRES computes x k ∈ K k ( A , b ) such that r k ≡ b − A x k satisfies � r k � = p ∈ π k � p ( A ) b � min (GMRES) ≤ � b � =1 min max p ∈ π k � p ( A ) b � (worst-case GMRES) ≤ p ∈ π k � p ( A ) � min (ideal GMRES) where π k = degree ≤ k polynomials with p (0) = 1 . 2

Two bounds on the GMRES residual norm � b � =1 min max p ∈ π k � p ( A ) b � ≤ min p ∈ π k � p ( A ) � They are equal if A is normal . [Greenbaum, Gurvits ’94; Joubert ’94] . The inequality can be strict if A is non-normal . [Toh ’97; Faber, Joubert, Knill, Manteuffel ’96] . 3

How to prove the equality for normal matrices? If A is normal, then � b � =1 min max p ∈ π k � p ( A ) b � = min p ∈ π k � p ( A ) � . [Joubert ’94] Proof using analytic methods of optimization theory , for real or complex data, only in the GMRES context. [Greenbaum, Gurvits ’94] : Proof based mostly on matrix theory , only for real data but in a more general form. These proofs are quite complicated . Is there a straightforward proof that uses, e.g., known classical results of approximation theory ? 4

Outline Normal matrices and classical approximation problems 1 Best polynomial approximation for f on Γ 2 Proof 3 Connection to results by Greenbaum and Gurvits 4 5

Link to classical approximation problems A is normal iff A = QΛQ ∗ , Q ∗ Q = I . Γ ≡ { λ 1 , . . . , λ n } is the set of eigenvalues of A . For any function g defined on Γ denote � g � Γ ≡ max z ∈ Γ | g ( z ) | . p ∈ π k means k α i z i . � p ( z ) = 1 − i =1 Then p ∈ π k � Q p ( Λ ) Q ∗ � = min min p ∈ π k � p ( A ) � = min p ∈ π k max | p ( λ i ) | λ i k � � � � α i z i � = min � 1 − . � � α 1 ,...,α k � � i =1 � Γ 7

Generalization Instead of 1 we consider a general function f defined on Γ . Instead of { z i } k i =1 we consider general basis functions ϕ i . We ask whether � b � =1 min max p ∈P k � f ( A ) b − p ( A ) b � = min p ∈P k � f ( A ) − p ( A ) � where A is normal and p is of the form k � p ( z ) = α i ϕ i ( z ) ∈ P k . i =1 A comment on R versus C → coefficients α i . As in the previous p ∈P k � f ( A ) − p ( A ) � min = p ∈P k � f ( z ) − p ( z ) � Γ . min 8

A polynomial of best approximation for f on Γ Definition and notation p ∗ ∈ P k is a polynomial of best approximation for f on Γ when � f − p ∗ � Γ = min p ∈P k � f − p � Γ . For p ∈ P k , define Γ( p ) ≡ { z ∈ Γ : | f ( z ) − p ( z ) | = � f − p � Γ } . 10

Characterization of best approximation for f on Γ [Chebyshev, Berstein, de la Vallée Poussing, Haar, Remez, Zuhovicki˘ ı, Kolmogorov] [Rivlin, Shapiro ’61] , [Lorentz ’86] Characterization theorem (complex case) p ∗ ∈ P k is a polynomial of best approximation for f on Γ if and only if there exist ℓ points µ i ∈ Γ( p ∗ ) where 1 ≤ ℓ ≤ 2 k + 1 , and ℓ real numbers ω 1 , . . . , ω ℓ > 0 with ω 1 + · · · + ω ℓ = 1 , such that ℓ � ω j p ( µ j ) [ f ( µ j ) − p ∗ ( µ j )] = 0 , ∀ p ∈ P k . j =1 Denote δ ≡ � f − p ∗ � Γ = | f ( µ j ) − p ∗ ( µ j ) | , j = 1 , . . . , ℓ . 11

Proof I It suffices to prove that � b � =1 min max p ∈P k � f ( A ) b − p ( A ) b � ≥ p ∈P k � f ( A ) − p ( A ) � min = p ∈P k � f ( z ) − p ( z ) � Γ . min Suppose that the eigenvalues of A are sorted such that λ j = µ j , j = 1 , . . . , ℓ. Define the vector w ξ ≡ [ √ ω 1 , . . . , √ ω ℓ , 0 , . . . , 0] T . w = Q ξ, Then ℓ � 0 = ω j p ( µ j ) [ f ( µ j ) − p ∗ ( µ j )] j =1 ξ H p ( Λ ) H [ f ( Λ ) − p ∗ ( Λ )] ξ = w H p ( A ) H [ f ( A ) − p ∗ ( A )] w . = 13

Proof II In other words, f ( A ) b − p ∗ ( A ) w ⊥ p ( A ) w , ∀ p ∈ P k , or, equivalently, � f ( A ) w − p ∗ ( A ) w � = min p ∈P k � f ( A ) w − p ( A ) w � . Moreover � f ( A ) w − p ∗ ( A ) w � 2 � [ f ( Λ ) − p ∗ ( Λ )] ξ � 2 = ℓ j | f ( µ j ) − p ∗ ( µ j ) | 2 � ξ 2 = j =1 ℓ ω j δ 2 = δ 2 � = j =1 � f ( A ) − p ∗ ( A ) � 2 . = 14

Proof III In summary, for p ∗ ∈ P k we have constructed w ∈ C n such that p ∈P k � f ( A ) − p ( A ) � min = � f ( A ) − p ∗ ( A ) � � f ( A ) w − p ∗ ( A ) w � 2 = = p ∈P k � f ( A ) w − p ( A ) w � min ≤ � b � =1 min max p ∈P k � f ( A ) b − p ( A ) b � . The proof for complex A is finished. 15

A note on the real case Assume that A , f ( A ) and ϕ i ( A ) are real . We look for a polynomial of a best approximation with real coefficients . Technical problem : A can have complex eigenvalues but we look for a real vector b that maximizes p ∈P k � f ( A ) b − p ( A ) b � . min Γ is a set of points that appear in complex conjugate pairs . This symmetry with respect to the real axes has been used to find a real b and to prove the equality [Liesen, T. 2013] . 16

Results by Greenbaum and Gurvits, Horn and Johnson Theorem [Greenbaum, Gurvits ’94] Let A 0 , A 1 , . . . , A k be normal matrices that commute . Then k k � � � v � =1 min max α 1 ,...,α k � A 0 v − α i A i v � = α 1 ,...,α k � A 0 − min α i A i � . i =1 i =1 Theorem [Theorem 2.5.5, Horn, Johnson ’90] Commuting normal matrices can be simultaneously unitarily diagonalized , i.e., there exists a unitary U so that U H A i U = Λ i , i = 0 , 1 , . . . , k. 18

Connection to results by Greenbaum and Gurvits Using the theorem by Horn and Johnson we can equivalently rewrite the problem k � α 1 ,...,α k � A 0 − min α i A i � i =1 in our notation k � α 1 ,...,α k � f ( A ) − min α i ϕ i ( A ) � i =1 where A is any diagonal matrix with distinct eigenvalues and f and ϕ i are any functions satisfying f ( A ) = Λ 0 , ϕ i ( A ) = Λ i , i = 1 , . . . , k. 19

Summary Inspired by the convergence analysis of GMRES we formulated two general approximation problems involving normal matrices. We used a direct link between approximation problems involving normal matrices, classical approximation problems and proved that � b � =1 min max p ∈P k � f ( A ) b − p ( A ) b � = min p ∈P k � f ( A ) − p ( A ) � . Our results represent a generalization of results by [Joubert ’94] , offer another point of view to [Greenbaum, Gurvits ’94] . 20

Related papers J. Liesen and P. Tichý , [Max-min and min-max approximation problems for normal matrices revisited, submitted to ETNA (2013).] A. Greenbaum and L. Gurvits , [Max-min properties of matrix factor norms, SISC, 15 (1994), pp. 348–358.] W. Joubert , [A robust GMRES-based adaptive polynomial preconditioning algorithm for nonsymmetric linear systems, SISC, 15 (1994), pp. 427–439.] M. Bellalij, Y. Saad, and H. Sadok , [Analysis of some Krylov subspace methods for normal matrices via approximation theory and convex optimization, ETNA, 33 (2008/09), pp. 17–30.] Thank you for your attention! 21