Intervals, Tridiagonal Matrices and the Lanczos Method Wolfgang W - PowerPoint PPT Presentation

Outline Lanczos Algorithm Clustered Ritz values Conclusion Intervals, Tridiagonal Matrices and the Lanczos Method Wolfgang W¨ ulling August, 21th 2007 Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm Clustered Ritz values Conclusion 1 Lanczos Algorithm Exact Arithmetic Finite Precision Arithmetic The residual quantity 2 Clustered Ritz values The Conjecture Estimates for Residual Quantity 3 Conclusion Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Symmetric Eigenvalue Problem Given a large, sparse and symmetric matrix A ∈ R N × N , find (approximations to) eigenvalues λ and eigenvectors u , i.e. Au = λ u Lanczos Method extracts solution / approximations ( θ, z ) to eigenpair ( λ, u ) via orthogonal projection from Krylov subspace(s) K = span( q , Aq , A 2 q , .. ) Az − θ z ⊥ Ritz value θ ∈ R : θ ≈ λ . Ritz vector z ∈ K , z � = 0: z ≈ u Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Lanzcos Recursion, Matrix Notation Q k T k + β k +1 q k +1 e ∗ AQ k = (1) k Q ∗ k Q k = I k (2)   α 1 β 2 β 2 α 2 β 3     ... ... ...   ∈ R k × k T k =     ... ...   β k   β k α k Q k = [ q 1 , . . . , q k ] ∈ R N × k orthogonal Lanczos vectors algorithm terminates with a β d = 0, d ≤ N + 1. Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Ritz approximations Spectral decomposition of T k T k s • = θ • s • T k = S k diag( θ 1 , ..., θ k ) S ∗ S ∗ k S k = S k S ∗ k , k = I k Ritz values at step k are eigenvalues θ • of T k Ritz vectors are z • = Q k s • , where s • eigenvector of T k . Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Qualitiy of Ritz approximations, δ k , • Quality of Ritz approximation to eigenvalue can be controlled by easily computable residual quantity: δ k , • := β k +1 | s k , • | � � . . . where s • = , s k , • bottom element of eigenvector of T k . s k , • min | λ − θ • | ≤ || Az • − θ • z • || = || Az • − θ • z • || = δ k , • || z • || Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Stabilized Ritz values Theorem (C. Paige) Persistence Theorem min µ | θ • − µ | ≤ δ k , • , µ Ritz value in subsequent step. Definition: A Ritz value θ • is called stabilized to within δ k , • . If δ k , • is small θ • is called stabilized. Ritz value can stabilize only close to an eigenvalue Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Perturbed Lanczos Recursion Rounding errors lead to perturbed Lanczos recursion Q k T k + β k + q k +1 e ∗ AQ k = k + rounding errors (3) Q ∗ k Q k = I k + rounding errors (4) Lanczos vectors q k may lose orthogonality – even for small number of iterations, k ≪ N . It is not guaranteed that Algorithm terminates (with β d = 0), might run forever Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Quality of Ritz approximations, δ k , • Quality of Ritz approximations in f.p.computations: δ k , • = β k +1 | s k , • | controls convergence also in f.p. computations – thanks to Theorem (C. Paige) At any step k of the (f.p.) Lanczos algorithm the following is valid min | λ − θ • | ≤ max { 2 . 5( δ k , • + small in ǫ ) , small in ǫ } (5) Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Exact Arithmetic Lanczos Algorithm Finite Precision Arithmetic Clustered Ritz values The residual quantity Conclusion Interim Conclusion Residual Quantity The residual quantity δ k , • = β k +1 | s k , • | controls convergence in exact and finite precision computations! Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Tight, well separated cluster λ r λ l γ δ If δ ≪ γ : tight, well separated cluster. Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion The Conjecture Rounding errors: Multiple copies of Ritz approximations to single eigenvalue are generated. Ritz values cluster closely to eigenvalues of A . Conjecture (Strakoˇ s, Greenbaum, 1992) For any Ritz value θ • being part of a tight, well separated cluster, the value of δ k • is small, i.e δ k , • ≪ 1. Furthermore, for any Ritz value in the cluster δ k • is proportional to � � cluster diameter δ γ = gap in spectrum Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Strakoˇ s Matrix, Example 1 Example 1 (Strakoˇ s) Apply Lanczos Algorithm to diagonal matrix A with eigenvalues λ ν = λ 1 + ν − 1 N − 1( λ N − λ 1 ) ρ N − ν , ν = 2 , ..., N − 1 , (6) where λ 1 = 0 . 1, λ N = 100, ρ = 0 . 7 and N = 24. We look at cluster of two Ritz values close to λ 22 ≈ = 44 . 7944. Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Strakoˇ s Matrix, Example 1 Example 1: Cluster of two Ritz values close to λ 22 ≈ 44 . 7944. 2 10 0 10 −2 10 −4 10 −6 10 −8 10 −10 10 −12 10 −14 10 −16 10 −18 10 20 21 22 23 24 25 26 27 28 29 30 Steps k of Lanczos Algorithm Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Counterexample Consider diagonal matrix A ∈ R 23 with eigenvalues λ 1 = − 100 , λ j +1 = λ j + 0 . 1 for j = 1 , . . . , 10 λ 12 = 0 and λ 12+ j = − λ 12 − j for j = 1 , . . . , 11 . λ 1 λ 11 λ 12 = 0 λ 13 λ 23 1 23 (1 , 1 , ..., 1) ∗ : Starting vector q 1 = √ Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Counterexample � δ k δ k , first δ k , second γ 4 . 6614 × 10 − 7 9 - - 7 . 0356 × 10 1 7 . 0356 × 10 1 9 . 7023 × 10 − 05 10 2 . 3067 × 10 − 9 11 - - 7 . 0365 × 10 1 7 . 0347 × 10 1 6 . 8258 × 10 − 06 12 1 . 0724 × 10 − 11 13 - - 6 . 7432 × 10 1 7 . 3162 × 10 1 4 . 6544 × 10 − 07 14 5 . 6899 × 10 − 14 15 - - 5 . 3119 × 10 1 6 . 0079 × 10 1 3 . 0700 × 10 − 08 16 4 . 1795 × 10 − 14 17 - - 3 . 7987 × 10 − 1 3 . 4773 × 10 − 1 1 . 8764 × 10 − 09 18 6 . 5049 × 10 − 12 19 - - 1 . 5658 × 10 − 3 4 . 8528 × 10 − 4 2 . 0156 × 10 − 10 20 Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Last example: Clustered Ritz values close to eigenvalue, but � δ δ k , • ≫ γ for all Ritz value in the cluster. But in (Strakoˇ s, Greenbaum, 1992), for some particular cases � δ estimates obtained with δ k , • ≤ γ O ( || A || ). Idea from (Strakoˇ s, Greenbaum, 1992): Estimate δ k , • using only information about Ritz values in several consecutive steps. Since δ k , • = β k +1 | s k , • | ���� ���� = O ( || A || ) eigenvector element concentrate on bottom elements of T k ’s eigenvectors. Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Constant Cluster Size: k − 1, k Theorem Suppose that number of Ritz values in a tight, well separated cluster is constant at steps k − 1 and k. Then: δ � ( s k , • ) 2 < 3 . γ − δ cluster γ Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method

Outline Lanczos Algorithm The Conjecture Clustered Ritz values Estimates for Residual Quantity Conclusion Outline of the proof Proof. Observe, at the k th step, for any Ritz value θ (in the cluster) we have ψ ′ ψ k ( θ ) k ( θ ) ψ k − 1 ( θ ) = 0 , ψ k − 1 ( θ ) � = 0 . Wolfgang W¨ ulling Intervals, Tridiagonal Matrices and the Lanczos Method