Iterative methods Krylov subspace methods for symmetric eigenvalue - PowerPoint PPT Presentation

CES 703/6 Imre P´ olik Outline Power method Iterative methods Krylov subspace methods for symmetric eigenvalue problems Jacobi-Davidson method Literature Imre P´ olik, PhD McMaster University School of Computational Engineering and Science February 11, 2008

CES 703/6 Outline Imre P´ olik Outline The power method and its variants 1 Power method Power method Krylov subspace methods Inverse power method Jacobi-Davidson Rayleigh quotient iteration method Subspace iteration method Literature Krylov subspace methods 2 Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method Jacobi-Davidson method 3

CES 703/6 Power method Imre P´ olik Find the dominating eigenvalue/eigenvector Outline Power method v k +1 = y k / � y k � 2 Power method Inverse power method y k +1 = Av k +1 Rayleigh quotient iteration Subspace iteration λ k +1 = v T k +1 y k +1 method Krylov subspace methods Only multiplication is involved Jacobi-Davidson method Converges unless v 0 ⊥ v max Literature Convergence rate: | λ max /λ max − 1 | Problems multiple/close largest eigenvalues only the largest eigenvalue is computed

CES 703/6 Inverse power method Imre P´ olik � ( A − σI ) − 1 � 1 Inner eigenvalues: λ = λ ( A ) − σ Outline Apply the power method to ( A − σI ) − 1 Power method Power method Inverse power method Rayleigh quotient v k +1 = y k / � y k � 2 iteration Subspace iteration y k +1 = ( A − σI ) − 1 v k +1 method Krylov subspace methods λ k +1 = v T k +1 y k +1 Jacobi-Davidson method Converges to the dominating eigenvalue of ( A − σI ) − 1 Literature Converges unless v 0 ⊥ v max Convergence rate: | ( λ max − σ ) / ( λ max − 1 − σ ) | , linear Viable only if ( A − σI ) y = v is easily solvable

CES 703/6 Rayleigh quotient iteration Imre P´ olik Change the shift in each iteration Outline Power method v k +1 = y k / � y k � 2 Power method Inverse power method k +1 Av k +1 / � v k +1 � 2 σ k +1 = v T Rayleigh quotient iteration Subspace iteration y k +1 = ( A − σ k +1 I ) − 1 v k +1 method Krylov subspace methods λ k +1 = v T k +1 y k +1 Jacobi-Davidson method Convergence properties are unclear Literature Finds an eigenvalue faster than inverse iteration Cubic convergence Does not necessarily find λ max May not converge to an eigenvalue A − σ k +1 I will become singular New factorization in every iteration

CES 703/6 Subspace iteration method Imre P´ olik Invariant subspaces are robust, eigenvectors are not Outline     2 0 0 2 0 0 Power method 0 1 ε 0 1 0     Power method 0 ε 1 0 0 1 + ε Inverse power method Rayleigh quotient iteration It is better to identify the invariant subspaces Subspace iteration method Krylov subspace QR factorize Y k = V k +1 R k +1 methods Y k +1 = AV k +1 Jacobi-Davidson method H k +1 = V T k +1 Y k +1 Literature Y, V ∈ R n × p , H ∈ R p × p The eigenvalues of H are the largest eigenvalues of A Clustered (not multiple) eigenvalues Choosing p smartly Can also be applied to ( A − σI ) − 1 Software: EA12 in HSL

CES 703/6 Krylov subspace methods Imre P´ olik Problems with power iteration based methods Outline Extremal eigenvalues only Power method Internal eigenvalues require solution of a linear system Krylov subspace A k v is used as the best guess for an eigenvector methods Rayleigh-Ritz method � � v, Av, . . . , A k v Krylov subspace: span Lanczos method Implicitly restarted Lanczos method Find the best approximate eigenvector (Ritz vectors) Band Lanczos method Jacobi-Davidson Columns of Q k are orthogonal, span Krylov space method λ ( Q T k AQ k ) approximates λ ( A ) Literature Choose a Q k to simplify the structure of Q T k AQ k � �� � T k

CES 703/6 Lanczos method Imre P´ olik Gradually build the Krylov subspace Outline Maintain an orthogonal basis Q k , T k tridiagonal Power method Find the corresponding Ritz vectors Krylov subspace methods Rayleigh-Ritz method Lanczos method v 0 = 0 , β 1 = 0 , v 1 random unit Implicitly restarted Lanczos method Band Lanczos method repeat Jacobi-Davidson q j = Av j − β j v j − 1 method Literature α j = q T j v j q j = q j − α j v j β j +1 = � q j � v j +1 = q j /β j +1 Extreme eigenvalues converge first Can also be applied to ( A − σI ) − 1 Memory consumption increases

CES 703/6 Implicitly restarted Lanczos method Imre P´ olik Prevents k growing too much Outline Applies shifts µ i to the algorithm Power method Equivalently, changes v 0 Krylov subspace methods Rayleigh-Ritz method How to choose the shifts? Lanczos method Implicitly restarted Flexible eigenvalue configurations Lanczos method Band Lanczos method Locking/purging eigenvalues Jacobi-Davidson method Software: ARPACK (also in Matlab) Literature

CES 703/6 Band Lanczos method Imre P´ olik Multiple starting vectors, finds more eigenvalues Outline � � V, AV, . . . , A k V span Power method Suitable for multiple/clustered eigenvalues Krylov subspace methods T is block tridiagonal Rayleigh-Ritz method Lanczos method Implicitly restarted Lanczos method Band Lanczos method Jacobi-Davidson method Literature

CES 703/6 Jacobi-Davidson method Imre P´ olik Problems with Lanczos Outline only efficient if the eigenvalues are well separated Power method needs ( A − σI ) − 1 y for internal eigenvalues Krylov subspace methods Build a different set of orthogonal vectors spanning the Jacobi-Davidson Galerkin vectors method Interior eigenvalues without inversion Literature Very good if A has multiple eigenvalues Software: JDQR (Matlab)

Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der CES 703/6 Vorst, editors. Imre P´ olik Templates for the Solution of Algebraic Eigenvalue Problems: Outline A Practical Guide . Power method SIAM, Philadelphia, 2000. Krylov subspace methods James W. Demmel. Jacobi-Davidson method Applied Numerical Linear Algebra . Literature SIAM, Philadelphia, 1997.