http://www.sam. ma th. et hz . h /~ mh g Deflation based preconditioning of linear systems of equations Martin H. Gutknecht Seminar for Applied Mathematics, ETH Zurich SC2011 — International Conference on Scientific Computing Santa Margherita di Pula, Sardinia, Italy October 10–14, 2011 Partly joint work with André Gaul, Jörg Liesen, Reinhard Nabben
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Outline Prerequisites History Augmentation and Deflation Deflated GMR ES and M IN R ES Oblique projections and truly deflated GMR ES Deflated QMR Conclusions M.H. Gutknecht SC2011 p. 2
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Iterative methods based on (Petrov-)Galerkin condition Ax = b A ∈ C N × N nonsingular. To solve: with Idea: compute sequence of approximate solutions x n such that their residuals r n : ≡ b − Ax n approach o in some norm. We choose x n from an n -dimensional affine search space x 0 + S n such that some Galerkin or Petrov-Galerkin condition is satisfied: x n ∈ x 0 + S n , r n = A ( x ⋆ − x n ) ⊥ � S n . That is, r n ∈ r 0 + A S n , r n ⊥ � S n . This means that r 0 is approximated from A S n such that “error” r n ⊥ � S n . M.H. Gutknecht SC2011 p. 3
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Simplified idea of deflation based preconditioning Ideal assumption: columns of U ∈ C N × k span an invariant subspace U of A belonging to eigenvalues close to 0 . Z : ≡ AU , Z : ≡ A U = U . Let Note: images of the restriction A − 1 � � Z are trivial to compute: if z = Zc ∈ Z , then A − 1 z = Uc . Z ⊕ Z ⊥ = C N . Main idea: split up C N into Split up r 0 accordingly: r 0 = r 0 − � r 0 r 0 � + . � �� � ���� ∈ Z ∈ Z ⊥ A − 1 ( r 0 − � r 0 ) is trivial to invert; A − 1 � r 0 will be approximated with a Krylov space solver. Essentially, the solver will act on Z ⊥ . M.H. Gutknecht SC2011 p. 4
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Since the (absolutely) small eigenvalues of A cause trouble in A on Z ⊥ , such that � the solver, we want to replace A by � A will no longer have these small eigenvalues (deflation). A will have the form � A : ≡ PA or � A : ≡ PAP . This looks like � preconditioning, but in our case P will be a projection. � � A Z ⊥ = � A � � Hopefully, Z ⊥ . Problems: Need work out details. E.g., how define/compute P , � A . We do not want to assume that Z is exactly A –invariant. Orthogonal decomposition Z ⊕ Z ⊥ turns out to be incompatible with CG optimality. � � If A is non-Hermitian, A Z ⊥ = � A � � Z ⊥ will not hold, even when Z is A -invariant. Need some approximate invariant subspace. M.H. Gutknecht SC2011 p. 5
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions How to find an approximate invariant subspace? It may be known from a theoretical analysis of the problem. It may result from the solution of previous systems with the same A . ( � linear system with multiple right-hand sides.) It may results from the solution of previous systems with nearby A . It may results from previous cycles of the solution process (if the method is restarted). There are lots of examples in the literature. M.H. Gutknecht SC2011 p. 6
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Prerequisites: Krylov (sub)space solvers (KSS) Given: linear system Ax = b , initial approx. x 0 ∈ C N . Construct: approximate solutions (“iterates”) x n and corresponding residuals r n : ≡ b − Ax n with x n ∈ x 0 + K n ( A , r 0 ) , r n ∈ r 0 + A K n ( A , r 0 ) , where r 0 : ≡ b − Ax 0 is the initial residual, and K n : ≡ K n ( A , r 0 ) : ≡ span { r 0 , Ar 0 , . . . , A n − 1 r 0 } is the n th Krylov subspace generated by A from r 0 . We can, e.g., construct x n such that � r n � is minimal. conjugate residual ( CR ) method (Stiefel, 1955), � M IN R ES (Paige and Saunders, 1975), � GCR and GMR ES . � M.H. Gutknecht SC2011 p. 7
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Prerequisites: preconditioning In practice, Krylov space solvers often do not work well without preconditioning : multiplication of A by some approximate inverse P , so that PA or AP is better conditioned than A . Normally, A and P ≈ A − 1 are nonsingular. Here we consider an alternative to preconditioning: (approximate) spectral deflation . Formally, it sometimes looks like preconditioning, but (in most cases) P is singular. So, PA is singular too. But we apply this formally preconditioned matrix or deflated matrix only in a suitably chosen invariant subspace. M.H. Gutknecht SC2011 p. 8
max | Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Buzz words and their meanings x n ∈ x 0 + K n ( � A , � r 0 ) + U , where Augmented bases: A = A � spec ( � A ) ⊂ spec ( A ) ∪ { 0 } or A � � A : ≡ PA s.t. small EVals � 0 (Spectral) deflation: A � � A : ≡ AP s.t. small EVals � | λ EVal translation: choice of U based on prev. cycles Krylov space recycling: adaptation of P at each restart Flexible KSS : While (spectral) deflation has been an indispensable tool for eigenvalue computations for at least 55 years, for solving linear systems deflation has become popular in the last 20 years only. Two basic approaches: Augmentation of basis with or without spectral deflation. EVal translation by suitable preconditioning. M.H. Gutknecht SC2011 p. 9
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions History Early contributions (many more papers appeared since): Nicolaides ’85 / ’87 SINUM : deflated 3-term CG (w/augm. basis) Dostál ’87 / ’88 IntJCompMath : deflated 2-term CG (w/augm. basis) Kharchenko / Yeremin ’92 / ’95 NLAA : GMRES with transl. EVals Morgan ’93 / ’95 SIMAX : GMRES with augmented basis de Sturler ’93 / ’96 JCAM : inner-outer GMRES/GCR (and, briefly, inner/outer BiCGStab/GCR) with augmented basis Erhel / Burrage / Pohl ’94 / ’96 JCAM GMRES with transl. EVals Chapman / Saad ’95 / ’97 NLAA GMRES with augmented basis Saad ’95 / ’97 SIMAX Analysis of KSS with augmented basis Burrage, / Erhel / Pohl / Williams ’95 / ’98 SISC Deflated stationary inner-outer iterations Baglama / Calvetti / Golub / Reichel ’96 / ’98 SISC Adaptively preconditioned GMRES M.H. Gutknecht SC2011 p. 10
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions History (contn’d) More recently, it was discovered by a group of authors that augmentation and deflation (= deflation based preconditioning) is algebraically very similar to multigrid, balancing Neumann-Neumann preconditioning (see Mandel ’93 CommApplNumMeth ). See, in particular: Erlangga / Nabben ’08 SIMAX , ’09 SISC Nabben / Vuik ’08 NLAA Tang / Nabben / Vuik/ Erlangga ’09 SISC M.H. Gutknecht SC2011 p. 11
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Augmentation and deflation based on orthogonal projection: the Wang/de Sturler/Paulino (2006) approach Let U ∈ C N × k contain approx. EVecs corr. to EVals close to 0. Define U : ≡ R ( U ) , Z : ≡ AU , Z : ≡ R ( Z ) = A U , Q : ≡ ZE − 1 Z H , P : ≡ I − Q = I − ZE − 1 Z H . E : ≡ Z H Z , Note that Q 2 = Q , P 2 = P , Q H = Q , P H = P . So, Q is the orthogonal projection onto Z ; dim Z = k , P is the orthogonal projection onto Z ⊥ ; dim Z ⊥ = N − k . r 0 : ≡ Pr 0 , A : ≡ PAP , � � Let K n : ≡ K n ( � A , � r 0 ) : ≡ span ( � r 0 , � A � r 0 , . . . , � A r 0 ) . � n − 1 � We choose x n ∈ x 0 + � r n : ≡ b − Ax n ∈ r 0 + A � K n + U , K n + Z . (1) M.H. Gutknecht SC2011 p. 12
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions In the inclusions x n ∈ x 0 + � r n ∈ r 0 + A � K n + U , K n + Z � K n ⊂ Z ⊥ . we have So, if Z ⊥ is an invariant subspace, A � K n ⊂ Z ⊥ . Then we could split r 0 − r n into two orthogonal components: K n ⊕ Z ⊂ Z ⊥ ⊕ Z . r 0 − r n ∈ A � A � K n ∩ Z � = { o } . But, in general, As mentioned, it is trivial to invert A on Z . So, if we split r 0 into r 0 = Pr 0 + Qr 0 ∈ Z ⊥ ⊕ Z , we are left with the problem of approximating A − 1 Pr 0 . When computing it, we may generate an extra component in Z , which we will avoid by replacing A by � A . M.H. Gutknecht SC2011 p. 13
Prereq. History Augment./Deflat. Defl.GMR ES Oblique projs. Defl.QMR Conclusions Deflated GMR ES x n ∈ x 0 + � with minimum � r n � 2 We can compute K n + U by a GMR ES -like method. Assume the cols. of Z are orthonormal, so that Q = ZZ H . Apply Arnoldi process to get ONBs for spaces � K n : AV n = V n + 1 H n , � v 0 : ≡ � r 0 /β . where AV n = PAPV n = PAV n . Note that here � Using coordinate vectors k n ∈ C n and m n ∈ C k we write x n = x 0 + V n k n + Um n , (2) so that r n = r 0 − AV n k n − Zm n . (3) M.H. Gutknecht SC2011 p. 14
Recommend
More recommend
