On optimal short recurrences for generating orthogonal Krylov subspace bases J¨ org Liesen based on joint work with Zdenˇ ek Strakoˇ s and Petr Tich´ y (Czech Academy of Sciences), Vance Faber (BD Biosciences, Seattle), Beresford Parlett (Berkeley), and Paul Saylor (Illinois)
Overview 1. Introduction: Krylov subspace methods 2. Optimal short recurrences 3. Characterization and examples
Introduction: Krylov subspace methods (1) • Methods that are based on ���������� ���� ��� ������ ��������� K � ( A, v � ) ≡ span { v � , Av � , . . . , A � − � v � } , n = 1 , 2 , . . . , where A is a given square matrix and v � is the initial vector. • Must generate bases of K � ( A, v � ), n = 1 , 2 , . . . . • Trivial choice: v � , Av � , . . . , A � − � v � . This is computationally infeasible (recall the Power Method).
Introduction: Krylov subspace methods (1) • Methods that are based on ���������� ���� ��� ������ ��������� K � ( A, v � ) ≡ span { v � , Av � , . . . , A � − � v � } , n = 1 , 2 , . . . , where A is a given square matrix and v � is the initial vector. • Must generate bases of K � ( A, v � ), n = 1 , 2 , . . . . • Trivial choice: v � , Av � , . . . , A � − � v � . This is computationally infeasible (recall the Power Method). • For numerical stability: Well conditioned basis. • For computational efficiency: Short recurrence. • Best of both worlds: ���������� ����� �������� �� ����� ���������� . • First such method for Ax = b : Conjugate gradient (CG) method of Hestenes and Stiefel (1952).
Introduction: Krylov subspace methods (2) The classical CG method of Hestenes and Stiefel (US National Bureau of Standards Preprint No. 1659, March 10, 1952) The residual vectors r � , r � , .. . , r � − � are generated by a short recurrence and form an orthogonal basis of K � ( A, r � ).
Introduction: Krylov subspace methods (3) • CG is for symmetric positive definite A . • (Paige and Saunders, 1975): Short recurrence & orthogonal basis methods for symmetric A .
Introduction: Krylov subspace methods (4) • By the end of the 1970s it was unknown if such methods existed also for general unsymmetric A . • ���� ����� ����� ���� ����������� �������� at Gatlinburg VIII (now Householder VIII) held in Oxford from July 5 to 11, 1981: ���� ���� ���� �����
Introduction: Krylov subspace methods (5) • We want to solve Ax = b iteratively, starting from x � . • Step n = 1 , 2 , . . . : x � = x � − � + α � − � p � − � , direction vector p � − � , scalar α � − � (both to be determined). • Krylov subspace method: span { p � , . . . , p � − � } = K � ( A, v � ) ( v � = r � = b − Ax � ). • CGClike descent method: Error is minimized in some given inner product norm, � � � � = �� , �� � � � � .
Introduction: Krylov subspace methods (5) • We want to solve Ax = b iteratively, starting from x � . • Step n = 1 , 2 , . . . : x � = x � − � + α � − � p � − � , direction vector p � − � , scalar α � − � (both to be determined). • Krylov subspace method: span { p � , . . . , p � − � } = K � ( A, v � ) ( v � = r � = b − Ax � ). • CGClike descent method: Error is minimized in some given inner product norm, � � � � = �� , �� � � � � . • � x − x � � � is minimal iff x − x � ⊥ � span { p � , . . . , p � − � } . • By construction, this is satisfied iff α � − � = � x − x � − � , p � − � � � and � p � − � , p � � � = 0 , j = 0 , . . . , n − 2 , � p � − � , p � − � � � i.e. p � , . . . , p � − � ���� �� � B ����������� ����� �� K � ( A, v � ) �
Introduction: Krylov subspace methods (6) • Faber and Manteuffel answered Golub’s question in 1984: For a general matrix A there exists �� CGClike descent method �������� ������� �� �������� ��� ������������ �� ��� ����� ������� ���� ��� ��� ������� �� ���� �������
Optimal short recurrences (1) Notation: • Matrix A ∈ � � × � , nonsingular. • Matrix B ∈ � � × � , Hermitian positive definite (HPD), defining the B Cinner product, � x, y � � ≡ y ∗ Bx . • Initial vector v � ∈ � � . • d = d ( A, v � ), the grade of v � with respect to A , K � ( A, v � ) ⊂ . . . ⊂ K � ( A, v � ) = K � �� ( A, v � ) = . . . = K � ( A, v � ) .
Optimal short recurrences (1) Notation: • Matrix A ∈ � � × � , nonsingular. • Matrix B ∈ � � × � , Hermitian positive definite (HPD), defining the B Cinner product, � x, y � � ≡ y ∗ Bx . • Initial vector v � ∈ � � . • d = d ( A, v � ), the grade of v � with respect to A , K � ( A, v � ) ⊂ . . . ⊂ K � ( A, v � ) = K � �� ( A, v � ) = . . . = K � ( A, v � ) . Our goal: Generate a B Corthogonal basis v � , . . . , v � of K � ( A, v � ). 1. span { v � , . . . , v � } = K � ( A, v � ) , for n = 1 , . . . , d , 2. � v � , v � � � = 0 , for j � = k , j, k = 1 , . . . , d .
Optimal short recurrences (2) • Standard way for generating the B Corthogonal basis: Arnoldi’s method. � v � �� = Av � − h ��� v � , n = 1 , . . . , d − 1 , � �� � Av � , v � � � h ��� = , d = dim K � ( A, v � ) . � v � , v � � � (No normalization for convenience.)
Optimal short recurrences (2) • Standard way for generating the B Corthogonal basis: Arnoldi’s method. � v � �� = Av � − h ��� v � , n = 1 , . . . , d − 1 , � �� � Av � , v � � � h ��� = , d = dim K � ( A, v � ) . � v � , v � � � (No normalization for convenience.) matrix of size d × ( d − 1) • Rewritten in matrix notation: AV � − � = V � H ��� − � , where h � � � � � � h � �� − � . ... . 1 . ... V � ≡ [ v � , . . . , v � ] , H ��� − � ≡ h � − � �� − � 1 V ∗ � BV � is diagonal , d = dim K � ( A, v � ) .
Optimal short recurrences (3) • The ���� ���������� in Arnoldi’s method, � v � �� = Av � − h ��� v � , n = 1 , . . . , d − 1 , � �� is an ������� � s + 2 ������ ���������� when � v � �� = Av � − h ��� v � , n = 1 , . . . , d − 1 . � � � − � • For s = 1: Optimal 3Cterm recurrence, v � �� = Av � − h ��� v � − h � − � �� v � − � • Why ������� ? �� ���� ��� �������������� ���� � �� ���������� � ���� ��� �������� � ! � ������� ��� ��"������
Optimal short recurrences (4) s d − s − 2 � �� � � �� � largest upper triangle that is zero ∗ ∗ � � � ∗ 0 � � � 0 . ... . ∗ ∗ ∗ � � � ∗ . ... ... ... ... 0 Optimal ( s + 2)Cterm recurrence: ... ... ... ∗ H ��� − � �� ( s + 2) ����� ��������� . ... ... ... . . ... ... ����� ������ ���������� � ������������ ∗ ... ∗ ∗
Optimal short recurrences (4) s d − s − 2 � �� � � �� � largest upper triangle that is zero ∗ ∗ � � � ∗ 0 � � � 0 . ... . ∗ ∗ ∗ � � � ∗ . ... ... ... ... 0 Optimal ( s + 2)Cterm recurrence: ... ... ... ∗ H ��� − � �� ( s + 2) ����� ��������� . ... ... ... . . ... ... ����� ������ ���������� � ������������ ∗ ... ∗ ∗ ��������� (L. and Strakoˇ s, 2007) Given A , B as above and a nonnegative integer s with s + 2 ≤ d ��� ( A ). ( d ��� ( A ) = degree of A ’s minimal polynomial.) Then A ������ ��� B �� ������� ( s + 2) ����� ���������� , if • for any v � the matrix H ��� − � is at most ( s + 2)Cband Hessenberg, and • for at least one v � the matrix H ��� − � is ( s + 2)Cband Hessenberg.
Recommend
More recommend