Iterative Methods for Symmetric Quasi-Definite Linear Systems Mario Arioli 1 Dominique Orban 2 1 Rutherford Appleton Laboratory, mario.arioli@stfc.ac.uk 2 GERAD and ´ Ecole Polytechnique de Montr´ eal, dominique.orban@gerad.ca
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Overview of talk ◮ Symmetric Quasi Positive Definite matrices 2 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Overview of talk ◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? 2 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Overview of talk ◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? ◮ Main properties 2 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Overview of talk ◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? ◮ Main properties ◮ Generalized singular values and minimization problem ◮ G-K bidiagonalization ◮ Generalized LSQR and Craig (Stopping criteria) 2 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Overview of talk ◮ Symmetric Quasi Positive Definite matrices ◮ Why SQD are important? ◮ Main properties ◮ Generalized singular values and minimization problem ◮ G-K bidiagonalization ◮ Generalized LSQR and Craig (Stopping criteria) ◮ Numerical examples 2 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Symmetric Quasi-Definite Systems � M � � x � � f � A M = M T ≻ 0 , N = N T ≻ 0 . = where A T − N y g ◮ Interior-point methods for LP, QP, NLP, SOCP, SDP, . . . ◮ Regularized/stabilized PDE problems ◮ Regularized least squares ◮ How to best take advantage of the structure? 3 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Main Property Theorem (Vanderbei, 1995) If K is SQD, it is strongly factorizable , i.e., for any permutation matrix P , there exists a unit lower triangular L and a diagonal D such that P T KP = LDL T . ◮ Cholesky-factorizable ◮ Used to speed up factorization in regularized least-squares (Saunders) and interior-point methods (Friedlander and O.) ◮ Stability analysis by Gill, Saunders, Shinnerl (1996). 4 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Centered preconditioning � � M � � � � � ˆ � � � M − 1 M − 1 M − 1 2 f A x 2 2 = N − 1 N − 1 N − 1 A T − N y ˆ 2 g 2 2 which is equivalent to A � �� � � � � ˆ � � M − 1 2 AN − 1 � M − 1 2 f I m x 2 = N − 1 2 A T M − 1 N − 1 ˆ y 2 g − I n 2 5 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Centered preconditioning � � M � � � � � ˆ � � � M − 1 M − 1 M − 1 2 f A x 2 2 = N − 1 N − 1 N − 1 A T − N y ˆ 2 g 2 2 which is equivalent to A � �� � � � � ˆ � � M − 1 2 AN − 1 � M − 1 2 f I m x 2 = N − 1 2 A T M − 1 N − 1 ˆ y 2 g − I n 2 Theorem (Saunders (1995)) A = M − 1 2 AN − 1 Suppose ˜ 2 has rank p ≤ m with nonzero singular values σ 1 , . . . , σ p . The eigenvalues of A are +1 , − 1 and ±√ 1 + σ k , k = 1 , . . . , p. 5 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Symmetric spectrum and Iterative methods A symmetric matrix with a symmetric spectrum can be transform preserving the symmetry of the spectrum in a SQD one. Moreover, Fischer (Theorem 6.9.9 in “Polynomial based iteration methods for symmetric linear systems”) Freund (1983), Freund Golub Nachtigal (1992), and Ramage Silvester Wathen (1995) give different poofs that MINRES and CG perform redundant iterations. 6 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Iterative Methods I Facts: SQD systems are symmetric, non-singular, square and indefinite. 7 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Iterative Methods I Facts: SQD systems are symmetric, non-singular, square and indefinite. ◮ MINRES ◮ SYMMLQ ◮ (F)GMRES?? ◮ QMRS???? 7 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Iterative Methods I Facts: SQD systems are symmetric, non-singular, square and indefinite. ◮ MINRES ◮ SYMMLQ ◮ (F)GMRES?? ◮ QMRS???? Fact: . . . none exploits the SQD structure and they are doing redundant iterations 7 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Related Problems: an example � M � � x � � b � A = A T − N y 0 8 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Related Problems: an example � M � � x � � b � A = A T − N y 0 are the optimality conditions of � ��� � � �� A 2 � � A � � b �� 2 M − 1 � � b � � 0 � � 2 1 1 � � min y − ≡ min y − � � � 1 � 2 2 I 0 I 0 m � � m 0 N y ∈ I R y ∈ I R E − 1 2 � � 2 + 8 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Normal equations for SQD Let assume that M = R T R N = U T U and 9 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Normal equations for SQD Let assume that M = R T R N = U T U and We observe that the normal equations � ˆ � � R − 1 b � x A 2 = A y ˆ 0 have a very interesting structure because 9 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Normal equations for SQD Let assume that M = R T R N = U T U and We observe that the normal equations � ˆ � � R − 1 b � x A 2 = A ˆ y 0 have a very interesting structure because � I m − n + ˜ � A ˜ A T 0 A 2 = = D A T ˜ I n + ˜ 0 A and DA = AD 9 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Normal equations for SQD Let assume that M = R T R N = U T U and � I m − n + ˜ � A ˜ A T 0 A 2 = = D A T ˜ I n + ˜ 0 A and DA = AD The Krylov space � � A j h 0 , . . . , A h 0 , h 0 K k ( A , h 0 ) = Range is K k ( A , h 0 ) = K ⌊ k / 2 ⌋ ( D , h 0 ) + K ⌊ ( k +1) / 2 ⌋ ( D , A h 0 ) 9 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban A− 1 A − 1 = D − 1 A = AD − 1 10 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Linear operators R m × m and N ∈ I R n × n be symmetric positive definite Let M ∈ I R m × n be a full rank matrix. matrices, and let A ∈ I R m ; � u � 2 M = v T Mv } , N = { q ∈ I R n ; � q � 2 N = q T Nq } M = { v ∈ I M ′ = { w ∈ I M − 1 = w T M − 1 w } , R m ; � w � 2 N ′ = { y ∈ I R n ; � y � 2 N − 1 = y T N − 1 y } 11 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Linear operators R m × m and N ∈ I R n × n be symmetric positive definite Let M ∈ I R m × n be a full rank matrix. matrices, and let A ∈ I R m ; � u � 2 R n ; � q � 2 M = v T Mv } , N = { q ∈ I N = q T Nq } M = { v ∈ I M ′ = { w ∈ I R m ; � w � 2 M − 1 = w T M − 1 w } , N ′ = { y ∈ I R n ; � y � 2 N − 1 = y T N − 1 y } Aq ∈ M ′ ∀ q ∈ N . � v , Aq � M , M ′ = v T Aq , 11 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Linear operators R m × m and N ∈ I R n × n be symmetric positive definite Let M ∈ I R m × n be a full rank matrix. matrices, and let A ∈ I R m ; � u � 2 M = v T Mv } , N = { q ∈ I R n ; � q � 2 N = q T Nq } M = { v ∈ I M ′ = { w ∈ I R m ; � w � 2 M − 1 = w T M − 1 w } , N ′ = { y ∈ I R n ; � y � 2 N − 1 = y T N − 1 y } Aq ∈ M ′ ∀ q ∈ N . � v , Aq � M , M ′ = v T Aq , The adjoint operator A ⋆ of A can be defined as A T g ∈ N ′ ∀ g ∈ M . � A ⋆ g , f � N ′ , N = f T A T g , 11 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Generalized SVD Given q ∈ M and v ∈ N , the critical points for the functional v T Aq � q � N � v � M are the “ elliptic singular values and singular vectors’ ’ of A . 12 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Generalized SVD Given q ∈ M and v ∈ N , the critical points for the functional v T Aq � q � N � v � M are the “ elliptic singular values and singular vectors’ ’ of A . The saddle-point conditions are � Aq i v T = σ i Mv i i Mv j = δ ij A T v i q T = σ i Nq i i Nq j = δ ij σ 1 ≥ σ 2 ≥ · · · ≥ σ n > 0 12 / 28
Sparse Days, Toulouse, 2012 Mario Arioli, Dominique Orban Generalized SVD Given q ∈ M and v ∈ N , the critical points for the functional v T Aq � q � N � v � M are the “ elliptic singular values and singular vectors’ ’ of A . The saddle-point conditions are � Aq i v T = σ i Mv i i Mv j = δ ij A T v i q T = σ i Nq i i Nq j = δ ij σ 1 ≥ σ 2 ≥ · · · ≥ σ n > 0 The elliptic singular values are the standard singular values of ˜ A = M − 1 / 2 AN − 1 / 2 . The elliptic singular vectors q i and v i , i = 1 , . . . , n are the transformation by M − 1 / 2 and N − 1 / 2 respectively of the left and right standard singular vector of ˜ A . 12 / 28
Recommend
More recommend
