Solving large scale eigenvalue problems Lecture 5, March 23, 2016: - PowerPoint PPT Presentation

Solving large scale eigenvalue problems Solving large scale eigenvalue problems Lecture 5, March 23, 2016: The QR algorithm II http://people.inf.ethz.ch/arbenz/ewp/ Peter Arbenz Computer Science Department, ETH Z¨ urich E-mail: arbenz@inf.ethz.ch Large scale eigenvalue problems, Lecture 5, March 23, 2016 1/30

Solving large scale eigenvalue problems Survey Survey of today’s lecture The QR algorithm is the most important algorithm to compute the Schur form of a dense matrix. ◮ Basic QR algorithm ◮ Hessenberg QR algorithm ◮ QR algorithm with shifts ◮ Double step QR algorithm for real matrices ◮ The symmetric QR algorithm ◮ The QZ algorithm for solving A x = λ B x . Large scale eigenvalue problems, Lecture 5, March 23, 2016 2/30

Solving large scale eigenvalue problems Spectral decomposition Spectral decomposition Theorem Let A ∈ C n × n be hermitian, A ∗ = A. Then there is a unitary matrix U ∈ C n × n such that U ∗ AU = Λ = diag ( λ 1 , . . . , λ n ) (1) is diagonal. The diagonal elements λ i of Λ are the eigenvalues of A. Let U = [ u 1 , u 2 , . . . , u n ]. Then A u i = λ i u i , 1 ≤ i ≤ n . u i is the eigenvector associated with the eigenvalue λ i . Large scale eigenvalue problems, Lecture 5, March 23, 2016 3/30

Solving large scale eigenvalue problems The symmetric QR algorithm The symmetric QR algorithm ◮ The QR algorithm can be applied straight to Hermitian or symmetric matrices. ◮ The QR algorithm generates a sequence { A k } of symmetric matrices. ◮ Taking into account the symmetry, the performance of the algorithm can be improved considerably. ◮ Hermitian matrices have a real spectrum. Therefore, we can restrict ourselves to single shifts. Large scale eigenvalue problems, Lecture 5, March 23, 2016 4/30

Solving large scale eigenvalue problems The symmetric QR algorithm Reduction to tridiagonal form Apply a sequence of Householder transformations to arrive at tridiagonal (= symmetric Hessenberg) form. First step: Let � 1 � 0 T u 1 ∈ C n , P 1 = , � u 1 � = 1 . 0 I n − 1 − 2 u 1 u ∗ 1 Then, A 1 := P ∗ 1 AP 1 = ( I − 2 u 1 u ∗ 1 ) A ( I − 2 u 1 u ∗ 1 ) = A − u 1 (2 u ∗ 1 A − 2( u ∗ 1 A u 1 ) u ∗ ) − (2 A u 1 − 2 u 1 ( u ∗ u ∗ 1 A u 1 )) 1 1 � �� � � �� � v 1 v ∗ 1 = A − u 1 v ∗ 1 − v 1 u ∗ 1 . Large scale eigenvalue problems, Lecture 5, March 23, 2016 5/30

Solving large scale eigenvalue problems The symmetric QR algorithm Reduction to tridiagonal form (cont.) In the k -th step of the reduction we similarly have A k = P ∗ k A k − 1 P k = A k − 1 − u k − 1 v ∗ k − 1 − v k − 1 u ∗ k − 1 , where the last n − k elements of u k − 1 and v k − 1 are nonzero. Essential computation in the k th step: v k − 1 = 2 A k − 1 u k − 1 − 2 u k − 1 ( u ∗ k − 1 A k − 1 u k − 1 ) which costs 2( n − k ) 2 + O ( n − k ) flops. Altogether, the reduction to tridiagonal form costs n − 1 = 4 � � 4( n − k ) 2 + O ( n − k ) � 3 n 3 + O ( n 2 ) flops. k =1 Large scale eigenvalue problems, Lecture 5, March 23, 2016 6/30

Solving large scale eigenvalue problems The symmetric QR algorithm The explicit tridiagonal QR algorithm In the explicit form, a QR step is essentially 1: Choose a shift µ 2: Compute the QR factorization A − µ I = QR 3: Update A by A = RQ + µ I . Of course, this is done by means of plane rotations and by respecting the symmetric tridiagonal structure of A . Shifting strategies: Rayleigh quotient shifts, Wilkinson shifts Large scale eigenvalue problems, Lecture 5, March 23, 2016 7/30

Solving large scale eigenvalue problems The symmetric QR algorithm The implicit tridiagonal QR algorithm In the more elegant implicit form of the algorithm we first compute the first Givens rotation G 0 = G (1 , 2 , ϑ ) of the QR factorization that zeros the (2 , 1) element of A − µ I , � � � a 11 − µ � � ∗ � c s = , c = cos( ϑ 0 ) , s = sin( ϑ 0 ) . − s c a 21 0 (2) Performing a similarity transformation with G 0 we have ( n = 5)   × × + × × ×   0 AG 0 = A ′ =   G ∗ + × × ×     × × ×   × × Large scale eigenvalue problems, Lecture 5, March 23, 2016 8/30

Solving large scale eigenvalue problems The symmetric QR algorithm The implicit tridiagonal QR algorithm (cont.) Similarly as with the double step Hessenberg QR algorithm we chase the bulge down the diagonal.     × × + × × 0 × × × × × × +     G 0 G 1     A − − − − − − − − − − → + × × × − − − − − − − − − − → 0 × × ×         = G (1 , 2 , ϑ 0 ) = G (2 , 3 , ϑ 1 ) × × × + × × ×     × × × ×     × × 0 × × × × × × × ×     G 2 G 3     − − − − − − − − − − → × × × + − − − − − − − − − − → × × × 0         = G (3 , 4 , ϑ 2 ) = G (4 , 5 , ϑ 3 ) 0 × × × × × ×     + × × 0 × × Large scale eigenvalue problems, Lecture 5, March 23, 2016 9/30

Solving large scale eigenvalue problems The symmetric QR algorithm The implicit tridiagonal QR algorithm (cont.) The full step is given by A = Q ∗ AQ , Q = G 0 G 1 · · · G n − 2 . Because G k e 1 = e 1 for k > 0 we have Q e 1 = G 0 G 1 · · · G n − 2 e 1 = G 0 e 1 . Both explicit and implicit QR step form the same first plane rotation G 0 . By referring to the Implicit Q Theorem we see that explicit and implicit QR step compute essentially the same A . Large scale eigenvalue problems, Lecture 5, March 23, 2016 10/30

Solving large scale eigenvalue problems The symmetric QR algorithm Symm. tridiag. QR algo with Wilkinson shifts 1: Let T ∈ R n × n be a symmetric tridiagonal matrix with diagonal entries a 1 , . . . , a n and off-diagonal entries b 2 , . . . , b n . This algorithm computes the eigenvalues λ 1 , . . . , λ n of T and corresponding eigenvectors q 1 , . . . , q n . The eigenvalues are stored in a 1 , . . . , a n . The eigenvectors are stored in the matrix Q , such that TQ = Q diag( a 1 , . . . , a n ). 2: m = n { Actual problem dimension. m is reduced in the convergence check. } 3: while m > 1 do d := ( a m − 1 − a m ) / 2; { Compute Wilkinson’s shift } 4: if d = 0 then 5: s := a m − | b m | ; 6: Large scale eigenvalue problems, Lecture 5, March 23, 2016 11/30

Solving large scale eigenvalue problems The symmetric QR algorithm Symm. tridiag. QR algo with Wilkinson shifts (cont.) else 7: � d 2 + b 2 s := a m − b 2 m / ( d + sign( d ) m ); 8: end if 9: x := a (1) − s ; { Implicit QR step begins here } 10: y := b (2); 11: for k = 1 to m − 1 do 12: if m > 2 then 13: [ c , s ] := givens ( x , y ); 14: else 15: � � c � c � � a 1 � − s b 2 s Determine [ c , s ] such that 16: s c b 2 a 2 − s c is diagonal end if 17: w := cx − sy ; 18: Large scale eigenvalue problems, Lecture 5, March 23, 2016 12/30

Solving large scale eigenvalue problems The symmetric QR algorithm Symm. tridiag. QR algo with Wilkinson shifts (cont.) d := a k − a k +1 ; z := (2 cb k +1 + ds ) s ; 19: a k := a k − z ; a k +1 := a k +1 + z ; 20: b k +1 := dcs + ( c 2 − s 2 ) b k +1 ; 21: x := b k +1 ; 22: if k > 1 then 23: b k := w ; 24: end if 25: if k < m − 1 then 26: y := − sb k +2 ; b k +2 := cb k +2 ; 27: end if 28: � c � s Q 1: n ; k : k +1 := Q 1: n ; k : k +1 ; 29: − s c end for { Implicit QR step ends here } 30: if | b m | < ε ( | a m − 1 | + | a m | ) then { Check for convergence } 31: Large scale eigenvalue problems, Lecture 5, March 23, 2016 13/30

Solving large scale eigenvalue problems The symmetric QR algorithm Symm. tridiag. QR algo with Wilkinson shifts (cont.) m := m − 1; 32: end if 33: 34: end while Large scale eigenvalue problems, Lecture 5, March 23, 2016 14/30

Solving large scale eigenvalue problems The symmetric QR algorithm Remark on deflation   a 1 b 2 b 2 a 2 b 3     0 b 3 a 3   T =   0 a 4 b 5     b 5 a 5 b 6   b 6 a 6 ◮ Shift for next step is determined from second block. ◮ First plane rotation is determined from shift and first block! ◮ The implicit shift algorithm then chases the bulge down the diagonal. Procedure finishes already in row 4 because b 4 = 0. ◮ This shift does not improve convergence. ◮ Explicit QR algorithm converges rapidly, but first block is not treated properly Large scale eigenvalue problems, Lecture 5, March 23, 2016 15/30

Solving large scale eigenvalue problems The symmetric QR algorithm Complexity of QR algorithm nonsymmetric case symmetric case without with without with Schurvectors eigenvectors 10 3 n 3 14 3 n 3 3 n 3 4 3 n 3 8 transformation to Hessenberg/tridiagonal form 20 3 n 3 50 3 n 3 24 n 2 6 n 3 real double step Hessenberg/ tridiagonal QR algorithm (2 steps per eigenvalues) 4 10 n 3 25 n 3 3 n 3 9 n 3 total Large scale eigenvalue problems, Lecture 5, March 23, 2016 16/30