what s so great about krylov subspaces
play

Whats so great about Krylov subspaces? David S. Watkins Department - PowerPoint PPT Presentation

Krylov subspaces David S. Watkins Whats so great about Krylov subspaces? David S. Watkins Department of Mathematics Washington State University Guwahati, 2013 Krylov Subspace Methods are Great Krylov subspaces David S. Watkins


  1. Need for iteration Krylov subspaces David S. Watkins Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

  2. Need for iteration Krylov subspaces David S. Watkins Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

  3. Need for iteration Krylov subspaces David S. Watkins Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

  4. Need for iteration Krylov subspaces David S. Watkins Hessenberg form is not the goal. Triangular form is the goal. Hessenberg is as far as we can get with a direct method. (Abel, Galois) need to iterate

  5. Power method, Subspace iteration Krylov subspaces David S. Watkins v , Av , A 2 v , A 3 v , . . . convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , . . . subspaces of dimension j ( | λ j +1 /λ j | )

  6. Power method, Subspace iteration Krylov subspaces David S. Watkins v , Av , A 2 v , A 3 v , . . . convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , . . . subspaces of dimension j ( | λ j +1 /λ j | )

  7. Power method, Subspace iteration Krylov subspaces David S. Watkins v , Av , A 2 v , A 3 v , . . . convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , . . . subspaces of dimension j ( | λ j +1 /λ j | )

  8. Power method, Subspace iteration Krylov subspaces David S. Watkins v , Av , A 2 v , A 3 v , . . . convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , . . . subspaces of dimension j ( | λ j +1 /λ j | )

  9. Power method, Subspace iteration Krylov subspaces David S. Watkins v , Av , A 2 v , A 3 v , . . . convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , . . . subspaces of dimension j ( | λ j +1 /λ j | )

  10. Power method, Subspace iteration Krylov subspaces David S. Watkins v , Av , A 2 v , A 3 v , . . . convergence rate | λ 2 /λ 1 | S , A S , A 2 S , A 3 S , . . . subspaces of dimension j ( | λ j +1 /λ j | )

  11. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  12. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  13. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  14. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  15. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  16. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  17. Power method, Subspace iteration Krylov subspaces David S. Watkins for greater flexibility shifts ρ 1 , . . . , ρ m ( m small) (explanation deferred) p ( A ) = ( A − ρ 1 I ) · · · ( A − ρ m I ) Substitute p ( A ) for A S , p ( A ) S , p ( A ) 2 S , p ( A ) 3 S , . . . convergence rate | p ( λ j +1 ) / p ( λ j ) |

  18. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  19. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  20. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  21. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  22. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  23. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  24. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  25. Subspace Iteration with changes of coordinate system Krylov subspaces David S. take S = span { e 1 , . . . , e j } Watkins p ( A ) S = span { p ( A ) e 1 , . . . , p ( A ) e j } = span { q 1 , . . . , q j } (orthonormal?) build unitary (or not) Q = [ q 1 · · · q j · · · ] change coordinate system: ˆ A = Q − 1 AQ q k → Q − 1 q k = e k span { q 1 , . . . , q j } → span { e 1 , . . . , e j } ready for next iteration

  26. Subspace Iteration with changes of coordinate system Krylov subspaces David S. This version of subspace iteration . . . Watkins . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span { e 1 , . . . , e j } is invariant. � A 11 � A 12 A → ( A 11 is j × j .) 0 A 22

  27. Subspace Iteration with changes of coordinate system Krylov subspaces David S. This version of subspace iteration . . . Watkins . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span { e 1 , . . . , e j } is invariant. � A 11 � A 12 A → ( A 11 is j × j .) 0 A 22

  28. Subspace Iteration with changes of coordinate system Krylov subspaces David S. This version of subspace iteration . . . Watkins . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span { e 1 , . . . , e j } is invariant. � A 11 � A 12 A → ( A 11 is j × j .) 0 A 22

  29. Subspace Iteration with changes of coordinate system Krylov subspaces David S. This version of subspace iteration . . . Watkins . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span { e 1 , . . . , e j } is invariant. � A 11 � A 12 A → ( A 11 is j × j .) 0 A 22

  30. Subspace Iteration with changes of coordinate system Krylov subspaces David S. This version of subspace iteration . . . Watkins . . . holds the subspace fixed while the matrix changes. . . . moving toward a matrix under which span { e 1 , . . . , e j } is invariant. � A 11 � A 12 A → ( A 11 is j × j .) 0 A 22

  31. Subspace iteration and Krylov subspaces Krylov subspaces David S. single vector q “determines” nested sequence Watkins � q , Aq , A 2 q , . . . , A j − 1 q � K j ( A , q ) = span , j = 1 , . . . , n step of power method: q → p ( A ) q . . . implies a nested sequence of subspace iterations because . . . p ( A ) K j ( A , q ) = K j ( A , p ( A ) q ), since p ( A ) A = Ap ( A )

  32. Subspace iteration and Krylov subspaces Krylov subspaces David S. single vector q “determines” nested sequence Watkins � q , Aq , A 2 q , . . . , A j − 1 q � K j ( A , q ) = span , j = 1 , . . . , n step of power method: q → p ( A ) q . . . implies a nested sequence of subspace iterations because . . . p ( A ) K j ( A , q ) = K j ( A , p ( A ) q ), since p ( A ) A = Ap ( A )

  33. Subspace iteration and Krylov subspaces Krylov subspaces David S. single vector q “determines” nested sequence Watkins � q , Aq , A 2 q , . . . , A j − 1 q � K j ( A , q ) = span , j = 1 , . . . , n step of power method: q → p ( A ) q . . . implies a nested sequence of subspace iterations because . . . p ( A ) K j ( A , q ) = K j ( A , p ( A ) q ), since p ( A ) A = Ap ( A )

  34. Subspace iteration and Krylov subspaces Krylov subspaces David S. single vector q “determines” nested sequence Watkins � q , Aq , A 2 q , . . . , A j − 1 q � K j ( A , q ) = span , j = 1 , . . . , n step of power method: q → p ( A ) q . . . implies a nested sequence of subspace iterations because . . . p ( A ) K j ( A , q ) = K j ( A , p ( A ) q ), since p ( A ) A = Ap ( A )

  35. Subspace iteration and Krylov subspaces Krylov subspaces David S. single vector q “determines” nested sequence Watkins � q , Aq , A 2 q , . . . , A j − 1 q � K j ( A , q ) = span , j = 1 , . . . , n step of power method: q → p ( A ) q . . . implies a nested sequence of subspace iterations because . . . p ( A ) K j ( A , q ) = K j ( A , p ( A ) q ), since p ( A ) A = Ap ( A )

  36. Let’s build an algorithm Krylov subspaces David S. start with upper Hessenberg A Watkins pick shifts ρ 1 , ρ 2 ( m = 2 for illustration)  x  y     z   q 1 = α p ( A ) e 1 = α ( A − ρ 1 I )( A − ρ 2 I ) e 1 =   0    .  .   .   0 cheap, don’t compute p ( A ). � � Q 0 = q 1 · · · q n , built from q 1

  37. Let’s build an algorithm Krylov subspaces David S. start with upper Hessenberg A Watkins pick shifts ρ 1 , ρ 2 ( m = 2 for illustration)  x  y     z   q 1 = α p ( A ) e 1 = α ( A − ρ 1 I )( A − ρ 2 I ) e 1 =   0    .  .   .   0 cheap, don’t compute p ( A ). � � Q 0 = q 1 · · · q n , built from q 1

  38. Let’s build an algorithm Krylov subspaces David S. start with upper Hessenberg A Watkins pick shifts ρ 1 , ρ 2 ( m = 2 for illustration)  x  y     z   q 1 = α p ( A ) e 1 = α ( A − ρ 1 I )( A − ρ 2 I ) e 1 =   0    .  .   .   0 cheap, don’t compute p ( A ). � � Q 0 = q 1 · · · q n , built from q 1

  39. Let’s build an algorithm Krylov subspaces David S. start with upper Hessenberg A Watkins pick shifts ρ 1 , ρ 2 ( m = 2 for illustration)  x  y     z   q 1 = α p ( A ) e 1 = α ( A − ρ 1 I )( A − ρ 2 I ) e 1 =   0    .  .   .   0 cheap, don’t compute p ( A ). � � Q 0 = q 1 · · · q n , built from q 1

  40. Let’s build an algorithm Krylov subspaces David S. start with upper Hessenberg A Watkins pick shifts ρ 1 , ρ 2 ( m = 2 for illustration)  x  y     z   q 1 = α p ( A ) e 1 = α ( A − ρ 1 I )( A − ρ 2 I ) e 1 =   0    .  .   .   0 cheap, don’t compute p ( A ). � � Q 0 = q 1 · · · q n , built from q 1

  41. Let’s build an algorithm Krylov subspaces David S. start with upper Hessenberg A Watkins pick shifts ρ 1 , ρ 2 ( m = 2 for illustration)  x  y     z   q 1 = α p ( A ) e 1 = α ( A − ρ 1 I )( A − ρ 2 I ) e 1 =   0    .  .   .   0 cheap, don’t compute p ( A ). � � Q 0 = q 1 · · · q n , built from q 1

  42. Let’s build an algorithm Krylov subspaces David S. Watkins ˜ A = Q − 1 0 AQ 0 power method + change of coordinate system now return to Hessenberg form A = ˜ ˆ Q − 1 ˜ A ˜ Q = Q − 1 AQ iteration complete! Now repeat!

  43. Let’s build an algorithm Krylov subspaces David S. Watkins ˜ A = Q − 1 0 AQ 0 power method + change of coordinate system now return to Hessenberg form A = ˜ ˆ Q − 1 ˜ A ˜ Q = Q − 1 AQ iteration complete! Now repeat!

  44. Let’s build an algorithm Krylov subspaces David S. Watkins ˜ A = Q − 1 0 AQ 0 power method + change of coordinate system now return to Hessenberg form A = ˜ ˆ Q − 1 ˜ A ˜ Q = Q − 1 AQ iteration complete! Now repeat!

  45. Let’s build an algorithm Krylov subspaces David S. Watkins ˜ A = Q − 1 0 AQ 0 power method + change of coordinate system now return to Hessenberg form A = ˜ ˆ Q − 1 ˜ A ˜ Q = Q − 1 AQ iteration complete! Now repeat!

  46. Let’s build an algorithm Krylov subspaces David S. Watkins ˜ A = Q − 1 0 AQ 0 power method + change of coordinate system now return to Hessenberg form A = ˜ ˆ Q − 1 ˜ A ˜ Q = Q − 1 AQ iteration complete! Now repeat!

  47. Let’s build an algorithm Krylov subspaces David S. Watkins ˜ A = Q − 1 0 AQ 0 power method + change of coordinate system now return to Hessenberg form A = ˜ ˆ Q − 1 ˜ A ˜ Q = Q − 1 AQ iteration complete! Now repeat!

  48. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  49. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  50. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  51. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  52. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  53. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  54. What does the algorithm do? Krylov ˆ A = Q − 1 AQ subspaces David S. q 1 = α p ( A ) e 1 Watkins ˆ A Hessenberg, so span { q 1 , . . . , q j } = K j ( A , q 1 ) = p ( A ) K j ( A , e 1 ) = p ( A ) span { e 1 , . . . , e j } subspace iteration + change of coordinate system: span { q 1 , . . . , q j } → span { e 1 , . . . , e j } j = 1, 2, . . . , n − 1

  55. What does the algorithm do? Krylov subspaces David S. Watkins Convergence rates: | p ( λ j +1 ) / p ( λ j ) | j = 1 , . . . , n − 1 All ratios matter.

  56. What does the algorithm do? Krylov subspaces David S. Watkins Convergence rates: | p ( λ j +1 ) / p ( λ j ) | j = 1 , . . . , n − 1 All ratios matter.

  57. Implementation Krylov subspaces David S. Watkins   x y     z   q 1 = α p ( A ) e 1 =   0    .  .   .   0 (case m = 2)

  58. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × ×     × × × ×     × × ×   × ×

  59. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × × ×     × × × ×     × × ×   × ×

  60. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × × ×     × × × × × ×     × × ×   × ×

  61. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × ×     × × × × ×     × × × × ×   × ×

  62. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × ×     × × × ×     × × × ×   × × × ×

  63. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × ×     × × × ×     × × ×   × × ×

  64. Implementation Krylov subspaces David S. Watkins  × × × × × ×  × × × × × ×     × × × × ×     × × × ×     × × ×   × ×

  65. Implementation Krylov subspaces David S. Watkins O ( n 2 ) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

  66. Implementation Krylov subspaces David S. Watkins O ( n 2 ) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

  67. Implementation Krylov subspaces David S. Watkins O ( n 2 ) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

  68. Implementation Krylov subspaces David S. Watkins O ( n 2 ) work per iteration (cheap!) This is nothing new. John Francis’s algorithm aka implicitly-shifted QR algorithm

Recommend


More recommend