CS475 / CM375 Lecture 23: Nov 29, 2011 Convergence of Iterative - PDF document

29/11/2011 CS475 / CM375 Lecture 23: Nov 29, 2011 Convergence of Iterative Methods CS475/CM375 (c) 2011 P. Poupart & J. Wan 1 Richardson Convergence • The iteration matrix is given by: � ���� ≡ � � � ���� �� � � � � �� � � � � �� • Suppose ��, �� is an eigenpair of � . Then � ���� � � � � �� � � � � ��� � 1 � �� � • Hence � ≡ 1 � �� is an eigenvalue of � ���� CS475/CM375 (c) 2011 P. Poupart & J. Wan 2 1

29/11/2011 Richardson Convergence • Lemma: Let � ��� and � ��� be the smallest and largest eigenvalue of � . Then � � ���� � max � 1 � �� ��� , 1 � �� ��� � • Proof: CS475/CM375 (c) 2011 P. Poupart & J. Wan 3 Richardson Convergence • Notes: – If � ��� � 0 and � ��� � 0 , then either 1 � �� ��� � 1 �� � 0� or 1 � �� ��� � 1 �� � 0� ⟹ � � ���� � 1 ⟹ Richardson method diverges CS475/CM375 (c) 2011 P. Poupart & J. Wan 4 2

29/11/2011 Richardson Convergence • Theorem: Assume all eigenvalues of � are positive. Then Richardson converges if and only if 0 � � � 2/� ��� • Proof: if 0 � � � 2/� ��� , then CS475/CM375 (c) 2011 P. Poupart & J. Wan 5 Richardson Convergence • Proof continued: Assume � � ���� � 1 . Then CS475/CM375 (c) 2011 P. Poupart & J. Wan 6 3

29/11/2011 Richardson Convergence • � ��� : � 1 � �� ��� � 1 � �� ��� � ��� � 2/�� ��� � � ��� � � ��� �� ��� • � ��� � 1 � � ��� � ��� � � ��� �� ��� � ��� /� ��� �� � � �� � � ��� /� ��� �� � � � �� • Picture: CS475/CM375 (c) 2011 P. Poupart & J. Wan 7 Jacobi Convergence • Theorem: If � and 2� � � are SPD, then Jacobi converges �� � � � � � �� � • Proof: Let � be an eigenval. of � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 8 4

29/11/2011 Jacobi Convergence • Proof continued: Since 2� � � is SPD, CS475/CM375 (c) 2011 P. Poupart & J. Wan 9 Gauss ‐ Seidel & SOR Convergence • Theorem: If � is SPD, then GS & SOR �0 � � � 2� converge. • Definition: � is an M ‐ matrix if � �� � 0 i. � �� � 0 ii. � �� exists and � �� �� � 0 ∀ �� iii. Theorem: If � is an M ‐ matrix, Jacobi and GS • converge. Moreover �� � � � � � � �� � � 1 � � � � �� � i.e. the convergence rate of GS is better than that of Jacobi CS475/CM375 (c) 2011 P. Poupart & J. Wan 10 5

29/11/2011 Example: Poisson Equation • Recall: ��� �� � � �� � �� • Theorem: Let � be the 2D Laplacian matrix. The eigenvalues of � are given by: � ��� ��� � � sin � � sin � � �� � 1 � �, � � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 11 Example: Poisson Equation • The smallest eigenvalue is attained for � � � � 1 � �� � � sin � � ��� � � • The largest eigenvalue is attained for � � � � � � ��� � � sin � � ��� � � � � � � � sin � � � 1 � � �� � ��� , �� � 1 � �� � �� � � cos � � � • � is SPD and an M ‐ matrix CS475/CM375 (c) 2011 P. Poupart & J. Wan 12 6

29/11/2011 Richardson • � � � �� � � �� � �� � � sin � � � cos � max 1 � � , 1 � � � � � � • Convergence holds for 0 � � � and � ��� � �� � � � � � ��� � � ��� �� ��� � � � ��� �� ��� �� � ��� �� ��� � 1 � 2 sin � � ��� � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 13 Jacobi �� � � optimal Richardson � • � � � � � � � ��� ∴ � � � � �� � � � ��� � 1 � 2 sin � �� � cos ���� 2 � � � � • By Taylor expansion: cos � � 1 � � � �! � ⋯ ∴ � � � � �� � � cos �� � 1 � � � 2 � � � ��� � � • For small mesh size � , � � � � 1 → slow convergence CS475/CM375 (c) 2011 P. Poupart & J. Wan 14 7

29/11/2011 Gauss Seidel (GS) � �� � � � � � � � �� � • � � � � �� � cos � ���� � 1 � sin � �� � 1 � � � � � � ��� � � • For small � , slow convergence for GS • Convergence rate � 2 � convergence rate of Jacobi CS475/CM375 (c) 2011 P. Poupart & J. Wan 15 Successive Over Relaxation (SOR) • For SOR: � � ��� � ����� �� ��� � � ��� � 1 � ��� ����� �� � ����� �� � 1 � 2�� � ��� � � • Optimal SOR is an order of magnitude better than GS and Jacobi CS475/CM375 (c) 2011 P. Poupart & J. Wan 16 8