CS475 / CM375 Lecture 14: Oct 27, 2011 Eigenvalue problems Reading: - PDF document

26/10/2011 CS475 / CM375 Lecture 14: Oct 27, 2011 Eigenvalue problems Reading: [TB] Chapters 24, 25 CS475/CM375 (c) 2011 P. Poupart & J. Wan 1 Eigenvalue problem 1 0 0 • Example: � � 2 �1 2 4 �4 5 det �� � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 2 1

26/10/2011 Example continued • � � 1: • � � 3: CS475/CM375 (c) 2011 P. Poupart & J. Wan 3 Example Continued • Thus �� � �Λ � • Note: we never compute eigenvalues by finding the roots of the characteristic polynomial CS475/CM375 (c) 2011 P. Poupart & J. Wan 4 2

26/10/2011 Eigenvalues • Gershgorin Theorem: Let � be any square matrix. The eigenvalues � of � are located in the union of the � disks: � � � �� � ∑ � �� ��� • Proof: Consider ��, �� such that �� � �� , � � 0 . Scale � such that � � � 1 � � � for some � � Then �� � � �� � � ∑ � � �� � � � ∑ � �� � � � �� � � ��� ��� ⟹ � � � �� � ∑ � �� � � � ∑ � �� � � � ∑ |� �� | ��� ��� ��� CS475/CM375 (c) 2011 P. Poupart & J. Wan 5 Example 4 �0.5 0 • � � 0.6 5 �0.6 0 0.5 3 • Picture: CS475/CM375 (c) 2011 P. Poupart & J. Wan 6 3

26/10/2011 Rayleigh quotient • Assume � is real and symmetric. Thus � has real eigenvalues and a complete set of orthogonal eigenvectors � � , … , � � , �� � , … , � � � � � � 1 • Def: The Rayleigh quotient of a vector � is: � � � � � �� � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 7 Rayleigh quotient • Notes If � is an eigenvector, then ���� is an eigenvalue 1. Given � , find � such that 2. � � ⋮ min � � � �� ( � � 1 least squares) � � � The normal equations: � � � � � � � �� � � ���� 3. Theorem: Let � � be an eigenvector and � � � � � then � � � � � � � � � � � � as � → � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 8 4

26/10/2011 Power Iteration • Let � � � approximate eigenvector, � � � 1 and � � � set of eigenvectors • Then � � � � � � � � � � � � � ⋯ � � � � � ⟹ �� � � � � � � � � � � � � � � � � ⋯ � � � � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 9 Power Iteration • Similarly, � � � � � � � � � � � � � � � � � � � � � ⋯ � � � � � � � � � � � � � � � � � � � � � � � � � � � � ⋯ � � � � � � � � � • Suppose � � � � � � ⋯ � |� � | . � � � → 0 as � → ∞ Then � � � � � � ~ � � � � � � � for large � � � � � i.e., � � ~ � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 10 5

26/10/2011 Example 21 7 �1 1 � ��� � • � � 5 7 7 1 4 �4 20 1 CS475/CM375 (c) 2011 P. Poupart & J. Wan 11 Power Iteration Algorithm � � � initial guess, � � � 1 for � � 1,2, … � � �� ��� � � � � � � �� � � � � � � Rayleigh quotient end CS475/CM375 (c) 2011 P. Poupart & J. Wan 12 6

26/10/2011 Power Iteration Algorithm • Notes We normalize �� ��� in each computation of � � 1. Theorem: Suppose � � � � � � � � � ⋯ � � � 2. � � � � 0 . Then and � � � � � � �� � � � � � and � � �� � � � � � � � � � as � → ∞ � � It only computes � � 3. � � The convergence is linear, the convergence rate � 4. � � The convergence can be slow if � � � � � 5. CS475/CM375 (c) 2011 P. Poupart & J. Wan 13 Inverse Iteration • Idea 1: Use � �� to compute the smallest eigenvalue Note: Λ � �� � � � � � � , � � , … , � � Thus: � � � � � � � � � � � � � ⋯ � � � � � • � �� � � � � � � � � � � � � ⋯ � � � � � � � ⋮ � � � �� � � � � � � � � � � ⋯ � � � � � � � � � � � � � � � � � � � � � � � ⋯ � � ��� � ��� � � � � � � � � � � ��� � ∴ � �� � � � � � � � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 14 7

26/10/2011 Inverse Iteration • Idea 2: Shifting. Consider � � � � �� � ∉ Λ��� Then � has the same eigenvectors as � and its eigenvalues are �� � � �� , where � � ∈ Λ��� . • If � is close to � � , then � � � � would be the smallest eigenvalue of � . • We can apply idea 1 to compute � � � � CS475/CM375 (c) 2011 P. Poupart & J. Wan 15 Example 21 7 �1 • � � 5 7 7 Λ � � 8,16,24 , � � 15 4 �4 20 CS475/CM375 (c) 2011 P. Poupart & J. Wan 16 8