Algebraic Eigenvalue Problem Algebraic Eigenvalue Problem Computers are useless. They can only give answers. Pablo Picasso 1 Fall 2010
Topics to Be Discussed Topics to Be Discussed � This unit requires the knowledge of eigenvalues This unit requires the knowledge of eigenvalues and eigenvectors in linear algebra. � The following topics will be presented: � The following topics will be presented: � The Power method for finding the largest eigenvalue and its corresponding eigenvector i l d it di i t � Coordinate rotation � Rotating a symmetric matrix � Classic Jacobi method (1846) for finding all � Classic Jacobi method (1846) for finding all eigenvalues and eigenvectors of a symmetric matrix 2
Eigenvalues & Eigenvectors: 1/3 Eigenvalues & Eigenvectors: 1/3 � Given a square matrix A, if one can find a Given a square matrix A, if one can find a number (real or complex) λ and a vector x such that A ⋅ x = λ x holds, λ is an eigenvalue and x an that A x λ x holds, λ is an eigenvalue and x an eigenvector corresponding to λ (of matrix A). � Since the right-hand side of A ⋅ x = λ x can be � Since the right-hand side of A ⋅ x = λ x can be rewritten as λ I ⋅ x, where I is the identity matrix, we have A ⋅ x = λ I ⋅ x and (A- λ I)x = 0 we have A ⋅ x = λ I ⋅ x and (A- λ I)x = 0. � Solving for λ from equation det(A- λ I) = 0 yields all eigenvalues of A, where det() is the ll i l f A h d t() i th determinant of a matrix. 3
Eigenvalues & Eigenvectors: 2/3 Eigenvalues & Eigenvectors: 2/3 � If A is a n × n matrix, det(A- λ I) = 0 is a If A is a n × n matrix, det(A λ I) 0 is a polynomial of degree n in λ , and has n roots ( i.e ., n possible values for λ ), some of which may be n possible values for λ ), some of which may be complex conjugates ( i.e ., a + b i and a - b i). � However people rarely use this method to find � However, people rarely use this method to find eigenvalues because (1) directly expanding det(A- λ I) = 0 to a polynomial is tedious, and (2) det(A- λ I) = 0 to a polynomial is tedious and (2) there is no close-form solution if n > 4. � M � Many methods transform A to simpler forms so th d t f A t i l f that det(A- λ I) = 0 can be obtained easily. 4
Eigenvalues & Eigenvectors: 3/3 Eigenvalues & Eigenvectors: 3/3 � The eigenvalues of a diagonal matrix are its g g diagonal entries. � For example, if we have a diagonal matrix: p , g ⎡ ⎤ d 1 ⎢ ⎥ d ⎢ ⎢ ⎥ ⎥ 2 = ⎢ ⎥ � A ⎢ ⎥ d n ⎢ ⎥ − 1 ⎢ ⎢ ⎥ ⎥ ⎣ ⎦ d n � Then, det(A- λ I)=0 is − λ λ − λ λ − λ λ − λ λ = ( )( ) … ( )( ) 0 d d d d − 1 2 1 n n � Hence, the roots of det(A- λ I)=0 are the d i ’s. 5
Pow er Method: 1/11 Pow er Method: 1/11 � What if we take a guess z and compute A ⋅ z? g p � If z is actually an eigenvector, then A ⋅ z = λ z. � Let w = A ⋅ z = λ z Since for every entry of w λ z. Since for every entry of w � Let w A z and z we have w i = λ z i and λ = w i / z i . � If z is not an eigenvector, then w may be a � If z is not an eigenvector, then w may be a vector closer to an eigenvector than z is. � Therefore, we may use w in the next iteration to � Therefore, we may use w in the next iteration to find an even better approximation. � From w, we have u = A ⋅ w; from u we have v = � From w, we have u A w; from u we have v A ⋅ u; etc. Hopefully, some vector x will satisfy A ⋅ x = λ x. 6
Pow er Method: 2/11 Pow er Method: 2/11 � Note that: if x is an eigenvector, α x is also an g , eigenvector because α (A ⋅ x)= α ( λ x) and A ⋅ ( α x) = λ ( α x)! � Therefore, we may scale an eigenvector. The simplest way is to scale the vector by the component with maximum absolute value. After scaling, the value of each component is in [-1,1]. i [ 1 1] � Example: Example: Let x be [15, -20, -8]. Since |-20| is th l the largest, the scaling factor is -20 and the t th li f t i 20 d th scaled x is [-15/20, 1, 8/20]. 7
Pow er Method: 3/11 Pow er Method: 3/11 � This scaling has an advantage. This scaling has an advantage. � Given a vector z, we compute w = A ⋅ z. � If w is a good approximate of λ z, we have w ≈ λ z f λ λ � If i d i t h = A ⋅ z. � Therefore, we should have w i ≈ λ z i for every i . � If vector z is scaled so that its largest entry, say z k , is 1, then w k ≈ λ z k = λ ! � In other words, the scaling factor is an In other words, the scaling factor is an approximation of an eigenvalue! 8
Pow er Method: 4/11 Pow er Method: 4/11 � We may start with a z and compute w=A ⋅ z. We may start with a z and compute w A z. � The largest component w k of w is an approximation of an eigenvalue λ ( i e w ≈ λ ) approximation of an eigenvalue λ ( i.e ., w k ≈ λ ). � Then, w is scaled with its largest component w k and used as a new z ( i.e ., z = w/ w k ). d d ( i / ) � This process is applied iteratively until we have |A ⋅ z – w k z| < ε , where ε is a tolerance value. 9
Pow er Method: 5/11 Pow er Method: 5/11 � Suppose this process starts with vector x 0 . pp p 0 � The computation of x i is x i = w i / w i,k = (A ⋅ x i -1 )/ w i,k , where w i,k is the maximum component of w i . � Since x i -1 = w i -1 / w i- 1 ,k = (A ⋅ x i -2 )/ w i- 1 ,k , we may rewrite the x i as follows for some c, d and g : x i = c (A ⋅ x i -1 ) = c ( d A(Ax i -2 )) = g A 2 x i -2 A 2 (A ) ( d A(A )) � Continuing this process, we have the following for some p : for some p : x i = p A i x 0 � Hence x is obtained by some power of A and � Hence, x i is obtained by some power of A, and, hence, the “power” method. 10
Pow er Method: 6/11 Pow er Method: 6/11 Example: Consider the following 2 × 2 matrix � Example: g ⎡ ⎤ 2 3 = ⎢ ⎢ ⎥ ⎥ A 1 1 4 4 ⎣ ⎣ ⎦ ⎦ � This matrix has eigenvalues 5 and 1 and corresponding eigenvectors [1,1] and [-3,1] � Let us start with z=[1/2,1] . Since the maximum entry of z is 1, no scaling is needed. t f i 1 li i d d � Compute w=A ⋅ z = [4,9/2]. 11
Pow er Method: 7/11 Pow er Method: 7/11 � Since w=[4,9/2] and its largest entry is 9/2, g y � The approximate eigenvalue is 9/2 � The scaled z = w/(9/2) = [8/9,1] � Compute w = A ⋅ z =[43/9,44/9]. Now, we have � The approximate eigenvalue is 44/9 � The new z = [43/44,1] � Compute w = A ⋅ z = [109/22,219/44] and we have � The approximate eigenvalue is 219/44 � The new z = [218/219,1] � After 3 iterations, we have an approximate eigenvalue 219/44 = 4.977 ≈ 5 and eigenvector [218/219,1] = [0.9954,1] ≈ [1,1]. [218/219 1] [0 9954 1] [1 1] 12
Pow er Method: 8/11 Pow er Method: 8/11 � A is the input matrix, z an approx. eigenvector A is the input matrix, z an approx. eigenvector z = random and scaled vector ! initialize DO ! loop until done w = A*z max = 1 ! find the |max| entry | | DO i = 2, n IF (ABS(w(i)) > ABS(w(max))) max = i END DO END DO eigen_value = w(max) ! ABS(w(max)) the largest DO i = 1, n ! Scale w(*) to z(*) z(i) = w(i)/eigen value z(i) = w(i)/eigen_value END DO IF (ABS(A*z – eigen_value*z)) < Tol) EXIT END DO END DO 13
Pow er Method: 9/11 Pow er Method: 9/11 � Exam Example: p le: Find an eigenvalue and its g corresponding eigenvector of A, x 0 = [1,1,1,1]: − − ⎡ ⎡ ⎤ ⎤ 11 26 3 12 ⎢ ⎥ − − 3 12 3 6 ⎢ ⎥ ⎢ ⎢ − − ⎥ ⎥ 31 99 15 44 ⎢ ⎢ ⎥ ⎥ − − − 9 10 3 4 ⎣ ⎦ � Iter 1: w=[ 24 12 97 8] approx λ = 97 and � Iter 1: w=[-24,-12,-97,-8], approx. λ = -97 and new z=w/(-97)=[0.247423,0.123711,1,0.0824742]. � Iter 2: w = [1 51546 1 76289 6 79381 2 34021] � Iter 2: w = [1.51546,1.76289,6.79381,-2.34021], approx. λ = 6.79381, z=w/(6.79381) = [0.223065,0.259484,1,-0.344461] [0.223065,0.259484,1, 0.344461] 14
Pow er Method: 10/11 Pow er Method: 10/11 � Iter 3: w = [2.84067,2.62216,11.3824,-2.20941], Iter 3: w [2.84067,2.62216,11.3824, 2.20941], approx. λ = 11.3824, new z=w/ λ =[0.249567, 0.230369,1,-0.194107] 0.230369,1, 0.194107] � Iter 4: w=[2.08492,2.14891,8.47074,-2.28116], approx λ = 8 47074 new z = w/ λ = approx λ = 8.47074, new z = w/ λ = [0.246132,0.253687,1,-0.269299] � 15 � 15 15 more 15 more more iterations more iterations iterations iterations ……… ……… � Iter 19: approx. λ = 9 and corresponding eigenvector ( i.e ., z) = [0.25, 0.25,1,-0.25] 15
Pow er Method: 11/11 Pow er Method: 11/11 � What What What does What does does pow er does pow er pow er method pow er method method do? method do? do? do? � It finds the largest eigenvalue ( i.e ., dominating eigenvalue ) and its corresponding eigenvector eigenvalue ) and its corresponding eigenvector. � If vector z is perpendicular to the eigenvector corresponding to the largest eigenvalue, power di t th l t i l method will not converge in exact arithmetic . � Thus, z may be a random vector, initially. � Convergence rate is | λ 2 / λ 1 |, where λ 1 and λ 2 are 2 1 1 2 the largest and second largest eigenvalues. � If rate is << 1, faster convergence is possible. If e s , s e co ve ge ce s poss b e. f it is close to 1, convergence will be very slow. 16
Recommend
More recommend
