Matrices with Application to Page Rank Markov Matrices Pagerank - PowerPoint PPT Presentation

Linear Algebra Review Introduction Matrices Matrices with Application to Page Rank Markov Matrices Pagerank Anil Maheshwari anil@scs.carleton.ca School of Computer Science Carleton University Canada

Matrices Linear Algebra Review Introduction A Rectangular Array 1 Matrices Operations: Addition; Multiplication; Diagonalization; 2 Markov Matrices Transpose; Inverse; Determinant Pagerank Row Operations; Linear Equations; Gaussian 3 Elimination Types: Identity; Symmetric; Diagonal; Upper/Lower 4 Traingular; Orthogonal; Orthonormal Transformations - Eigenvalues and Eigenvectors 5 Rank; Column and Row Space; Null Space 6 Applications: Page Rank, Dimensionality Reduction, 7 . . .

Matrix Vector Product Linear Algebra Review Matrix-vector product: Ax = b Introduction Matrices Markov Matrices � � 4 Pagerank � 2 � � 6 � 1 = 3 4 − 2 4

Matrix Vector Product Linear Algebra Review Ax = b as linear combination of columns: Introduction � � 4 Matrices � 2 � � 2 � � 1 � 1 = 4 − 2 Markov Matrices 3 4 − 2 3 4 Pagerank

Matrix-Matrix Product Linear Algebra Review Matrix-matrix product A = BC : Introduction Matrices � 2 � � 2 � � 4 � 0 4 8 = Markov Matrices 3 1 0 4 6 16 Pagerank

Matrix-Matrix Product Linear Algebra Review A = BC as sum of rank 1 matrices: Introduction Matrices � 2 � � 2 � � 2 � � � 0 � � 0 4 � � = 2 4 + 0 4 Markov Matrices 3 1 0 4 3 1 Pagerank

RREF Linear Algebra Review   2 2 0 Introduction Let A = 2 4 8 Matrices   10 16 24 Markov Matrices Pagerank 1st Pivot: Replace r 2 by r 2 − r 1 , and r 3 by r 3 − 5 r 1 :  2 2 0  0 2 8   0 6 24 2nd Pivot: Replace r 3 by r 3 − 3 r 2 :   2 2 0 0 2 8   0 0 0

RREF contd. Linear Algebra Review Divide the first row by 2 , the second row by 2 : Introduction Matrices   1 1 0 Markov Matrices 0 1 4 Pagerank   0 0 0 Replace r 1 by r 1 − r 2 :   1 0 − 4 R = 0 1 4   0 0 0

Rank Linear Algebra Review     2 2 0 1 0 − 4 Introduction  RREF  = R A = 2 4 8 − − − → 0 1 4 Matrices   10 16 24 0 0 0 Markov Matrices Pagerank Rank = Number of non-zero pivots = 2 Basis vectors of the row space = rows corresponding to the non-zero pivots in R � 1 � 0 � � v 1 = and v 2 = 0 1 − 4 4 Basis vectors of the column space = Columns of A corresponding to non-zero pivots of R . � 2 � 2 � � u 1 = and u 2 = 2 4 10 16 � 2 � 2 � � A = u 1 v T 1 + u 2 v T 2 = [ 1 0 − 4 ] + [ 0 1 4 ] 2 4 10 16

Null Space Linear Algebra Review The null space of A = All vectors x such that Ax = 0 . Introduction � 0 � Matrices This includes the 0 vector 0 0 Markov Matrices Is there a vector x = ( x 1 , x 2 , x 3 ) ∈ R 3 , such that Pagerank � 2 � 2 � 0 � 0 � � � � Ax = x 1 + x 2 + x 3 = 2 4 8 0 10 16 24 0 x = (1 , − 1 , 1 / 4) , or any of its scalar multiples, satisfies Ax = 0 Dimension of Null Space of A = Number of columns ( A ) - rank( A )= 3 − 2 = 1

Spaces for A Linear Algebra Review Let A be m × n matrix with real entries. Introduction Let R be RREF of A consisting of r ≤ min { m, n } Matrices Markov Matrices non-zero pivots. Pagerank rank ( A ) = r 1 Column space is a subspace of R m of dimension r , 2 and its basis vectors are the columns of A corresponding to the non-zero pivots in R . Row space is a subspace of R n of dimension r , and 3 its basis vectors are the rows of R corresponding to the non-zero pivots. The null-space of A consists of all the vectors x ∈ R n 4 satisfying Ax = 0 . They form a subspace of dimension n − r .

Eigenvalues and Eigenvectors Linear Algebra Review Given an n × n matrix A . Introduction A non-zero vector v is an eigenvector of A , if Av = λv for Matrices Markov Matrices some scalar λ . λ is the eigenvalue corresponding to Pagerank vector v . � 2 � 1 A = 3 4

Example: Eigenvalues and Eigenvectors Linear Algebra Review Introduction Example Matrices � 2 � 1 Markov Matrices Let A = 3 4 Pagerank Observe that � � 1 � 1 � 2 � � 1 � � 1 � � 2 � � 1 1 = 5 and = 1 3 4 3 3 3 4 − 1 − 1 Thus, λ 1 = 5 and λ 2 = 1 are the eigenvalues of A . Corresponding eigenvectors are v 1 = [1 , 3] and v 2 = [1 , − 1] , as Av 1 = λ 1 v 1 and Av 2 = λ 2 v 2 .

Eigenvalues of A k Linear Algebra Review Let Av i = λ i v i Introduction Matrices Consider: Markov Matrices A 2 v i = A ( Av i ) = A ( λ i v i ) = λ i ( Av i ) = λ i ( λ i v i ) = λ 2 i v i Pagerank ⇒ A 2 v i = λ 2 = i v i Eigenvalues of A k For an integer k > 0 , A k has the same eigenvectors as A , but the eigenvalues are λ k .

Matrices with distinct eigenvalues Linear Algebra Review Introduction Propertry Matrices Let A be an n × n real matrix with n distinct eigenvalues. Markov Matrices The corresponding eigenvectors are linearly independent. Pagerank

Matrices with distinct eigenvalues Linear Algebra Review Let A be an n × n real matrix with n distinct eigenvalues. Introduction Let λ 1 , . . . , λ n be the distinct eigenvalues and let Matrices Markov Matrices x 1 , . . . , x n be the corresponding eigenvectors, Pagerank respectively. Let each x i = [ x i 1 , x i 2 , . . . , x in ] . Define an eigenvector matrix X :   x 11 x 21 . . . x n 1 . . . . . . . . X =   . . . .   x 1 n x 2 n . . . x nn Since eigenvectors are linearly independent, we know that X − 1 exists.

Matrices with distinct eigenvalues (contd.) Linear Algebra Review Define a diagonal n × n matrix Λ : Introduction Matrices  λ 1 0 0 . . . 0  Markov Matrices 0 λ 2 0 . . . 0 Pagerank     0 0 λ 3 . . . 0 Λ =    . . . . .  . . . . .   . . . . .   0 0 . . . 0 λ n Consider the matrix product AX ,      =  = X Λ AX = A  x 1 . . . x n  λ 1 x 1 . . . λ n x n

Matrices with distinct eigenvalues (contd.) Linear Algebra Review Since X − 1 exists, we multiply by X − 1 on both the sides Introduction from left and obtain Matrices Markov Matrices X − 1 AX = X − 1 X Λ = Λ (1) Pagerank and when we multiply on the right we obtain AXX − 1 = A = X Λ X − 1 (2)

Matrices with distinct eigenvalues (contd.) Linear Algebra Review Consider diagonalization given by equation A = X Λ X − 1 Introduction Matrices Consider A 2 : Markov Matrices = ( X Λ X − 1 )( X Λ X − 1 ) = X Λ( X − 1 X )Λ X − 1 = X Λ 2 X − 1 Pagerank ⇒ A 2 has the same set of eigenvectors as A , but its = eigenvalues are squared. Similarly, A k = X Λ k X − 1 . Eigenvectors of A k are same as that of A and its eigenvalues are raised to the power of k .

Symmetric Matrices Linear Algebra Review Introduction Example Matrices Consider symmetric matrix S = [ 3 1 1 3 ] . Markov Matrices Its eigenvalues are λ 1 = 4 and λ 2 = 2 and the Pagerank √ √ corresponding eigenvectors are q 1 = (1 / 2 , 1 / 2) and √ √ q 2 = (1 / 2 , − 1 / 2) , respectively. Note that eigenvalues are real and the eigenvectors are orthonormal. √ √ √ √ � 3 � � 1 / � � 4 � � 1 / � 1 2 1 / 2 0 2 1 / 2 √ √ √ √ S = = 1 3 0 2 1 / 2 − 1 / 2 1 / 2 − 1 / 2 Eigenvalues of Symmetric Matrices All the eigenvalues of a real symmetric matrix S are real. Moreover, all components of the eigenvectors of a real symmetric matrix S are real.

Symmetric Matrices (contd.) Linear Algebra Review Introduction Property Matrices Any pair of eigenvectors of a real symmetric matrix S Markov Matrices corresponding to two different eigenvalues are Pagerank orthogonal.

Symmetric Matrices (contd.) Linear Algebra Review 2020-11-03 Property Any pair of eigenvectors of a real symmetric matrix S Matrices corresponding to two different eigenvalues are orthogonal. Symmetric Matrices (contd.) Proof: Let q 1 and q 2 be two eigenvectors corresponding to λ 1 � = λ 2 , respectively. Thus, Sq 1 = λ 1 q 1 and Sq 2 = λ 2 q 2 . Since S is symmetric, q T 1 S = λ 1 q T Multiply by q 2 on the right and we obtain λ 1 q T 1 . 1 q 2 = q T 1 Sq 2 = q T 1 λ 2 q 2 . Since λ 1 � = λ 2 and λ 1 q T 1 q 2 = q T 1 λ 2 q 2 , this implies that q T 1 q 2 = 0 and thus the eigenvectors q 1 and q 2 are orthogonal.

Symmetric Matrices (contd.) Linear Algebra Review Introduction Symmetric matrices with distinct eigenvalues Matrices Let S be a n × n symmetric matrix with n distinct Markov Matrices eigenvalues and let q 1 , . . . , q n be the corresponding Pagerank orthonormal eigenvectors. Let Q be the n × n matrix consiting of q 1 , . . . , q n as its columns. Then S = Q Λ Q − 1 = Q Λ Q T . Furthermore, S = λ 1 q 1 q T 1 + λ 2 q 2 q T 2 + · · · + λ n q n q T n √ � 1 / √ √ √ √ √ � 3 1 � � 1 / 2 � � 2 � � √ √ � � S = = 4 1 / 2 1 / 2 + 2 1 / 2 − 1 / 2 1 3 1 / 2 − 1 / 2