Course : Data mining Lecture : Spectral graph analysis Aristides - PowerPoint PPT Presentation

Course : Data mining Lecture : Spectral graph analysis Aristides Gionis Department of Computer Science Aalto University visiting in Sapienza University of Rome fall 2016

spectral graph theory

spectral graph theory objective : • view the adjacency (or related) matrix of a graph with a linear algebra lens • identify connections between spectral properties of such a matrix and structural properties of the graph • connectivity • bipartiteness • cuts • ... • spectral properties = eigenvalues and eigenvectors • in other words, what does the eigenvalues and eigenvectors of the adjacency (or related) matrix tell us about the graph? Data mining — Spectral graph analysis 3

background: eigenvalues and eigenvectors • consider a real n × n matrix A , i.e., A ∈ R n × n • λ ∈ C is an eigenvalue of A if there exists x ∈ C n , x � = 0 such that A x = λ x • such a vector x is called eigenvector of λ • alternatively, ( A − λ I ) x = 0 or det ( A − λ I ) = 0 • it follows that A has n eigenvalues (possibly complex and possibly with multiplicity > 1) Data mining — Spectral graph analysis 4

background: eigenvalues and eigenvectors • consider a real and symmetric n × n matrix A (e.g., the adjacency matrix of an undirected graph) • then – all eigenvalues of A are real – eigenvectors of different eigenvalues are orthogonal i.e., if x 1 an eigenvector of λ 1 and x 2 an eigenvector of λ 2 then λ 1 � = λ 2 implies x 1 ⊥ x 2 (or x T 1 x 2 = 0) • A is positive semi-definite if x T A x ≥ 0 for all x ∈ R n • a symmetric positive semi-definite real matrix has real and non negative eigenvalues Data mining — Spectral graph analysis 5

background: eigenvalues and eigenvectors • consider a real and symmetric n × n matrix A • the eigenvalues λ 1 , . . . , λ n of A can be ordered λ 1 ≤ . . . ≤ λ n • theorem [variational characterization of eigenvalues] x T A x λ n = max x T x x � = 0 x T A x λ 1 = min x T x x � = 0 x T A x λ 2 = min and “so on” for the other eigenvalues x T x x � = 0 x T x 1 = 0 • very useful way to think about eigenvalues Data mining — Spectral graph analysis 6

background: eigenvalues and eigenvectors • the inverse holds, i.e., x T A x � ij A ij x i x j λ 1 = min = min x T x i x 2 � x � = 0 x � = 0 i • and if x is an optimal vector, then x is eigenvector of λ 1 • similarly � x T A x ij A ij x i x j λ 2 = min = min x T x i x 2 � x � = 0 x � = 0 i x T x 1 = 0 x T x 1 = 0 • and if x is an optimal vector, then x is eigenvector of λ 2 Data mining — Spectral graph analysis 7

spectral graph analysis • apply the eigenvalue characterization for graphs • question : which matrix to consider ? – the adjacency matrix A of the graph – some matrix B so that x T B x is related to a structural – property of the graph • consider G = ( V , E ) an undirected and d -regular graph (regular graph is used wlog for simplicity of expositions) • let A be the adjacency matrix of G : • define the laplacian matrix of G as  1 if i = j L = I − 1  d A or L ij = − 1 / d if ( i , j ) ∈ E , i � = j 0 if ( i , j ) �∈ E , i � = j  Data mining — Spectral graph analysis 8

spectral graph analysis • for the laplacian matrix L = I − 1 d A it is x T L x = 1 � | x u − x v | 2 d ( u , v ) ∈ E • here, x u is the coordinate of the eigenvector x that corresponds to vertex u ∈ V • eigenvector x is seen as a one-dimensional embedding • i.e., mapping the vertices of the graph onto the real line Data mining — Spectral graph analysis 9

spectral graph analysis high-level remark • many graph problems can be modeled as mapping of vertices to a discrete space e.g., a cut is a mapping of vertices to { 0 , 1 } • we aim to find a spectral formulation so that an eigenvector x is a relaxation of the discrete graph problem i.e., optimizes the same objective but without the integrality constraint Data mining — Spectral graph analysis 10

the smallest eigenvalue apply the eigenvalue characterization theorem for L • what is λ 1 ? ( u , v ) ∈ E | x u − x v | 2 � x T L x λ 1 = min x T x = min u ∈ V x 2 d � x � = 0 x � = 0 u • observe that λ 1 ≥ 0 • can it be λ 1 = 0 ? • yes : take x to be the constant vector Data mining — Spectral graph analysis 11

the second smallest eigenvalue apply the eigenvalue characterization theorem for L • what is λ 2 ? ( u , v ) ∈ E | x u − x v | 2 � x T L x λ 2 = min x T x = min u ∈ V x 2 d � x � = 0 x � = 0 u x T x 1 = 0 x T x 1 = 0 • can it be λ 2 = 0 ? • λ 2 = 0 if and only if the graph is disconnected map the vertices of each connected component to a different constant Data mining — Spectral graph analysis 12

the k -th smallest eigenvalue • alternative characterization for λ k ( u , v ) ∈ E | x u − x v | 2 � λ k = min max u ∈ V x 2 d � x � = 0 u x ∈ S S : k -dim • λ k = 0 if and only if the graph has at least k connected components Data mining — Spectral graph analysis 13

the largest eigenvalue • what about λ n ? ( u , v ) ∈ E | x u − x v | 2 � x T L x λ n = max x T x = max u ∈ V x 2 d � x � = 0 x � = 0 u • consider a boolean version of this problem • restrict mapping to {− 1 , + 1 } ( u , v ) ∈ E | x u − x v | 2 � λ n ≥ max u ∈ V x 2 d � x ∈{− 1 , + 1 } n u Data mining — Spectral graph analysis 14

the largest eigenvalue • mapping of vertices to {− 1 , + 1 } corresponds to a cut S then ( u , v ) ∈ E | x u − x v | 2 � λ n ≥ max u ∈ V x 2 d � x ∈{− 1 , + 1 } n u 4 E ( S , V \ S ) = max d n S ⊆ V 4 E ( S , V \ S ) = max 2 | E | S ⊆ V 2 maxcut ( G ) = | E | • it follows that if G bipartite then λ n ≥ 2 (because if G bipartite exists S that cuts all edges) Data mining — Spectral graph analysis 15

the largest eigenvalue • on the other hand ( u , v ) ∈ E | x u − x v | 2 � λ n = max u ∈ V x 2 d � x � = 0 u u ∈ V x 2 ( u , v ) ∈ E ( x u + x v ) 2 2 d � u − � = max u ∈ V x 2 d � x � = 0 u ( u , v ) ∈ E ( x u + x v ) 2 � = 2 − min u ∈ V x 2 d � x � = 0 u • first note that λ n ≤ 2 • λ n = 2 iff there is x s.t. x u = − x v for all ( u , v ) ∈ E • λ n = 2 iff G has a bipartite connected component Data mining — Spectral graph analysis 16

summary so far eigenvalues and structural properties of G : • λ 2 = 0 iff G is disconnected • λ k = 0 iff G has at least k connected components • λ n = 2 iff G has a bipartite connected component Data mining — Spectral graph analysis 17

robustness • how robust are these results ? • for instance, what if λ 2 = ǫ ? is the graph G almost disconnected ? i.e., does it have small cuts ? • or, what if λ n = 2 − ǫ ? does it have a component that is “close” to bipartite ? Data mining — Spectral graph analysis 18

the second eigenvalue ( u , v ) ∈ E ( x u − x v ) 2 ( u , v ) ∈ E ( x u − x v ) 2 � � λ 2 = min = min u ∈ V x 2 d d � � ( u , v ) ∈ V 2 ( x u − x v ) 2 x � = 0 x � = 0 u n x T x 1 = 0 x T x 1 = 0 where V 2 is the set of ordered pairs of vertices why? � 2 �� ( x u − x v ) 2 = n � � � � x 2 x 2 v − 2 x u x v = n v − 2 x u v u , v v u ( u , v ) ∈ V 2 � x u = 0 since x T x 1 = 0 and u Data mining — Spectral graph analysis 19

the second eigenvalue ( u , v ) ∈ E ( x u − x v ) 2 � E ( u , v ) ∈ E [( x u − x v ) 2 ] n λ 2 = min ( u , v ) ∈ V 2 ( x u − x v ) 2 = min d E ( u , v ) ∈ V 2 [( x u − x v ) 2 ] d � x � = 0 x � = 0 n x T x 1 = 0 x T x 1 = 0 consider again discrete version of the problem, x u ∈ { 0 , 1 } E ( u , v ) ∈ E [( x u − x v ) 2 ] n n E ( S , S ) E ( u , v ) ∈ V 2 [( x u − x v ) 2 ] = min = usc ( G ) min d d x � = { 0 , 1 } n | S | | S | S ⊆ V x non const usc ( G ) : uniform sparsest cut of G Data mining — Spectral graph analysis 20

uniform sparsest cut • it can be shown that � λ 2 ≤ usc ( G ) ≤ 8 λ 2 • the first inequality holds the by definition of relaxation • second inequality is constructive : • if x is an eigenvector of λ 2 then there is some t ∈ V such that the cut ( S , V \ S ) = ( { u ∈ V | x u ≤ x t } , { u ∈ V | x u > x t } ) has cost usc ( S ) ≤ √ 8 λ 2 Data mining — Spectral graph analysis 21

conductance • conductance : another measure for cuts • the conductance of a set S ⊆ V is defined as φ ( S ) = E ( S , V \ S ) d | S | • expresses the probability to “move out” of S by following a random edge from S • we are interested in sets of small conductance • the conductance of the graph G is defined as φ ( G ) = φ ( S ) min S ⊆ V 0 ≤ S ≤| V | / 2 Data mining — Spectral graph analysis 22

Cheeger’s inequality • Cheeger’s inequality: λ 2 2 ≤ usc ( G ) � ≤ φ ( G ) ≤ 2 λ 2 2 ⇒ conductance is small if and only if λ 2 is small • the two leftmost inequalities are “easy” to show • the first follows by the definition of relaxation • the second follows by usc ( S ) E ( S , V \ S ) ≤ E ( S , V \ S ) = n = φ ( S ) 2 2 d | S || V \ S | d | S | since | V \ S | ≥ n / 2 Data mining — Spectral graph analysis 23