http://cs224w.stanford.edu Better and better clusters (k), (score) - PowerPoint PPT Presentation

CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University http://cs224w.stanford.edu

Better and better clusters Φ (k), (score) Clusters get worse and worse Best cluster has ~100 nodes k, (cluster size) 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 2

Denser and denser network core Small good communities Nested core-periphery 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 3

 Intuition: Self-similarity  Object is similar to a part of itself (i.e. the whole has the same shape as one or more of the parts  Mimic recursive graph / community growth Recursive expansion Initial graph  Kronecker Product is a way of generating self-similar matrices 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 4

[PKDD ‘05] Intermediate stage (3x3) (9x9) Initiator graph After the growth phase 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 5

 Kronecker product of matrices A and B is given by N x M K x L N*K x M*L  Define a Kronecker product of two graphs as a Kronecker product of their adjacency matrices 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 6

[PKDD ‘05]  Continuing multypling with K 1 we obtain K 4 and so on … K 1 9 x 9 3 x 3 K 4 adjacency matrix 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 7

[PKDD ‘05] 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 8

[PKDD ‘05]  Kronecker graph: a growing sequence of graphs by iterating the Kronecker product K 1  Note: One can easily use multiple initiator matrices ( K 1 ’’’ ) (even of different ’ , K 1 ’’ , K 1 sizes) 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 9

[PKDD ‘05]  For K 1 on N 1 nodes and E 1 edges K k (k th Kronecker power of K 1 ) has:  N 1 k nodes  E 1 k edges K 1  We get densification power-law:  𝑭 𝒖 ∝ 𝑶 𝒖 𝒃 , What is a? 𝐦𝐦𝐦 𝑭 𝒖 𝐦𝐦𝐦 𝑭 𝟐  𝒃 = = 𝐦𝐦𝐦 ( 𝑶 𝟐 ) 𝐦𝐦𝐦 𝑶 𝒖 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 10

[PKDD ’05]  Kronecker graphs have many properties found in real networks:  Properties of static networks  Power-Law like Degree Distribution  Power-Law eigenvalue and eigenvector distribution  Small Diameter  Properties of dynamic networks  Densification Power Law  Shrinking/Stabilizing Diameter 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 11

[PKDD ’05]  Observation: Edges in Kronecker graphs: where X are appropriate nodes in G and H  Why?  An entry in matrix G ⊗ H is a multiplication of entries in G and H. 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 12

[PKDD ’05]  Theorem: Constant diameter : If G , H have diameter d then G ⊗ H has diameter d  What is distance between nodes u, v in G ⊗ H ?  Let u=[a,b], v=[a’,b’] (using notation from last slide) t hen edge (u,v) in G ⊗ H iif (a,a’) ∈ G and (b,b’) ∈ H  So, path a to a’ in G is less d steps: a 1 ,a 2 ,a 3 ,…,a d  And path b to b’ in H is less d steps: b 1 ,b 2 ,b 3 ,…,b d  Then: edge ([a 1 ,b 1 ], [a 2 ,b 2 ]) is in G ⊗ H  So it takes <d steps to get from u to v in G ⊗ H  Consequence:  If K 1 has diameter d then graph K k also has diameter d 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 13

[PKDD ’05]  Create N 1 × N 1 probability matrix Θ 1  Compute the k th Kronecker power Θ k  For each entry p uv of Θ k include an edge ( u,v ) in K k with probability p uv Probability of edge p ij Kronecker 0.25 0.10 0.10 0.04 multiplication 0.5 0.2 Instance 0.05 0.15 0.02 0.06 0.1 0.3 matrix K 2 0.05 0.02 0.15 0.06 0.01 0.03 0.03 0.09 Θ 1 flip biased Θ 2 = Θ 1 ⊗ Θ 1 coins 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 14

[Mahdian-Xu, WAW ’07] What is known about Stochastic Kronecker?  Undirected Kronecker graph model with:  Connected , if: a b Θ 1 =  b+c > 1 b c  Connected component of size Θ (n) , if: a>b>c  (a+b)(b+c) > 1  Constant diameter , if:  b+c > 1  Not searchable by a decentralized algorithm 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 15

 Given a real network G a b Θ 1 = Want to estimate the initiator matrix: b d  Method of moments [Gleich&Owen ‘09]  Compare counts of and solve the system of equations  For every of the 4 subgraphs, we get an equation:  2 E[# ] = (a+2b+c) k - (a+c) k where k = log 2 (N)  2 E[# ] = …  …  Now solve the system of equations by trying all possible values (a,b,c) 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 16

[ICML ‘07]  Maximum Likelihood Estimation Θ P( | ) Kronecker arg max 1 Θ 1  Naïve estimation takes O(N!N 2 ) : Θ 1 = a b  N! for different node labelings: c d  Solution: Metropolis sampling: N!  (big) const  N 2 for traversing graph adjacency matrix  Solution: Kronecker product ( E << N 2 ): N 2  E  Do gradient descent 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 17

KronFit: Maximum likelihood estimation  Given real graph G  Find Kronecker initiator graph Θ (i.e., ) a b c d which Θ arg max P ( G | ) Θ  We then need to (efficiently) calculate Θ P ( G | )  And maximize over Θ (e.g., using gradient descent) 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 18

[ICML ‘07]  Given a graph G and Kronecker matrix Θ we calculate probability that Θ G generated G P(G| Θ ) 1 0 1 1 0.25 0.10 0.10 0.04 0 1 0 1 0.05 0.15 0.02 0.06 0.5 0.2 1 0 1 1 0.05 0.02 0.15 0.06 0.1 0.3 1 1 1 1 0.01 0.03 0.03 0.09 Θ Θ k G P (G| Θ ) Θ = Π Θ Π − Θ P ( G | ) [ u , v ] ( 1 [ u , v ]) k k ∈ ∉ ( u , v ) G ( u , v ) G 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 19

[ICML ‘07] Θ k  Nodes are unlabeled Θ 0.25 0.10 0.10 0.04  Graphs G’ and G” should 0.5 0.2 0.05 0.15 0.02 0.06 have the same probability 0.1 0.3 0.05 0.02 0.15 0.06 P(G’| Θ ) = P(G”| Θ ) 0.01 0.03 0.03 0.09 σ  One needs to consider all G’ node correspondences σ 1 0 1 0 1 3 0 1 1 1 ∑ Θ = Θ σ σ P ( G | ) P ( G | , ) P ( ) 2 1 1 1 1 σ 4 0 0 1 1  All correspondences are a G” 2 1 0 1 1 4 priori equally likely 0 1 0 1 1 1 0 1 1  There are O(N!) 3 1 1 1 1 correspondences P(G’| Θ ) = P(G”| Θ ) 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 20

[ICML ‘07]  Assume that we solved the node correspondence problem  Calculating Θ = Π Θ Π − Θ P ( G | ) [ u , v ] ( 1 [ u , v ]) k k ∈ ∉ ( u , v ) G ( u , v ) G  Takes O(N 2 ) time 1 0 1 1 0.25 0.10 0.10 0.04 0 1 0 1 0.05 0.15 0.02 0.06 σ 1 0 1 1 0.05 0.02 0.15 0.06 0 0 1 1 0.01 0.03 0.03 0.09 Θ k G P(G| Θ , σ ) 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 21

 Experimental setup Θ = a b c d  Given real graph G  Gradient descent from random initial point  Obtain estimated parameters Θ  Generate synthetic graph K using Θ  Compare properties of graphs G and K  Note:  We do not fit the graph properties themselves  We fit the likelihood and then compare the properties 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 28

 Can gradient descent recover true parameters?  Generate a graph from random parameters  Start at random point and use gradient descent  We recover true parameters 98% of the times 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 29

[ICML ‘07]  Real and Kronecker are very close: 0.99 0.54 Θ 1 = 0.49 0.13 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 30

[JMLR ‘10]  What do estimated parameters tell us about the network structure? b edges a b Θ = a edges d edges c d c edges 11/28/2011 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu 31