Axioms for graph clustering objective functions Twan van Laarhoven - PowerPoint PPT Presentation

Introduction Axioms Modularity Adaptive Modularity Conclusion Axioms for graph clustering objective functions Twan van Laarhoven Institute for Computing and Information Sciences Radboud University Nijmegen, The Netherlands 28th June 2013 1 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Outline Introduction Axioms Modularity Adaptive Modularity Conclusion 2 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion The motivation • There is no strict definition of clustering. • Can we formalize our intuition? • Previous work is about distance based clustering (hierarchical clustering, K-means, etc.) • What about graphs? 3 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion The setting Definition (Graph) A symmetric weighted graph is a pair ( V , E ) of • a finite set V of nodes , and • a function E : V × V → R ≥ 0 of edge weights , such that E ( i , j ) = E ( j , i ) for all i , j ∈ V . • Larger weight = stronger connection. • We allow self loops. 4 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion The setting (cont.) Definition (Clustering) A clustering C of a graph G = ( V , E ) is a partition of its nodes. Definition (Clustering function) A graph clustering function f is a function from graphs G to clusterings of G . Definition (Objective function) A graph clustering objective function Q is a function from graphs G and clusterings of G to R . • Larger objective value = better. 5 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Outline Introduction Axioms Modularity Adaptive Modularity Conclusion 6 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion The form of axioms Things that define clusterings Form Notation 1 Clustering function f ( G ) = argmax C Q ( G , C ) 2 Objective function Q ( G , C ) Q ( G , C ) ≥ Q ( G , D ) or C ≥ G D 3 Objective relation 7 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 1: Scale invariance (first form) A graph clustering objective function Q is scale invariant if • for all graphs G = ( V , E ), • all constants α > 0, f ( G ) = f ( α G ). (where α G = ( V , ( i , j ) �→ α E ( i , j )).) Example     b d b d b d  a  a  = a  = f f e e e c c c 8 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 1: Scale invariance (second form) A graph clustering objective function Q is scale invariant if • for all graphs G = ( V , E ), • all constants α > 0, • all clusterings C of G , Q ( G , C ) = Q ( α G , C ). (where α G = ( V , ( i , j ) �→ α E ( i , j )).) Example     b d b d  a  a  = Q Q  e e c c 8 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 1: Scale invariance (second form) A graph clustering objective function Q is scale invariant if • for all graphs G = ( V , E ), • all constants α > 0, • all clusterings C of G , Q ( G , C ) = α Q ( α G , C ) ??? (where α G = ( V , ( i , j ) �→ α E ( i , j )).) Example     b d b d  a  a  = α Q Q  e e c c 8 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 1: Scale invariance (third form) A graph clustering objective function Q is scale invariant if • for all graphs G = ( V , E ), • all constants α > 0, • all clusterings C 1 , C 2 of G , Q ( G , C 1 ) ≥ Q ( G , C 2 ) if and only if Q ( α G , C 1 ) ≥ Q ( α G , C 2 ). (where α G = ( V , ( i , j ) �→ α E ( i , j )).) Example � � � � � � � � Q ≥ Q ⇐ ⇒ Q ≥ Q 8 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 2: permutation invariance A graph clustering objective function Q is permutation invariant if • for all graphs G = ( V , E ) and • all isomorphisms f : V → V ′ , it is the case that Q ( G , C ) = Q ( f ( G ) , f ( C )). (where f is extended to graphs and clusterings in the obvious way.) Example    x  y b d  a z  = Q Q   e c u v 9 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 3: Richness A graph clustering objective function Q is rich if • for all sets V and • all partitions C ∗ of V , there is • a graph G = ( V , E ) • such that C ∗ is the optimal clustering of G . Intuition: • No trivial objective functions. • No fixed number of clusters. 10 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Definition (Consistent improvement) Let • G = ( V , E ) and G ′ = ( V , E ′ ) be graphs, and • C be a clustering of G and G ′ . Then G ′ is a C-consistent improvement of G if • E ′ ( i , j ) ≥ E ( i , j ) for all i ∼ C j and • E ′ ( i , j ) ≤ E ( i , j ) for all i �∼ C j . Intuition: • Consistent improvements make a clustering fit better. 11 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Basic axioms Axiom 4: Monotonicity A graph clustering objective function Q is monotonic if • for all graphs G , • all clusterings C of G and • all C -consistent improvements G ′ of G it is the case that Q ( G ′ , C ) ≥ Q ( G , C ). Example     b d b d  a  a  ≥ Q Q  e e c c 12 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Local changes Definition (agreement) Let • G 1 = ( V 1 , E 1 ) and G 2 = ( V 2 , E 2 ) be two graphs and • V a ⊆ V 1 ∩ V 2 . The graphs agree on V a if E 1 ( i , j ) = E 2 ( i , j ) for all i , j ∈ V a . Definition (agreement on neighborhood) The graphs also agree on the neighborhood of V a if E 1 ( i , j ) = E 2 ( i , j ) for all i ∈ V a , j ∈ V 1 ∩ V 2 , and E 1 ( i , j ) = 0 for all i ∈ V a , j ∈ V 1 \ V 2 , and E 2 ( i , j ) = 0 for all i ∈ V a , j ∈ V 2 \ V 1 . What this means: • For nodes/clusters in V a , all incident edges are the same. 13 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Local changes Definition (agreement) Let • G 1 = ( V 1 , E 1 ) and G 2 = ( V 2 , E 2 ) be two graphs and • V a ⊆ V 1 ∩ V 2 . The graphs agree on V a if E 1 ( i , j ) = E 2 ( i , j ) for all i , j ∈ V a . Definition (agreement on neighborhood) The graphs also agree on the neighborhood of V a if E 1 ( i , j ) = E 2 ( i , j ) for all i ∈ V a , j ∈ V 1 ∩ V 2 , and E 1 ( i , j ) = 0 for all i ∈ V a , j ∈ V 1 \ V 2 , and E 2 ( i , j ) = 0 for all i ∈ V a , j ∈ V 2 \ V 1 . What this means: • For nodes/clusters in V a , all incident edges are the same. 13 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Local changes Axiom 5: Locality A graph clustering objective function Q is local if • for all graphs G 1 = ( V 1 , E 1 ) and G 2 = ( V 2 , E 2 ) that agree on a set V a and its neighborhood, • for all clusterings C 1 of V 1 \ V a , C 2 of V 2 \ V a and C a , D a of V a . if Q ( G 1 , C a ∪ C 1 ) ≥ Q ( G 1 , D a ∪ C 1 ) then Q ( G 2 , C a ∪ C 2 ) ≥ Q ( G 2 , D a ∪ C 2 ). 14 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Local changes Example     · · b b a a Q ≥ Q     · · · · c c �     · · b b a a     Q ≥ Q  · ·   · ·  c c         15 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Local changes Special cases • G 1 = G 2 : change part of a clustering. In practice: optimize parts separately (divide and conquer). • V a = ∅ : union of two disjoint graphs. 16 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Interlude: Related work Theorem (Kleinberg 2002) There is no clustering function that is permutation invariant, scale invariant, monotonic and rich. Theorem (Ackerman, Ben-David 2008) There is a clustering quality function that is permutation invariant, scale invariant, monotonic and rich. 17 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Discontinuity is magic Theorem There is a graph clustering function that is scale invariant, permutation invariant, monotonic, rich and local. Connected components f coco ( G ) = the connected components of G Q coco ( G , C ) = 1 [ C are the connected components of G ] Huh!?!? • Doesn’t this contradict Kleinberg’s theorem? • No: edge weight 0 = distance ∞ . 18 / 32

Introduction Axioms Modularity Adaptive Modularity Conclusion Discontinuity is magic Theorem There is a graph clustering function that is scale invariant, permutation invariant, monotonic, rich and local. Connected components f coco ( G ) = the connected components of G Q coco ( G , C ) = 1 [ C are the connected components of G ] Huh!?!? • Doesn’t this contradict Kleinberg’s theorem? • No: edge weight 0 = distance ∞ . 18 / 32