Kernel Normalized Cut: a Theoretical Revisit * Yoshikazu Terada 1,3 - PowerPoint PPT Presentation

Kernel Normalized Cut: a Theoretical Revisit * Yoshikazu Terada 1,3 & Michio Yamamoto 2,3 1 Graduate School of Engineering Science, Osaka University 2 Graduate School of Environmental and Life Science, Okayama University 3 RIKEN Center for Advanced Intelligence Project (AIP) Unsupervised Learning (Room 103) 2 2 12:05 - 12:10, Jun 13, 2019 (Thu) 1 1 ICML2019@Long Beach 0 0 −1 −1 −2 −2 0 1 2 3 4 5 0 1 2 3 4 5

What is Normalized cut? � 2 Normalized cut (Ncut; Shi and Malik, 2000 ) Ncut = Graph partitioning method Goal = To find “clusters” in the graph: Cluster 1 Many edges inside the cluster Cluster 2 Fewer edges between different clusters Ncut Ncut Mcut Ncut = Balanced cut Each cluster is “reasonably large”! Cut between different clusters is small. Objective function of Ncut (Number of clusters = 2) ‣ : Similarity matrix, ‣ Min cut: Balancing term!

Normalized cut and its related methods � 3 Normalized cut, Spectral clustering, Weighted kernel k -means Ncut is an NP hard problem Normalized Spectral clustering (SC) = Continuous relaxation of Ncut Ncut and Weighted Kernel K -Means (WKKM) (Dhillon et al., 2007) ‣ WKKM with kernel h and weight : ‣ Ncut = WKKM with Setting 100 2 2 80 1 1 Kernel Graph 60 0 0 function cut 40 −1 −1 20 −2 −2 0 1 2 3 4 0 1 2 3 4 0 20 40 60 80 100 Data points Similarity matrix Clustering result!

Theoretical properties of Ncut � 4 Overview of this study We study theoretical properties of clustering based on Ncut! Shi and Malik Dhillon et al. (2000, IEEE PAMI) Empirical (2007, IEEE PAMI) Norm. graph Weighted KM Ncut for Norm. SC for Laplacian in = data points data points n -dim. space (eigenvector) von Luxburg et al. This study (2008, AoS) Population Optimality Limit operator Ncut for Weighted KM of the partition in func. space population 6 = in RKHS is not clear (eigenfunction) distribution We also derive the fast rate of convergence of the normalized cut!

Nromalized Cut Normalized SC in both Ncut same tuning we used the Numerical experiments −2 −1 ! " # −2 −1 ! " # parameter Note that ! ! and SC! " " # # $ $ % % & & −2 −1 ! " # −2 −1 ! " # ! ! " " # # $ $ % % & & −2 −1 ! " # −2 −1 ! " # ! ! " " # # $ $ % % Spectral clustering & & −2 −1 ! " # −2 −1 ! " # ! ! " " # # $ $ % % & & 0.05 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0 Normalized cut 500 1000 Normalized cut Normalized SC 1500 2000 � 5