Lecture: k-means & mean-shift clustering Juan Carlos Niebles and - PowerPoint PPT Presentation

Clustering Lecture: k-means & mean-shift clustering Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 22-Oct-2019 1 St Stanfor ord University

CS 131 Roadmap Clustering Pixels Segments Images Videos Web Recognition Neural networks Convolutions Resizing Motion Detection Convolutional Edges Segmentation Tracking Machine learning neural networks Descriptors Clustering 22-Oct-2019 2 St Stanfor ord University

Recap: Image Segmentation • Goal: identify groups of pixels that go together Clustering 22-Oct-2019 3 St Stanfor ord University

Recap: Gestalt Theory • Gestalt: whole or group – Whole is greater than sum of its parts – Relationships among parts can yield new properties/features Clustering • Psychologists identified series of factors that predispose set of elements to be grouped (by human visual system) “I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, 22-Oct-2019 and trees.” Max Wertheimer (1880-1943) Untersuchungen zur Lehre von der Gestalt, Psychologische Forschung , Vol. 4, pp. 301-350, 1923 http://psy.ed.asu.edu/~classics/Wertheimer/Forms/forms.htm 4 Stanfor St ord University

Recap: Gestalt Factors Clustering 22-Oct-2019 • These factors make intuitive sense, but are very difficult to translate into algorithms. 5 St Stanfor ord University

What will we learn today? • K-means clustering • Mean-shift clustering Clustering 22-Oct-2019 Reading: [FP] Chapters: 14.2, 14.4 D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature Space Analysis, PAMI 2002. 6 St Stanfor ord University

What will we learn today? • K-means clustering • Mean-shift clustering Clustering 22-Oct-2019 Reading: [FP] Chapters: 14.2, 14.4 D. Comaniciu and P. Meer, Mean Shift: A Robust Approach toward Feature Space Analysis, PAMI 2002. 7 St Stanfor ord University

Image Segmentation: Toy Example white pixels 3 black pixels gray 2 1 pixels Clustering input image intensity • These intensities define the three groups. 22-Oct-2019 • We could label every pixel in the image according to which of these primary intensities it is. – i.e., segment the image based on the intensity feature. • What if the image isn’t quite so simple? Slide credit: Kristen Grauman 8 St Stanfor ord University

Pixel count Input image Clustering Intensity Pixel count 22-Oct-2019 Input image Intensity Slide credit: Kristen Grauman 9 St Stanfor ord University

Pixel count Clustering Input image Intensity • Now how to determine the three main intensities that define our groups? • We need to cluster. 22-Oct-2019 Slide credit: Kristen Grauman 10 St Stanfor ord University

190 255 0 Intensity 3 2 1 Clustering • Goal: choose three “centers” as the representative intensities, and label every pixel according to which of these centers it is nearest to. Slide credit: Kristen Grauman • Best cluster centers are those that minimize Sum of Square 22-Oct-2019 Distance (SSD) between all points and their nearest cluster center c i : 2 ∑ ∑ ( ) SSD = x − c i clusteri x ∈ clusteri 11 Stanfor St ord University

Clustering for Summarization Goal: cluster to minimize variance in data given clusters – Preserve information Clustering Cluster center Data N K 1 c * , δ * = argmin 2 ∑ ∑ ( ) c i − x j δ ij N 22-Oct-2019 c , δ j i x j c i Whether is assigned to 12 Stanfor St ord University Slide: Derek Hoiem

Clustering • With this objective, it is a “chicken and egg” problem: – If we knew the cluster centers , we could allocate points to groups by assigning each to its closest center. Clustering – If we knew the group memberships , we could get the Slide credit: Kristen Grauman centers by computing the mean per group. 22-Oct-2019 13 St Stanfor ord University

K-means clustering 1. Initialize ( ): cluster centers t = 0 c 1 ,..., c K δ t 2. Compute : assign each point to the closest center Clustering x j – δ t denotes the set of assignment for each to cluster at iteration t c i N K 1 δ t = argmin 2 ( ) ∑ ∑ δ t − 1 c t − 1 i − x j ij N δ j i 1. Computer : update cluster centers as the mean of the points c t N K 1 c t = argmin 2 ( ) 22-Oct-2019 ∑ ∑ δ t c t − 1 i − x j ij N c j i 1. Update , Repeat Step 2-3 till stopped t = t + 1 14 Slide: Derek Hoiem St Stanfor ord University

K-means clustering 1. Initialize ( ): cluster centers t = 0 c 1 ,..., c K Commonly used: random initialization • Or greedily choose K to minimize residual • δ t 2. Compute : assign each point to the closest center Clustering Typical distance measure: • x ) = x T ! • Euclidean sim ( x , ! x x ) = x T ! Cosine • ( ) sim ( x , ! x x ⋅ x ! Others • 1. Computer : update cluster centers as the mean of the points c t N K 1 c t = argmin 2 ( ) ∑ ∑ δ t c t − 1 i − x j 22-Oct-2019 ij N c j i 2. Update , Repeat Step 2-3 till stopped t = t + 1 c t doesn’t change anymore. • 15 Slide: Derek Hoiem St Stanfor ord University

K-means clustering Clustering 1. Initialize 2. Assign Points to 3. Re-compute Repeat (2) and (3) Cluster Centers Clusters Means 22-Oct-2019 • Java demo: http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html 16 St Stanfor ord University Illustration Source: wikipedia

K-means clustering • Converges to a local minimum solution – Initialize multiple runs Clustering • Better fit for spherical data 22-Oct-2019 • Need to pick K (# of clusters) 17 St Stanfor ord University

Segmentation as Clustering Clustering 2 clusters 22-Oct-2019 Original image 18 3 clusters St Stanfor ord University

K-Means++ • Can we prevent arbitrarily bad local minima? Clustering 1.Randomly choose first center. 2.Pick new center with prob. proportional to 2 ( ) x − c i – (Contribution of x to total error) 3.Repeat until K centers. 22-Oct-2019 Slide credit: Steve Seitz • Expected error * optimal ( ) = O log K Arthur & Vassilvitskii 2007 19 St Stanfor ord University

Feature Space • Depending on what we choose as the feature space , we can group pixels in different ways. Clustering • Grouping pixels based on intensity similarity 22-Oct-2019 • Feature space: intensity value (1D) Slide credit: Kristen Grauman 20 St Stanfor ord University

Feature Space • Depending on what we choose as the feature space , we can group pixels in different ways. R=255 G=200 • Grouping pixels based Clustering B=250 on color similarity B R=245 G G=220 B=248 22-Oct-2019 R R=15 R=3 G=189 G=12 B=2 B=2 • Feature space: color value (3D) 21 Slide credit: Kristen Grauman St Stanfor ord University

Feature Space • Depending on what we choose as the feature space , we can group pixels in different ways. • Grouping pixels based Clustering on texture similarity F 1 F 2 Filter bank of 24 filters 22-Oct-2019 … F 24 • Feature space: filter bank responses (e.g., 24D) 22 Slide credit: Kristen Grauman Stanfor St ord University

Smoothing Out Cluster Assignments • Assigning a cluster label per pixel may yield outliers: Clustering Labeled by cluster center’s intensity Original ? 22-Oct-2019 3 • How can we ensure they 2 are spatially smooth? 1 23 Slide credit: Kristen Grauman St Stanfor ord University

Segmentation as Clustering • Depending on what we choose as the feature space , we can group pixels in different ways. Clustering • Grouping pixels based on intensity+position similarity Intensity Y 22-Oct-2019 X Þ Way to encode both similarity and proximity. 24 Slide credit: Kristen Grauman St Stanfor ord University

K-Means Clustering Results • K-means clustering based on intensity or color is essentially vector quantization of the image attributes – Clusters don’t have to be spatially coherent Clustering Image Intensity-based clusters Color-based clusters Image source: Forsyth & Ponce 22-Oct-2019 25 St Stanfor ord University

K-Means Clustering Results • K-means clustering based on intensity or color is essentially vector quantization of the image attributes Clustering – Clusters don’t have to be spatially coherent • Clustering based on (r,g,b,x,y) values enforces more spatial coherence Image source: Forsyth & Ponce 22-Oct-2019 26 St Stanfor ord University

How to evaluate clusters? • Generative Clustering – How well are points reconstructed from the clusters? • Discriminative – How well do the clusters correspond to labels? • Can we correctly classify which pixels belong to the panda? – Note: unsupervised clustering does not aim to be discriminative as we don’t have the 22-Oct-2019 labels. 27 St Stanfor ord University Slide: Derek Hoiem

How to choose the number of clusters? Try different numbers of clusters in a validation set and look at performance. Clustering 22-Oct-2019 28 St Stanfor ord University Slide: Derek Hoiem