data driven clustering via parameterized lloyds families
play

Data-driven Clustering via Parameterized Lloyds Families Travis - PowerPoint PPT Presentation

Jan 03, 2023 •144 likes •675 views

Data-driven Clustering via Parameterized Lloyds Families Travis Dick Joint work with Maria-Florina Balcan and Colin White Carnegie Mellon University NeurIPS 2018 Data-driven Clustering Data-driven Clustering Clustering aims to divide a

  1. Data-driven Clustering via Parameterized Lloyds Families Travis Dick Joint work with Maria-Florina Balcan and Colin White Carnegie Mellon University NeurIPS 2018

  2. Data-driven Clustering

  3. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters.

  4. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering.

  5. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering.

  6. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering.

  7. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering. • However, most clustering algorithms minimize a clustering cost function.

  8. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering. • However, most clustering algorithms minimize a clustering cost function. • Hope that low-cost clusterings recover the natural clusters.

  9. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering. • However, most clustering algorithms minimize a clustering cost function. • Hope that low-cost clusterings recover the natural clusters. • There are many algorithms and many objectives.

  10. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering. • However, most clustering algorithms minimize a clustering cost function. • Hope that low-cost clusterings recover the natural clusters. • There are many algorithms and many objectives. How do we choose the best algorithm for a specific application?

  11. Data-driven Clustering • Clustering aims to divide a dataset into self-similar clusters. • Goal: find some unknown natural clustering. • However, most clustering algorithms minimize a clustering cost function. • Hope that low-cost clusterings recover the natural clusters. • There are many algorithms and many objectives. How do we choose the best algorithm for a specific application? Can we automate this process?

  12. Learning Model

  13. Learning Model • An unknown distribution ! over clustering instances.

  14. Learning Model • An unknown distribution ! over clustering instances. • Given a sample " # , … , " & ∼ ! annotated by their target clusterings.

  15. Learning Model • An unknown distribution ! over clustering instances. • Given a sample # $ , … , # ' ∼ ! annotated by their target clusterings. • Find an algorithm " that produces clusterings similar to the target clusterings.

  16. Learning Model • An unknown distribution ! over clustering instances. • Given a sample # $ , … , # ' ∼ ! annotated by their target clusterings. • Find an algorithm " that produces clusterings similar to the target clusterings. • Want " to also work well for new instances from ! !

  17. Learning Model • An unknown distribution ! over clustering instances. • Given a sample # $ , … , # ' ∼ ! annotated by their target clusterings. • Find an algorithm " that produces clusterings similar to the target clusterings. • Want " to also work well for new instances from ! ! • In this work: 1. Introduce large parametric family of clustering algorithms, (*, +) -Lloyds.

  18. Learning Model • An unknown distribution ! over clustering instances. • Given a sample # $ , … , # ' ∼ ! annotated by their target clusterings. • Find an algorithm " that produces clusterings similar to the target clusterings. • Want " to also work well for new instances from ! ! • In this work: 1. Introduce large parametric family of clustering algorithms, (*, +) -Lloyds. 2. Efficient procedures for finding best parameters on a sample.

  19. Learning Model • An unknown distribution ! over clustering instances. • Given a sample # $ , … , # ' ∼ ! annotated by their target clusterings. • Find an algorithm " that produces clusterings similar to the target clusterings. • Want " to also work well for new instances from ! ! • In this work: 1. Introduce large parametric family of clustering algorithms, (*, +) -Lloyds. 2. Efficient procedures for finding best parameters on a sample. 3. Generalization: optimal parameters on sample are nearly optimal on ! .

  20. Lloyds Method

  21. Lloyds Method • Maintains ! centers " # , … , " & that define clusters.

  22. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers.

  23. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center.

  24. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points.

  25. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points. 3. Repeat until convergence.

  26. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points. 3. Repeat until convergence.

  27. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points. 3. Repeat until convergence.

  28. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points. 3. Repeat until convergence.

  29. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points. 3. Repeat until convergence.

  30. Lloyds Method • Maintains ! centers " # , … , " & that define clusters. • Perform local search to improve the ! -means cost of the centers. 1. Assign each point to nearest center. 2. Update each center to be the mean of assigned points. 3. Repeat until convergence.

  31. Initial Centers are Important!

  32. Initial Centers are Important! • Lloyd’s method can get stuck if initial centers are chosen poorly

  33. Initial Centers are Important! • Lloyd’s method can get stuck if initial centers are chosen poorly • Initialization is a well-studied problem with many proposed procedures (e.g., ! -means++)

  34. Initial Centers are Important! • Lloyd’s method can get stuck if initial centers are chosen poorly • Initialization is a well-studied problem with many proposed procedures (e.g., ! -means++) • Best method will depend on properties of the clustering instances.

  35. The (", $) -Lloyds Family

  36. The (", $) -Lloyds Family Initialization: Parameter "

  37. The (", $) -Lloyds Family Initialization: Parameter " • Use & ' -sampling (generalizing & ( -sampling of ) -means++)

  38. The (", $) -Lloyds Family Initialization: Parameter " • Use & ' -sampling (generalizing & ( -sampling of ) -means++) • Choose initial centers from dataset * randomly.

  39. The (", $) -Lloyds Family Initialization: Parameter " • Use & ' -sampling (generalizing & ( -sampling of ) -means++) • Choose initial centers from dataset * randomly. ' . • Probability that point + ∈ * is center - . is proportional to & +, - / , … , - .1/

  40. The (", $) -Lloyds Family Initialization: Parameter " • Use & ' -sampling (generalizing & ( -sampling of ) -means++) • Choose initial centers from dataset , randomly. ' . • Probability that point - ∈ , is center / 0 is proportional to & -, / 1 , … , / 031 " = 0 : random initialization

  41. The (", $) -Lloyds Family Initialization: Parameter " • Use & ' -sampling (generalizing & ( -sampling of ) -means++) • Choose initial centers from dataset - randomly. ' . • Probability that point . ∈ - is center 0 1 is proportional to & ., 0 2 , … , 0 142 " = 0 : random initialization " = 2 : ) -means++

  42. The (", $) -Lloyds Family Initialization: Parameter " • Use & ' -sampling (generalizing & ( -sampling of ) -means++) • Choose initial centers from dataset . randomly. ' . • Probability that point / ∈ . is center 1 2 is proportional to & /, 1 3 , … , 1 253 " = 0 : random initialization " = 2 : ) -means++ " = ∞ : farthest first

Recommend

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Biology-Driven Clustering of Microarray Data Applications to the NCI60 Data Set K.R. Coombes,

Biology-Driven Clustering of Microarray Data Applications to the NCI60 Data Set K.R. Coombes,

Biology-Driven Clustering of Microarray Data Applications to the NCI60 Data Set K.R. Coombes, K.A. Baggerly, D.N. Stivers, J. Wang, D. Gold, H.G. Sung, and S.J. Lee Introduction Microarray data is more than a large, unstructured matrix.

670 views • 25 slides
Clustering Clustering What? Given some input data, partition the data in multiple groups

Clustering Clustering What? Given some input data, partition the data in multiple groups

Clustering Clustering What? Given some input data, partition the data in multiple groups Why? Approximate large/infinite/continuous set of objects with finite set of representatives Eg. Vector quantization, codebook learning,

846 views • 68 slides
Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 7: Clustering in Bioinformatics Clustering Gene Expression Data Chlo-Agathe Azencott & Karsten Borgwardt February 18 to March 1, 2013 Machine Learning & Computational Biology Research Group Max

538 views • 51 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Parameterized Complexity of Integer Linear Programming (ILP) Sebastian Ordyniak Parameterized

Parameterized Complexity of Integer Linear Programming (ILP) Sebastian Ordyniak Parameterized

Parameterized Complexity of Integer Linear Programming (ILP) Sebastian Ordyniak Parameterized Graph Algorithms & Data Reduction (Shonan Meeting 144, 2019) 1 / 62 Integer Linear Programming (ILP) archetypical problem for NP -complete

1.25k views • 88 slides
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management of web data Outline

607 views • 41 slides
k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering example Clustering example k -means clustering algorithm 1. Start with k randomly chosen means 2. Color data points by the shortest distance to any

1.56k views • 19 slides
Data Mining in Bioinformatics Day 8: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 8: Clustering in Bioinformatics Clustering Gene Expression Data

Data Mining in Bioinformatics Day 8: Clustering in Bioinformatics Clustering Gene Expression Data Chlo-Agathe Azencott & Karsten Borgwardt February 10 to February 21, 2014 Machine Learning & Computational Biology Research Group Max

522 views • 50 slides
Bi-clustering and co-clustering Going further in cluster analysis and classifjcation: publics ou

Bi-clustering and co-clustering Going further in cluster analysis and classifjcation: publics ou

HAL Id: hal-01810380 manant des tablissements denseignement et de Summer School on Clustering, Data Analysis and Visualization of Complex Data, May 2018, Catania, Going further in cluster analysis and classifjcation: Bi-clustering and

1.23k views • 67 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is

Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is

Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is similar to each other, and dissimilar to other cluster data Finding structure in data Hard vs. Soft/Fuzzy vs. Exclusive Understanding and/or summarise

400 views • 26 slides
1 Data-dr Data-driven philosophy n philosophy Data-dr Data-driven: push n: push 7 8

1 Data-dr Data-driven philosophy n philosophy Data-dr Data-driven: push n: push 7 8

What What is is a a senso sensor network? network? 2 Tiny, untethered nodes with severe resource constraints Sensors, e.g., light, moisture, Tiny CPU and memory Da Data ta-Driven Pr -Driven Proc ocessing essing

177 views • 7 slides
DS504/CS586: Big Data Analytics Big Data Clustering II Prof. Yanhua Li Time: 6pm 8:50pm Thu

DS504/CS586: Big Data Analytics Big Data Clustering II Prof. Yanhua Li Time: 6pm 8:50pm Thu

Welcome to DS504/CS586: Big Data Analytics Big Data Clustering II Prof. Yanhua Li Time: 6pm 8:50pm Thu Location: AK 232 Fall 2016 More Discussions, Limitations v Center based clustering K-means BFR algorithm v Hierarchical clustering

873 views • 52 slides
Centroidal Voronoi Tessellations on Meshes Applications LLoyds Algorithm Normal Leonardo K.

Centroidal Voronoi Tessellations on Meshes Applications LLoyds Algorithm Normal Leonardo K.

Centroidal Voronoi Tessellations on Meshes Definition and Centroidal Voronoi Tessellations on Meshes Applications LLoyds Algorithm Normal Leonardo K. Sacht & Thiago S. Pereira Clustering Mixing Both Approaches Visgraf - IMPA

1.38k views • 94 slides
Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science

Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science

Clustering Big Data Anil K. Jain (with Radha Chitta and Rong Jin) Department of Computer Science Michigan State University November 29, 2012 Outline Big Data How to extract information? Data clustering Clustering Big Data

625 views • 44 slides
Clustering Data Clustering with user constraints The clustering problem : Given a set of

Clustering Data Clustering with user constraints The clustering problem : Given a set of

Clustering Data Clustering with user constraints The clustering problem : Given a set of objects, find groups of similar objects Cluster: a collection of data objects Similar to one another within the same cluster

478 views • 36 slides
Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned

Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned

Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned into subsets (clusters), according to a similarity measure, such thatsimilardata is grouped into the same cluster. Unlabeled Data

442 views • 22 slides
CHAPTER VIII VIII CHAPTER Data Clustering and Data Clustering and Self- -Organizing Feature

CHAPTER VIII VIII CHAPTER Data Clustering and Data Clustering and Self- -Organizing Feature

Ugur HALICI - METU EEE - ANKARA 11/18/2004 CHAPTER VIII VIII CHAPTER Data Clustering and Data Clustering and Self- -Organizing Feature Maps Organizing Feature Maps Self CHAPTER VI : VI : Data Clustering CHAPTER Data Clustering &

337 views • 16 slides
Lecture 12: Clustering Geoffrey Hinton Clustering We assume that the data was generated from

Lecture 12: Clustering Geoffrey Hinton Clustering We assume that the data was generated from

CSC321: Neural Networks Lecture 12: Clustering Geoffrey Hinton Clustering We assume that the data was generated from a number of different classes. The aim is to cluster data from the same class together. How do we decide the number of

529 views • 21 slides
Data Clustering with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course

Data Clustering with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course

Data Clustering with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 1 / 62 Contents Introduction Data Clustering with R The Iris Dataset

968 views • 64 slides
DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

905 views • 79 slides
CS145: INTRODUCTION TO DATA MINING Clustering Evaluation and Practical Issues Instructor: Yizhou

CS145: INTRODUCTION TO DATA MINING Clustering Evaluation and Practical Issues Instructor: Yizhou

CS145: INTRODUCTION TO DATA MINING Clustering Evaluation and Practical Issues Instructor: Yizhou Sun yzsun@cs.ucla.edu November 7, 2017 Learnt Clustering Methods Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave

500 views • 18 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides

More recommend