clustering algorithms
play

Clustering Algorithms Dalya Baron (Tel Aviv University) XXX Winter - PowerPoint PPT Presentation

Sep 05, 2022 •267 likes •815 views

Clustering Algorithms Dalya Baron (Tel Aviv University) XXX Winter School, November 2018 Clustering Feature 2 Feature 1 Clustering cluster #1 Feature 2 cluster #2 Feature 1 Clustering Why should we look for clusters? cluster #1 Feature

  1. Clustering Algorithms Dalya Baron (Tel Aviv University) XXX Winter School, November 2018

  2. Clustering Feature 2 Feature 1

  3. Clustering cluster #1 Feature 2 cluster #2 Feature 1

  4. Clustering Why should we look for clusters? cluster #1 Feature 2 cluster #2 Feature 1

  5. Clustering

  6. K-means Input: measured features, and the number of clusters, k . The algorithm will classify all the objects in the sample into k clusters. Feature 2 Feature 1

  7. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 Feature 1

  8. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 Two centroids are randomly placed Feature 1

  9. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 The objects are associated to the closest cluster centroid (Euclidean distance). Feature 1

  10. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 New cluster centroids are computed using the average location of the cluster members. Feature 1

  11. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 The objects are associated to the closest cluster centroid (Euclidean distance). Feature 1

  12. K-means The algorithm places randomly k points that represent the centroids of the clusters. (I) The algorithm performs several iterations, in each of them: (II) The algorithm associates each object with a single cluster, according to its distance from the cluster centroid. (III) The algorithm recalculates the cluster centroid according to the objects that are associated with it. Feature 2 The process stops when the objects that are associated with a given class do not change. Feature 1

  13. The anatomy of K-means cluster Internal choices and/or internal cost function: centroids (I) Initial centroids are randomly selected from the set of examples. (II) The global cost function that is minimized by K-means: cluster Euclidean members distance

  14. The anatomy of K-means cluster Internal choices and/or internal cost function: centroids (I) Initial centroids are randomly selected from the set of examples. (II) The global cost function that is minimized by K-means: cluster Euclidean members distance k=3, and two di ff erent random placements of centroids

  15. The anatomy of K-means Input dataset: a list of objects with measured features. For which datasets should we use K-means? Feature 2 Feature 2 Feature 1 Feature 1

  17. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when we have an outlier in the dataset? Feature 2 outlier! Feature 1

  18. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when we have an outlier in the dataset? Feature 2 outlier! Feature 1

  19. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when the features have di ff erent physical units? input dataset K-means output

  20. The anatomy of K-means Input dataset: a list of objects with measured features. What happens when the features have di ff erent physical units? How can we avoid this? input dataset K-means output

  21. The anatomy of K-means Hyper-parameters: the number of clusters, k. Can we find the optimal k using the cost function? k=2 k=3 k=5

  22. The anatomy of K-means Hyper-parameters: the number of clusters, k. Can we find the optimal k using the cost function? k=2 k=3 k=5 Minimal cost function Elbow Number of clusters

  23. Questions?

  24. Hierarchal Clustering or, how to visualize complicated similarity measures Correa-Gallego+ 2016

  25. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Feature 2 Feature 1

  26. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 Feature 1

  27. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  28. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  29. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  30. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  31. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  32. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  33. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. Next: the algorithm merges the two closest clusters into a single cluster. Then, the algorithm re-calculates the distance of the newly-formed cluster to all the rest. Feature 2 distance Feature 1 Dendrogram

  34. Hierarchal Clustering Input: measured features, or a distance matrix that represents the pair-wise distances between the objects. Also, we must specify a linkage method . Initialization: each object is a cluster of size 1. The process stops when all the objects are merged into a single cluster Feature 2 distance Feature 1 Dendrogram

Recommend

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for

Week 7 Video 3 Advanced Clustering Algorithms Today Multiple advanced algorithms for clustering Gaussian Mixture Models Often called EM-based clustering Kind of a misnomer in my opinion What distinguishes this algorithm

681 views • 21 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Algorithms Johannes Bl omer WS 2015/16 1 / 20 Introduction Clustering techniques

Clustering Algorithms Johannes Bl omer WS 2015/16 1 / 20 Introduction Clustering techniques

Clustering Algorithms Johannes Bl omer WS 2015/16 1 / 20 Introduction Clustering techniques for data management and analysis that classify/group given set of objects into categories/subgroups or clusters 2 / 20 Introduction Clustering

1.01k views • 51 slides
INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language Technology Group University of Oslo Oct. 2, 2014 Agenda Yesterday Flat clustering k -Means Today Bottom-up hierarchical clustering.

976 views • 44 slides
Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document clustering Motivations Document representations Success criteria Clustering algorithms Partitional Hierarchical Ch. 16 What is

766 views • 61 slides
Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic Links Applications Algorithms SimRank Yin X., Han J. and Yu P. S., 2006, LinkClus: Efficient ReCoM Clustering via Heterogeneous

221 views • 8 slides
Streaming algorithms for k -center clustering with outliers and with anonymity Richard Matthew

Streaming algorithms for k -center clustering with outliers and with anonymity Richard Matthew

Streaming algorithms for k -center clustering with outliers and with anonymity Richard Matthew McCutchen and Samir Khuller University of Maryland {rmccutch,samir}@cs.umd.edu k -center clustering problem Input: n points in an arbitrary

751 views • 63 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel

Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel

Improved Clustering Algorithms for the Random Cluster Graph Model Ron Shamir Dekel Tsur Tel Aviv University 1/18 The Clustering Problem Input: A graph G . (edges in G represent similarity between the vertices) Output: A partition of the

485 views • 34 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed Rafea Outline Introduction Clustering Algorithms K-means Fuzzy C-means Hierarchical clustering Introduction(1)

394 views • 23 slides
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) September 23, 2015 Agenda Last week Supervised vs unsupervised learning. Vectors space

1.15k views • 54 slides
INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal &

INF4820 Algorithms for AI and NLP Evaluating Classifiers Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) September 23, 2015 Agenda Last week Supervised vs unsupervised learning. Vectors space

496 views • 14 slides
INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal & Stephan

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal & Stephan

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal & Stephan Oepen Language Technology Group (LTG) October 7, 2015 Agenda Last week Evaluation of classifiers Machine learning for class discovery:

892 views • 38 slides
CS270: Lecture 1. Algorithms. Example Problem: clustering. Undergraduate. This class. 1.

CS270: Lecture 1. Algorithms. Example Problem: clustering. Undergraduate. This class. 1.

CS270: Lecture 1. Algorithms. Example Problem: clustering. Undergraduate. This class. 1. Classical. Modern. Flavor of the week? 1. Overview Points: documents, dna, preferences. 2. Cleanly Stated Problems. Shortest Paths, max-flow, MST.

725 views • 5 slides
Spectral Clustering Spectral Clustering? Spectral methods Methods using eigenvectors of

Spectral Clustering Spectral Clustering? Spectral methods Methods using eigenvectors of

Spectral Clustering Spectral Clustering? Spectral methods Methods using eigenvectors of some matrices Involve eigen-decomposition (or spectral decomposition) Seungjin Choi Spectral clustering methods: Algorithms that cluster data

501 views • 10 slides
Algorithms for Matching and Clustering Using only Ordinal Information Elliot Anshelevich

Algorithms for Matching and Clustering Using only Ordinal Information Elliot Anshelevich

Blind, Greedy, and Random: Algorithms for Matching and Clustering Using only Ordinal Information Elliot Anshelevich (together with Shreyas Sekar) Rensselaer Polytechnic Institute (RPI), Troy, NY Maximum Utility Matching Edges have

953 views • 35 slides
MapReduce and Streaming Algorithms for Center-Based Clustering in Doubling Spaces Geppino Pucci

MapReduce and Streaming Algorithms for Center-Based Clustering in Doubling Spaces Geppino Pucci

MapReduce and Streaming Algorithms for Center-Based Clustering in Doubling Spaces Geppino Pucci DEI, University of Padova, Italy Based on joint works with: M. Ceccarello, A. Mazzetto, and A. Pietracaprina 1 Center-based clustering

533 views • 32 slides
Clustering Tendency Assistant Professor : Dr. Mohammad Javad Fadaeieslam By : Amir Shokri

Clustering Tendency Assistant Professor : Dr. Mohammad Javad Fadaeieslam By : Amir Shokri

Clustering Tendency Assistant Professor : Dr. Mohammad Javad Fadaeieslam By : Amir Shokri Farshad Asgharzade Sajad Dehghan Amin Nazari Amirsh.nll@gmail.com Introduction Problems clustering algorithms : Most clustering

633 views • 25 slides
Anonymization Algorithms - Microaggregation and Clustering Li Xiong CS573 Data Privacy and

Anonymization Algorithms - Microaggregation and Clustering Li Xiong CS573 Data Privacy and

Anonymization Algorithms - Microaggregation and Clustering Li Xiong CS573 Data Privacy and Anonymity Anonymization using Microaggregation or Clustering Practical Data-Oriented Microaggregation for Statistical Disclosure Control,

902 views • 55 slides
INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov & Stephan Oepen Language Technology Group University of Oslo Oct. 1, 2014 Last week Supervised vs unsupervised learning. Vectors space classification. How to

581 views • 41 slides
Clustering+Algorithms+ for+Streaming+and+Online+Se5ngs++ + + Claire+Monteleoni+

Clustering+Algorithms+ for+Streaming+and+Online+Se5ngs++ + + Claire+Monteleoni+

Clustering+Algorithms+ for+Streaming+and+Online+Se5ngs++ + + Claire+Monteleoni+ Computer)Science) George)Washington)University + + + Big+Data+Challenges+for+ML+ We+face+an+explosion+in+data!++ + Internet+transac@ons+ +DNA+sequencing ++

708 views • 51 slides

More recommend