evaluation of hierarchical clustering algorithms for
play

Evaluation of Hierarchical Clustering Algorithms for Document - PowerPoint PPT Presentation

Jan 10, 2024 •301 likes •974 views

Evaluation of Hierarchical Clustering Algorithms for Document Datasets Paper by Ying Zhao and George Karypis University of Minnesota (2002) CS 6501 Paper Presentation - April 6, 2016 Matthew Hawthorn, Nikhil Mascarenhas, Shannon Mitchell

  1. Evaluation of Hierarchical Clustering Algorithms for Document Datasets Paper by Ying Zhao and George Karypis University of Minnesota (2002) CS 6501 Paper Presentation - April 6, 2016 Matthew Hawthorn, Nikhil Mascarenhas, Shannon Mitchell

  2. Motivation ● Hierarchical clustering of documents ○ Intuitive, clustering of different levels of granularity. ● Two major approaches ○ Partitional ○ Agglomerative ● General view was that partitional algorithms are inferior ● Authors ran an experiment to compare these approaches. ● Defined a new algorithm, a hybrid “constrained agglomerative algorithm”

  3. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves

  4. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  5. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  6. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  7. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  8. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  9. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  10. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  11. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  12. Hierarchical Clustering: Partitional Algorithms Top-down - Start with one cluster with all documents - Start at root, divide down to leaves - Split the cluster which most improves the criterion function - Complexity: O (n log n)

  13. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  14. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  15. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  16. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  17. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  18. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  19. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  20. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  21. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  22. Hierarchical Clustering: Agglomerative Algorithms Bottom-up - Each document starts as its own cluster - Start at leaves, merge to root - Complexity: O (n 2 log n) When caching of intermediate values of the objective is possible O(n 3 ) Otherwise

  23. Criterion Functions Global criterion functions drive the clustering process. Graph Based Internal Functions External Functions Hybrid Functions Functions Considers only Considers how Simultaneously Constructs a graph documents within various clusters are consider internal which represents the a cluster different from each and external relationships other. criterion functions between documents.

  24. m Number of terms n Number of documents Internal Criterion Functions k Number of clusters S 1 , S 2 ,... S k Each one of k clusters n 1 , n 2 ,…. n k Size of each cluster d 1 , d 2 , …. d n Tf idf vector for a document D A Sum of all vectors in cluster A C A Centroid vector of cluster A

  25. m Number of terms n Number of documents External Criterion Functions k Number of clusters S 1 , S 2 ,... S k Each one of k clusters n 1 , n 2 ,…. n k Size of each cluster d 1 , d 2 , …. d n Tf idf vector for a document D A Sum of all vectors in cluster A C A Centroid vector of cluster A

  26. Traditional Agglomerative Clustering Criteria Single-linkage Group average Complete-linkage minimum distance average of distances maximum distance Authors’ abbreviation: ‘slink’ ‘UPGMA’ ‘clink’

  27. Hierarchical Clustering: Constrained Agglomerative ● Hybrid technique ● Constrains agglomerative clustering by initializing with intermediate hierarchical partitional clustering ● More likely to avoid early merge mistakes of agglomerative techniques ● But takes advantage of the ease with which agglomerative techniques find small and cohesive clusters

  28. Hierarchical Clustering: Constrained Agglomerative 1. Find k clusters using partitional clustering

  29. Hierarchical Clustering: Constrained Agglomerative 1. Find k clusters using partitional clustering

  30. Hierarchical Clustering: Constrained Agglomerative 1. Find k clusters using partitional clustering

  31. Hierarchical Clustering: Constrained Agglomerative 1. Find k clusters using partitional clustering

  32. Hierarchical Clustering: Constrained Agglomerative 1. Find k clusters using partitional clustering

  33. Hierarchical Clustering: Constrained Agglomerative 2. Cluster the documents in these clusters using agglomerative clustering

  34. Hierarchical Clustering: Constrained Agglomerative 2. Cluster the documents in these clusters using agglomerative clustering

  35. Hierarchical Clustering: Constrained Agglomerative 2. Cluster the documents in these clusters using agglomerative clustering

  36. Hierarchical Clustering: Constrained Agglomerative 2. Cluster the documents in these clusters using agglomerative clustering

  37. Hierarchical Clustering: Constrained Agglomerative 2. Cluster the documents in these clusters using agglomerative clustering

  38. Hierarchical Clustering: Constrained Agglomerative 2. Cluster the documents in these clusters using agglomerative clustering

  39. Hierarchical Clustering: Constrained Agglomerative 3. Cluster the k clusters using agglomerative clustering

  40. Hierarchical Clustering: Constrained Agglomerative 3. Cluster the k clusters using agglomerative clustering

  41. Hierarchical Clustering: Constrained Agglomerative 3. Cluster the k clusters using agglomerative clustering

  42. Hierarchical Clustering: Constrained Agglomerative 3. Cluster the k clusters using agglomerative clustering

  43. Hierarchical Clustering: Constrained Agglomerative 3. Cluster the k clusters using agglomerative clustering

  44. Computational Complexity ● Partitional clustering of data into k clusters: < O(n log(n)) (the cost of an entire partitional clustering) log(n) levels O(n) comparison and reassignment operations at each level

Recommend

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal &amp; Stephan

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal &amp; Stephan

INF4820 Algorithms for AI and NLP Hierarchical Clustering Erik Velldal &amp; Stephan Oepen Language Technology Group (LTG) October 7, 2015 Agenda Last week Evaluation of classifiers Machine learning for class discovery:

892 views • 38 slides
Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical Clustering Hierarchical Clustering Hierarchical Representations Hierarchical Algorithms Agglomeration &quot;Nearest&quot; Clusters Complexity

769 views • 49 slides
LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

933 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov &amp; Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov &amp; Stephan Oepen Language

INF4820: Algorithms for AI and NLP Clustering Milen Kouylekov &amp; Stephan Oepen Language Technology Group University of Oslo Oct. 2, 2014 Agenda Yesterday Flat clustering k -Means Today Bottom-up hierarchical clustering.

976 views • 44 slides
RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering

RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering

RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering Reto West Department of Political Science and International Relations University of Geneva 1 Clustering Clustering Hierarchical Clustering

237 views • 22 slides
Discovering Packet Structure through Lightweight Hierarchical Clustering Abdulrahman Hijazi

Discovering Packet Structure through Lightweight Hierarchical Clustering Abdulrahman Hijazi

Understanding Network Traffic Approach Network Traffic Clustering: ADHIC Evaluation Discovering Packet Structure through Lightweight Hierarchical Clustering Abdulrahman Hijazi Hajime Inoue Ashraf Matrawy P .C. van Oorschot Anil Somayaji

540 views • 29 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei

Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei

Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei Clustering 02 1 / 21 Hierarchical clustering Hierarchical clustering is a widely used data analysis tool. D. Blei Clustering 02 2 / 21

1.1k views • 83 slides
Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document clustering Motivations Document representations Success criteria Clustering algorithms Partitional Hierarchical Ch. 16 What is

767 views • 61 slides
Generalization of Hierarchical Crisp Clustering Algorithms to Fuzzy Logic Mathias Bank Dr.

Generalization of Hierarchical Crisp Clustering Algorithms to Fuzzy Logic Mathias Bank Dr.

Generalization of Hierarchical Crisp Clustering Algorithms to Fuzzy Logic Mathias Bank Dr. Friedhelm Schwenker mathias.bank@uni-ulm.de friedhelm.schwenker@uni-ulm.de Faculty for Mathematics and Economics Institute of Neural Information

594 views • 21 slides
Hierarchical Clustering Lecture 15 David Sontag New York

Hierarchical Clustering Lecture 15 David Sontag New York

Hierarchical Clustering Lecture 15 David Sontag New York University Agglomerative Clustering Agglomerative clustering: First merge very similar instances Incrementally build larger clusters out

465 views • 7 slides
Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Plan Separation and convexity properties of hierarchical and non hierarchical clustering Patrice Bertrand 1 1 CEREMADE, Universit e Paris-Dauphine, Paris, France Joint work with Jean Diatta 2 2 LIM, Universit e de La R eunion,

401 views • 29 slides
Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods Hierarchical Methods 1 2 What is Clustering?

617 views • 24 slides
10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under uncertainty Putting it

862 views • 44 slides
Hierarchical)&amp;)Spectral)clustering) Lecture)13) David&amp;Sontag&amp;

Hierarchical)&amp;)Spectral)clustering) Lecture)13) David&amp;Sontag&amp;

Hierarchical)&amp;)Spectral)clustering) Lecture)13) David&amp;Sontag&amp; New&amp;York&amp;University&amp; Slides adapted from Luke Zettlemoyer, Vibhav Gogate, Carlos Guestrin, Andrew Moore, Dan Klein Agglomerative Clustering Agglomerative

496 views • 25 slides
Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed Rafea Outline Introduction Clustering Algorithms K-means Fuzzy C-means Hierarchical clustering Introduction(1)

394 views • 23 slides
Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit e de

Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit e de

Geometric Data Analysis Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit e de Montr eal Fall 2019 MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 1 / 17 Outline Hierarchical

709 views • 56 slides

More recommend