Hierarchical Clustering 36-350: Data Mining 25 September 2006 Last - PowerPoint PPT Presentation

Apr 02, 2024 •260 likes •386 views

Hierarchical Clustering 36-350: Data Mining 25 September 2006 Last time... Unsupervised learning problems; finding clusters K means divide into k clusters to minimize within- cluser variance*cluster size local search, local

Hierarchical Clustering 36-350: Data Mining 25 September 2006
Last time... • Unsupervised learning problems; finding clusters • K means • divide into k clusters to minimize within- cluser variance*cluster size • local search, local minima
Limits of k-Means • Local search can get stuck • Random starts help • Sum-of-squares likes ball-shaped clusters • How to pick k? • No relations between clusters
Hierarchical Clustering • Basic idea: cluster the clusters • High-level clusters contain multiple low-level clusters • Clusters are now related • Don’t need to chose k • Assumes a hierarchy makes sense...
Ward’s Method 1. Start with every point in its own cluster 2. For each pair of clusters, calculate “merging cost” = increase in sum of squares 3. Merge least-costly pair 4. Stop when merging cost takes a big jump
Ward’s method applied ocean6 to the images from royalblue ocean1 royalblue ocean5 lecture 3: ocean, tigers, lightskyblue3 ocean4 azure3 flowers ocean2 darkslategray.2 ocean7 darkslategray.2 ocean3 midnightblue tiger2 Jump in merging cost gray10 tiger1 lightgoldenrod3 suggests 3 clusters - tiger4 darkseagreen4 tiger9 almost exactly right darkseagreen4 tiger8 antiquewhite2 tiger6 ones, too (but thinks gray10 darkseagreen4 flower5 flower5 is a tiger) tiger5 burlywood2 tiger3 gray36 3.0 tiger7 flower4 2.5 plum4 flower1 gray59.2 2.0 merging cost flower9 1.5 gray32 flower8 orchid3 flower7 1.0 orchid3 flower6 orchid3 0.5 flower3 darkmagenta 0.0 flower2 2 4 6 8 10 clusters
• Don’t have to chose k • Sum of squares is worse, generally, than k- means (for equal k) • more constrained search • prefers to merge small clusters, all else equal
Minimizing the mean distance from the center tends to make spheres, which can be silly k-Means Ward’s note how Ward’s is less balanced
Single-link clustering 1. Start with every point in its own cluster 2. Calculate gaps between every pair of clusters = distance between 2 closest points in each cluster 3. Merge clusters with smallest gap
k-Means Ward’s Single-link
Examples where single-link doesn’t work so well k-Means Ward’s Single-link

Recommend

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical Clustering Hierarchical Clustering Hierarchical Representations Hierarchical Algorithms Agglomeration "Nearest" Clusters Complexity

769 views • 49 slides

RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering

RECSM Summer School: Machine Learning for Social Sciences Session 3.4: Hierarchical Clustering Reto West Department of Political Science and International Relations University of Geneva 1 Clustering Clustering Hierarchical Clustering

237 views • 22 slides

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides

Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei

Hierarchical clustering David M. Blei COS424 Princeton University February 28, 2008 D. Blei Clustering 02 1 / 21 Hierarchical clustering Hierarchical clustering is a widely used data analysis tool. D. Blei Clustering 02 2 / 21

1.1k views • 83 slides

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides

Hierarchical Clustering Lecture 15 David Sontag New York

Hierarchical Clustering Lecture 15 David Sontag New York University Agglomerative Clustering Agglomerative clustering: First merge very similar instances Incrementally build larger clusters out

465 views • 7 slides

Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Plan Separation and convexity properties of hierarchical and non hierarchical clustering Patrice Bertrand 1 1 CEREMADE, Universit e Paris-Dauphine, Paris, France Joint work with Jean Diatta 2 2 LIM, Universit e de La R eunion,

401 views • 29 slides

Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods Hierarchical Methods 1 2 What is Clustering?

617 views • 24 slides

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview

10601 Machine Learning Hierarchical clustering Reading: Bishop: 9-9.2 Second half: Overview Clustering - Hierarchical, semi-supervised learning Graphical models - Bayesian networks, HMMs, Reasoning under uncertainty Putting it

862 views • 44 slides

Hierarchical)&)Spectral)clustering) Lecture)13) David&Sontag&

Hierarchical)&)Spectral)clustering) Lecture)13) David&Sontag& New&York&University& Slides adapted from Luke Zettlemoyer, Vibhav Gogate, Carlos Guestrin, Andrew Moore, Dan Klein Agglomerative Clustering Agglomerative

496 views • 25 slides

Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit e de

Geometric Data Analysis Hierarchical Clustering MAT 6480W / STT 6705V Guy Wolf guy.wolf@umontreal.ca Universit e de Montr eal Fall 2019 MAT 6480W (Guy Wolf) Hierarchical Clustering UdeM - Fall 2019 1 / 17 Outline Hierarchical

709 views • 56 slides

Clustering ! Hierarchical methods ! Model-based methods ! Density-based methods 1 2 What is

Preview ! Introduction Lecture 10 ! Partitioning methods Clustering ! Hierarchical methods ! Model-based methods ! Density-based methods 1 2 What is Clustering? Examples of Clustering Applications ! Cluster: a collection of data objects !

301 views • 8 slides

Agglomerative 2-3 Hierarchical Agglomerative 2-3 Hierarchical Clustering: theoretical

Agglomerative 2-3 Hierarchical Agglomerative 2-3 Hierarchical Clustering: theoretical Clustering: theoretical improvements and tests improvements and tests Sergiu Chelcea 1 , Patrice Bertrand , Patrice Bertrand 1,2 1,2 , Brigitte Trousse ,

651 views • 15 slides

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides

Hierarchical and Ensemble Clustering Ke Chen Reading: [7.8-7.10, EA], [25.5, KPM], [Fred &

Hierarchical and Ensemble Clustering Ke Chen Reading: [7.8-7.10, EA], [25.5, KPM], [Fred & Jain, 2005] COMP24111 Machine Learning Outline Introduction Cluster Distance Measures Agglomerative Algorithm Example and

268 views • 26 slides

Building an IT Taxonomy with Co- occurrence Analysis, Hierarchical Clustering, and

Building an IT Taxonomy with Co- occurrence Analysis, Hierarchical Clustering, and Multidimensional Scaling Chia-jung Tsui, Ping Wang , Kenneth R. Fleischmann, Asad B. Sayeed, Amy Weinberg, and Douglas Oard The abundance of IT is a challenges

203 views • 9 slides

Discovering Packet Structure through Lightweight Hierarchical Clustering Abdulrahman Hijazi

Understanding Network Traffic Approach Network Traffic Clustering: ADHIC Evaluation Discovering Packet Structure through Lightweight Hierarchical Clustering Abdulrahman Hijazi Hajime Inoue Ashraf Matrawy P .C. van Oorschot Anil Somayaji

540 views • 29 slides