clustering
play

Clustering Data Mining: Concepts and October 18, 2019 Techniques - PowerPoint PPT Presentation

Feb 02, 2024 •563 likes •893 views

Clustering Data Mining: Concepts and October 18, 2019 Techniques 1 Chapter 8. Cluster Analysis What is Cluster Analysis? Types of Data in Cluster Analysis A Categorization of Major Clustering Methods Partitioning Methods

  1. Clustering Data Mining: Concepts and October 18, 2019 Techniques 1

  2. Chapter 8. Cluster Analysis  What is Cluster Analysis?  Types of Data in Cluster Analysis  A Categorization of Major Clustering Methods  Partitioning Methods  Hierarchical Methods  Density-Based Methods  Grid-Based Methods  Model-Based Clustering Methods  Outlier Analysis  Summary October 18, 2019 Data Mining: Concepts and Techniques 2

  3. What is Cluster Analysis?  Cluster: a collection of data objects  Similar to one another within the same cluster  Dissimilar to the objects in other clusters  Cluster analysis  Grouping a set of data objects into clusters  Clustering is unsupervised classification: no predefined classes  Typical applications  As a stand-alone tool to get insight into data distribution  As a preprocessing step for other algorithms

  4. General Applications of Clustering  Pattern Recognition  Spatial Data Analysis  create thematic maps in GIS by clustering feature spaces  detect spatial clusters and explain them in spatial data mining  Image Processing  Economic Science (especially market research)  WWW  Document classification  Cluster Weblog data to discover groups of similar access patterns October 18, 2019 Data Mining: Concepts and Techniques 4

  5. Examples of Clustering Applications  Marketing: Help marketers discover distinct groups in their customer bases, and then use this knowledge to develop targeted marketing programs  Land use: Identification of areas of similar land use in an earth observation database  Insurance: Identifying groups of motor insurance policy holders with a high average claim cost  City-planning: Identifying groups of houses according to their house type, value, and geographical location  Earth-quake studies: Observed earth quake epicenters should be clustered along continent faults October 18, 2019 Data Mining: Concepts and Techniques 5

  6. What Is Good Clustering?  A good clustering method will produce high quality clusters with  high intra-class similarity  low inter-class similarity  The quality of a clustering result depends on both the similarity measure used by the method and its implementation.  The quality of a clustering method is also measured by its ability to discover some or all of the hidden patterns. October 18, 2019 Data Mining: Concepts and Techniques 6

  7. Requirements of Clustering in Data Mining  Scalability  Ability to deal with different types of attributes  Discovery of clusters with arbitrary shape  Minimal requirements for domain knowledge to determine input parameters  Able to deal with noise and outliers  Insensitive to order of input records  High dimensionality  Interpretability and usability October 18, 2019 Data Mining: Concepts and Techniques 7

  8. Chapter 8. Cluster Analysis  What is Cluster Analysis?  Types of Data in Cluster Analysis  A Categorization of Major Clustering Methods  Partitioning Methods  Hierarchical Methods  Density-Based Methods  Grid-Based Methods  Model-Based Clustering Methods  Outlier Analysis  Summary October 18, 2019 Data Mining: Concepts and Techniques 8

  9. Data Structures   x ... x ... x 11 1f 1p    Data matrix ... ... ... ... ...     x ... x ... x i1 if ip    ... ... ... ... ...    x ... x ... x   n1 nf np     0    Dissimilarity matrix d(2,1) 0     d(3,1 ) d ( 3 , 2 ) 0   : : :     d ( n , 1 ) d ( n , 2 ) ... ... 0   October 18, 2019 Data Mining: Concepts and Techniques 9

  10. Type of data in clustering analysis  Interval-scaled variables:  Binary variables:  Nominal, ordinal, and ratio variables:  Variables of mixed types: October 18, 2019 Data Mining: Concepts and Techniques 10

  11. Interval-valued variables  Standardize data  Calculate the mean absolute deviation: 1        s (| x m | | x m | ... | x m |) n f 1 f f 2 f f nf f where 1   m (x x x   ... ) n . f 1 f 2 f nf  Calculate the standardized measurement ( z-score )  x m  if f z s if f  Using mean absolute deviation is more robust than using standard deviation October 18, 2019 Data Mining: Concepts and Techniques 11

  12. Similarity and Dissimilarity Between Objects  Distances are normally used to measure the similarity or dissimilarity between two data objects  Some popular ones include: Minkowski distance :   q   q    q d ( i , j ) (| x x | | x x | ... | x x | ) q i j i j i j 1 1 2 2 p p where i = ( x i1 , x i2 , …, x ip ) and j = ( x j1 , x j2 , …, x jp ) are two p -dimensional data objects, and q is a positive integer  If q = 1 , d is Manhattan distance        d ( i , j ) | x x | | x x | ... | x x | i j i j i j 1 1 2 2 p p October 18, 2019 Data Mining: Concepts and Techniques 12

  13. Similarity and Dissimilarity Between Objects (Cont.)  If q = 2 , d is Euclidean distance:        2 2 2 d ( i , j ) (| x x | | x x | ... | x x | ) i j i j i j 1 1 2 2 p p  Properties  d(i,j)  0  d(i,i) = 0  d(i,j) = d(j,i)  d(i,j)  d(i,k) + d(k,j)  Also one can use weighted distance, parametric Pearson product moment correlation, or other disimilarity measures. October 18, 2019 Data Mining: Concepts and Techniques 13

  14. Binary Variables  A contingency table for binary data Object j 1 0 sum  1 a b a b  0 c d c d Object i   sum a c b d p  Simple matching coefficient (invariant, if the binary  variable is symmetric ): b c  d ( i , j )    a b c d  Jaccard coefficient (noninvariant if the binary variable is asymmetric ):  b c  d ( i , j )   a b c October 18, 2019 Data Mining: Concepts and Techniques 14

  15. Dissimilarity between Binary Variables  Example Name Gender Fever Cough Test-1 Test-2 Test-3 Test-4 Jack M Y N P N N N Mary F Y N P N P N Jim M Y P N N N N  gender is a symmetric attribute  the remaining attributes are asymmetric binary  let the values Y and P be set to 1, and the value N be set to 0  0 1   d ( jack , mary ) 0 . 33   2 0 1  1 1   d ( jack , jim ) 0 . 67   1 1 1  1 2   d ( jim , mary ) 0 . 75   1 1 2 October 18, 2019 Data Mining: Concepts and Techniques 15

  16. Nominal Variables  A generalization of the binary variable in that it can take more than 2 states, e.g., red, yellow, blue, green  Method 1: Simple matching  m : # of matches, p : total # of variables  p m  d ( i , j ) p  Method 2: use a large number of binary variables  creating a new binary variable for each of the M nominal states October 18, 2019 Data Mining: Concepts and Techniques 16

  17. Ordinal Variables  An ordinal variable can be discrete or continuous  order is important, e.g., rank  Can be treated like interval-scaled  r { 1 ,..., M }  replacing x if by their rank if f  map the range of each variable onto [0, 1] by replacing i -th object in the f -th variable by  r 1  if z  M 1 if f  compute the dissimilarity using methods for interval- scaled variables October 18, 2019 Data Mining: Concepts and Techniques 17

  18. Chapter 8. Cluster Analysis  What is Cluster Analysis?  Types of Data in Cluster Analysis  A Categorization of Major Clustering Methods  Partitioning Methods  Hierarchical Methods  Density-Based Methods  Grid-Based Methods  Model-Based Clustering Methods  Outlier Analysis  Summary October 18, 2019 Data Mining: Concepts and Techniques 18

  19. Ratio-Scaled Variables  Ratio-scaled variable: a positive measurement on a nonlinear scale, approximately at exponential scale, such as Ae Bt or Ae -Bt  Methods:  treat them like interval-scaled variables — not a good choice! (why?)  apply logarithmic transformation y if = log(x if )  treat them as continuous ordinal data treat their rank as interval-scaled. October 18, 2019 Data Mining: Concepts and Techniques 19

  20. Variables of Mixed Types  A database may contain all the six types of variables  symmetric binary, asymmetric binary, nominal, ordinal, interval and ratio.  One may use a weighted formula to combine their effects.   d p ( f ) ( f )   f 1 ij ij d ( i , j )   p ( f )  f 1 ij  f is binary or nominal: (f) = 0 if x if = x jf , or d ij (f) = 1 o.w. d ij  f is interval-based: use the normalized distance  f is ordinal or ratio-scaled r  1 z if  compute ranks r if and if  M  1 f  and treat z if as interval-scaled October 18, 2019 Data Mining: Concepts and Techniques 20

Recommend

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides
Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics

Clustering Hierarchical clustering and k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Many

768 views • 43 slides
Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find

Clustering DSE 210 Clustering in R d Two common uses of clustering: Vector quantization Find a finite set of representatives that provides good coverage of a complex, possibly infinite, high-dimensional space. Finding meaningful

396 views • 36 slides
Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides
Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides
Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Clustering A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods 1 2 What is Clustering? What is Clustering? Typical

647 views • 18 slides
Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative

LUDWIG- MAXIMILIANS- INSTITUTE DATABASE UNIVERSITT FOR SYSTEMS MNCHEN MNCHEN INFORMATICS INFORMATICS GROUP GROUP Subspace Clustering Ensemble Clustering Subspace Clustering, Ensemble Clustering, Alternative Clustering, Multiview

513 views • 9 slides
Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides
Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K- means clustering What is clustering?

1.32k views • 66 slides
PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin

PAC-Bayesian Analysis of Co-clustering, Graph Clustering and Pairwise Clustering Yevgeny Seldin Motivation Example Clustering cannot be analyzed without specifying what it will be used for! be used for! Example Cluster then pack

293 views • 28 slides
Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means clustering Hierarchical clustering Incremental clustering Probability-based clustering Self-Organising Maps Classification vs. Clustering

1k views • 67 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as

Graph Clustering Why graph clustering is useful? Distance matrices are graphs as useful as any other clustering Identification of communities in social networks Webpage clustering for better data management of web data Outline

607 views • 41 slides
Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean

Clustering: Models and Algorithms Shikui Tu 2019-02-28 1 Outline Clustering K-mean clustering, hierarchical clustering Adaptive learning (online learning) CL, FSCL, RPCL Gaussian Mixture Models (GMM)

805 views • 34 slides
Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document

Web Information Retrieval Lecture 15 Clustering Todays Topic: Clustering Document clustering Motivations Document representations Success criteria Clustering algorithms Partitional Hierarchical Ch. 16 What is

766 views • 61 slides
LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Clustering (unsupervised) vs.

526 views • 30 slides
Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review

Clustering k-mean clustering Genome 373 Genomic Informatics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity and high separation Clustering (unsupervised) vs.

356 views • 31 slides
Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic

Outline Linkage-based Clustering Motivation Definitions Clustering with Semantic Links Applications Algorithms SimRank Yin X., Han J. and Yu P. S., 2006, LinkClus: Efficient ReCoM Clustering via Heterogeneous

221 views • 8 slides
Clustering in Go May 2016 Wilfried Schobeiri MediaMath

Clustering in Go May 2016 Wilfried Schobeiri MediaMath

5/26/2016 Clustering in Go Clustering in Go May 2016 Wilfried Schobeiri MediaMath http://127.0.0.1:3999/clustering-in-go.slide#1 1/42 5/26/2016 Clustering in Go Who am I? Go enthusiast These days, mostly codes for fun Focused on

917 views • 42 slides
LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

DATA MINING LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

998 views • 78 slides
Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods

Clustering What is Clustering? Types of Data in Cluster Analysis Cl Clustering t i A Categorization of Major Clustering Methods Partitioning Methods Hierarchical Methods Hierarchical Methods 1 2 What is Clustering?

617 views • 24 slides

More recommend