conceptual clustering using lingo algorithm evaluation on
play

Conceptual Clustering Using Lingo Algorithm: Evaluation on Open - PowerPoint PPT Presentation

Mar 19, 2023 •478 likes •722 views

Conceptual Clustering Using Lingo Algorithm: Evaluation on Open Directory Project Data Stanis law Osi nski Dawid Weiss Institute of Computing Science Pozna n University of Technology May 20th, 2004 Some background: how to evaluate

  1. Conceptual Clustering Using Lingo Algorithm: Evaluation on Open Directory Project Data Stanis� law Osi´ nski Dawid Weiss Institute of Computing Science Pozna´ n University of Technology May 20th, 2004

  2. Some background: how to evaluate an SRC algorithm? About various goals of evaluation. . . Reconstruction of a predefined structure Test data: merge-then-cluster, manual labeling Measures: precision-recall, entropy measures. . . Labels “quality”, descriptiveness User surveys, click-distance methods

  3. Some background: how to evaluate an SRC algorithm? What types of “errors” can an algorithm make structure-wise? Misassignment errors (document → cluster) Missing documents in a cluster Incorrect clusters (unexplainable) Missing clusters (undetected) Granularity level confusion (subcluster domination problems)

  4. Evaluation of Lingo’s performance We tried to answer the following questions: Clusters’ structure: 1 Is Lingo able to cluster similar documents? 2 Is Lingo able to highlight outliers and “minorities”? 3 Is Lingo able to capture generalizations of closely-related subjects? 4 How does Lingo compare to Suffix Tree Clustering? Quality of cluster labels Are clusters labelled appropriately? Are they informative?

  5. Data set for the experiment Data set: a subset of the Open Directory Project Rationale: Human-created and maintained structure Human-created and maintained labels Descriptions resemble search results (snippets) Free availability

  6. ODP Categories chosen for the experiment MOVIES HEALTH CARE Ortho LRings BRunner PHOTOGRAPHY Infra COMPUTER SCIENCE DATABASES MISC. DWare MySQL JavaTut XMLDB Vi Postgr

  7. Test sets for the experiement Test sets Test sets were combinations of categories designed to help in answering the set of questions. Identifier Merged categories Test set rationale G1 LRings , MySQL Separation of two unrelated categories. G2 LRings , MySQL , Ortho Separation of three unrelated categories. G3 LRings , MySQL , Ortho , Separation of four unrelated categories, highligting Infra small topics ( Infra ). G4 MySQL , XMLDB , DWare , Separation of four conceptually close categories, all Postgr connected to database. G5 MySQL , XMLDB , DWare , Four conceptually very close categories (database) Postgr , JavaTut , Vi plus two distinct, but within the same abstract topic (computer science). G6 MySQL , XMLDB , DWare , Outlier highlight test – four dominating conceptually Postgr , Ortho close categories (databases) and one outlier ( Ortho ) G7 All categories All categories mixed together. Cross-topic cluster de- tection test (movies, databases).

  8. The experiment Lingo’s implementation → Carrot 2 framework The algorithm’s thresholds: Fixed at “good guess” values (same as those used in the on-line demo) Stemming and stop-word detection applied to the input data

  9. The results Method of analysis Manual investigation of document-to-cluster assignment charts. Helps understand the internal structure of results Prevents compensations inherent in aggregative measures

  10. → Is Lingo able to cluster similar documents? Categories-in-clusters view, input test: G3 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 30 25 20 15 10 5 0 MySQL News Information on Infrared Images Galleries Foot Orthotics Lord of the Rings Movie Orthopedic Products Humor Lotr Site Shoes Links Medical Database Support Stockings Middle Earth Manager lord of the rings mysql orthopedic infrared photography G1–G3: clear separation of topics, but with some extra clusters G1: granularity problem

  11. → Is Lingo able to cluster similar documents? Categories-in-clusters view, input test: G5 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 25 20 15 10 5 0 Java Tutorial Vim Page Federated Data Warehouse Native Xml Database Web Postgresql Database Mysql Server Free Links Development Tool Quick Reference Data Warehousing Mysql Client Object Oriented Postgres Driver java tutorials vi mysql data warehouses articles native xml databases postgres G5: misassignment problem

  12. → Is Lingo able to highlight outliers and “minorities”? Categories-in-clusters view, input test: G6 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 18 16 14 12 10 8 6 4 2 0 Mysql Database Federated Data Warehouse Foot Orthotics Orthopedic Products Access Postgresql Web Mysql Server Medical Shoes Designed Orthopaedic Postgres Mysql Client Data Warehousing Offers Software Development Tool Innovative mysql data warehouses articles native xml databases postgres orthopedic Ortho category (outlier), XMLDB consumed by MySQL!

  13. → Is Lingo able to highlight outliers and “minorities”? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres Infra category (outlier)

  14. → Is Lingo able to highlight outliers and “minorities”? Categories-in-clusters view, input test: G5 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 25 20 15 10 5 0 Java Tutorial Vim Page Federated Data Warehouse Native Xml Database Web Postgresql Database Mysql Server Free Links Development Tool Quick Reference Data Warehousing Mysql Client Object Oriented Postgres Driver java tutorials vi mysql data warehouses articles native xml databases postgres XMLDB category (outlier)

  15. → Is Lingo able to capture generalizations? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres “movie review” cluster is a generalization, but. . .

  16. → Is Lingo able to capture generalizations? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres Clusters are usually orthogonal with SVD, so no good results should be expected in this area.

  17. → How does Lingo compare to Suffix Tree Clustering? Categories-in-clusters view, input test: G7 Cluster assignment=0.250, Candidate cluster=0.775, top 16 clusters 50 45 40 35 30 25 20 15 10 5 0 Blade Runner Mysql Database Java Tutorial Lord of the Rings News Movie Review Information on Infrared Data Warehouse Images Galleries BBC Film Vim Macro Web Site Fan Fiction Fan Art Custom Orthotics Layout Management DBMS Online blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres

  18. → How does Lingo compare to Suffix Tree Clustering? Categories-in-clusters view, input test: G7 STC algorithm, Top 16 clusters 35 30 25 20 15 10 5 0 xml,native,native xml database includes blade runner,blade,runner information dm,dm review article,dm review used database ralph,article by ralph kimball,ralp [...] mysql articles written site cast dm review article by douglas hackne [...] review characters blade runner infrared photography java tutorials lord of the rings mysql orthopedic vi data warehouses articles native xml databases postgres

  19. Key differences between Lingo and STC Size-dominated clusters in STC Cluster labels much less informative Common-term clusters in STC

  20. Cluster labels quality Performed manually Problems: Single term labels usually ambiguous or too broad (“news”, “free”) Level of granularity usually unclear (need for hierarchical methods?)

  21. A word about analytical comparison methods. . . Can these conclusions be derived using formulas? We think so: cluster contamination measures might help.

  22. Online demo A nice form of evaluation (although scientifically doubtful), is the online demo’s popularity and feedback we get from users.

  24. http://carrot.cs.put.poznan.pl Thank you. Questions?

Recommend

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

905 views • 79 slides
1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 18 notes: K-means and Factor Analysis Tues, 4.17 1 K-means clustering The K-means clustering algorithm can be seen as

547 views • 3 slides
Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides
LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

DATA MINING LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

998 views • 78 slides
Week 7 Video 1 Clustering Clustering A type of Structure Discovery algorithm This type of

Week 7 Video 1 Clustering Clustering A type of Structure Discovery algorithm This type of

Week 7 Video 1 Clustering Clustering A type of Structure Discovery algorithm This type of method is also referred to as Dimensionality Reduction , based on a common application Clustering You have a large number of data points You

974 views • 63 slides
A fuzzy clustering method using Genetic Algorithm and Fuzzy Subtractive Clustering Thanh Le, Tom

A fuzzy clustering method using Genetic Algorithm and Fuzzy Subtractive Clustering Thanh Le, Tom

A fuzzy clustering method using Genetic Algorithm and Fuzzy Subtractive Clustering Thanh Le, Tom Altman and Katheleen Gardiner University of Colorado Denver July 18, 2012 Overview Introduction Fuzzy clustering using Fuzzy C-Means

345 views • 22 slides
Clustering 2 Clustering 2 Nov 3 2008 HAC Algorithm HAC Algorithm St t Start with all objects in

Clustering 2 Clustering 2 Nov 3 2008 HAC Algorithm HAC Algorithm St t Start with all objects in

Clustering 2 Clustering 2 Nov 3 2008 HAC Algorithm HAC Algorithm St t Start with all objects in their own cluster. ith ll bj t i th i l t Until there is only one cluster: Among the current clusters determine the two Among the current

345 views • 23 slides
CB-DBSCAN: A Novel Clustering Algorithm for Adjacent Clusters with Different Densities Gashin

CB-DBSCAN: A Novel Clustering Algorithm for Adjacent Clusters with Different Densities Gashin

CB-DBSCAN: A Novel Clustering Algorithm for Adjacent Clusters with Different Densities Gashin Ghazizadeh Outline Introduction Multi-density Clustering CB-DBSCAN Results CB-DBSCAN: A Novel Clustering Algorithm for Adjacent

454 views • 33 slides
K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW K-means Clustering Algorithm K-means++ Initialization Algorithm Experiment Datasets Conclusion K-MEANS CLUSTERING ALGORITHM A

313 views • 14 slides
A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with SSQ and the basic k-means algorithm 1.1 Discrete case 1.2 Continuous version 2. SSQ clustering for stratified survey sampling Dalenius (1950/51) 3.

587 views • 24 slides
Clustering on Graphs: The Markov Cluster Algorithm (MCL) CS 595D Presentation By Kathy Macropol

Clustering on Graphs: The Markov Cluster Algorithm (MCL) CS 595D Presentation By Kathy Macropol

Clustering on Graphs: The Markov Cluster Algorithm (MCL) CS 595D Presentation By Kathy Macropol MCL Algorithm Based on the PhD thesis by Stijn van Dongen Van Dongen, S. (2000) Graph Clustering by Flow Simulation . PhD Thesis, University of

1.04k views • 46 slides
Clustering Genetic Algorithm Petra Kudov Department of Theoretical Computer Science Institute

Clustering Genetic Algorithm Petra Kudov Department of Theoretical Computer Science Institute

Introduction Clustering Genetic Algorithm Experimental results Conclusion Clustering Genetic Algorithm Petra Kudov Department of Theoretical Computer Science Institute of Computer Science Academy of Sciences of the Czech Republic ETID

400 views • 18 slides
Clustering in Swedish The Impact of some Properties of the Swedish Language on Document

Clustering in Swedish The Impact of some Properties of the Swedish Language on Document

Introduction Clustering Evaluation Clustering in Swedish Clustering in Swedish The Impact of some Properties of the Swedish Language on Document Clustering and an Evaluation Method Magnus Rosell 2006-03-15 Magnus Rosell Clustering in

388 views • 28 slides
EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the

EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the

EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the EM algorithm? a) Agglomerative clustering b) Divisive clustering c) K-means clustering d) Growing neural gas EM algorithm E step:

445 views • 17 slides
k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering example Clustering example k -means clustering algorithm 1. Start with k randomly chosen means 2. Color data points by the shortest distance to any

1.56k views • 19 slides
Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering

Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering

Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering Clayton Greenberg, Asad Sayeed, and Vera Demberg Saarland University / M 2 CI Cluster of Excellence June 1, 2015 Clustering for thematic fit

918 views • 22 slides
TSX-V: LM | OTCQB: LMDCF APRIL 2018 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT REGARDING

TSX-V: LM | OTCQB: LMDCF APRIL 2018 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT REGARDING

TSX-V: LM | OTCQB: LMDCF APRIL 2018 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT REGARDING FORWARD LOOKING INFORMATION This presentation has been prepared by Lingo Media Corporation (Lingo Media or the "Company") exclusively

500 views • 22 slides
Clustering: k-means, the EM algorithm Based partly on: Dr. P Matuszek, Dr. Mooney:

Clustering: k-means, the EM algorithm Based partly on: Dr. P Matuszek, Dr. Mooney:

12/6/16 Clustering: k-means, the EM algorithm Based partly on: Dr. P Matuszek, Dr. Mooney: www.cs.utexas.edu/~mooney/cs388/slides/TextClustering.ppt Bookkeeping No HW 6 Phase II New eleusis.py, Adversary class Summary:

326 views • 16 slides
SCLOPE: An Algorithm for Clustering Data Streams of Categorical Attributes Kok-Leong Ong 1 ,

SCLOPE: An Algorithm for Clustering Data Streams of Categorical Attributes Kok-Leong Ong 1 ,

SCLOPE: An Algorithm for Clustering Data Streams of Categorical Attributes Kok-Leong Ong 1 , Wenyuan Li 2 , Wee-Keong Ng 2 , and Ee-Peng Lim 2 1 School of Information Technology, Deakin University Waurn Ponds, Victoria 3217, Australia

301 views • 16 slides
A Net-Reduction based Clustering Preprocessing Algorithm Jianhua Li, Laleh Behjat University of

A Net-Reduction based Clustering Preprocessing Algorithm Jianhua Li, Laleh Behjat University of

A Net-Reduction based Clustering Preprocessing Algorithm Jianhua Li, Laleh Behjat University of Calgary, Calgary, Canada ISPD April 12, 2006 A net reduction-based clustering preprocessing algorithm - ISPD 2006 Outline Introduction

771 views • 35 slides
Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and

Clustering Hierarchical clustering, k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein A quick review The clustering problem: partition genes into distinct sets with high homogeneity

772 views • 40 slides
TSX-V: LM OTC:LMDCF FSE:LIMA April 2020 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT

TSX-V: LM OTC:LMDCF FSE:LIMA April 2020 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT

TSX-V: LM OTC:LMDCF FSE:LIMA April 2020 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT REGARDING FORWARD LOOKING INFORMATION This presentation has been prepared by Lingo Media Corporation (Lingo Media or the "Company")

349 views • 17 slides
TSX-V: LM | OTCQB: LMDCF SEPTEMBER 2018 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT

TSX-V: LM | OTCQB: LMDCF SEPTEMBER 2018 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT

TSX-V: LM | OTCQB: LMDCF SEPTEMBER 2018 PROPRIETARY - LINGO MEDIA 1 CAUTIONARY STATEMENT REGARDING FORWARD LOOKING INFORMATION This presentation has been prepared by Lingo Media Corporation (Lingo Media or the "Company")

569 views • 23 slides

More recommend