Multi-variable Optimization K-means clustering K-means clustering - PowerPoint PPT Presentation

Mar 27, 2024 •276 likes •383 views

Multi-variable Optimization K-means clustering K-means clustering on points is finding K central locations that reduce the distance of each point to the nearest central location (summed over all points) K=3 K-means clustering For

Multi-variable Optimization
K-means clustering K-means clustering on points is finding K “central locations” that reduce the distance of each point to the nearest “central location” (summed over all points) K=3
K-means clustering For examples like the previous one, it is easy to find which points should be “grouped together” Once you have a group of points, you can mathematically find the best “central location” (“center of mass” with equally massive points)
K-means clustering Suppose you wanted to find the best spot to put 5 “central locations” here:
K-means clustering Suppose you wanted to find the best spot to put 5 “central locations” here: far from any center? dense? dense dense?? dense dense
K-means clustering Turns out you can do this the other way around as well... If you have the “central locations” (x,y) coordinates, you can find which location all points should go to (minimum distance)
Other random topics We have a problem: 1. If we knew point groupings, we could find the best central locations 2. If we knew central locations positions, we could find point groupings
Other random topics One common way to solve this issue when you have multiple unknowns that depend on each other is to simply guess, then try to optimize So, initially just make random groupings Then find the best central locations base off of the groupings Then find the best groupings... and repeat
Other random topics If you set up the problem correctly (and have a “well behaved” metric), this will converge In fact, you can do this even if you have more than two unknowns Just make one variable while fixing all others and optimize that one ... then pick a new variable to “optimize”
Other random topics This technique actually works in a large range of settings: K-means clustering (this) Bayesian networks (probabilistic reasoning) Markov Decision Processes (policy selection) Expectation–Maximization (parameter optimization)

Recommend

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 18 notes: K-means and Factor Analysis Tues, 4.17 1 K-means clustering The K-means clustering algorithm can be seen as

547 views • 3 slides

Clustering: Hierarchical Clustering and K- Means Clustering Machine

Clustering: Hierarchical Clustering and K- Means Clustering Machine Learning 10-601B Seyoung Kim Many of these slides are derived from William Cohen,

660 views • 46 slides

Clustering and Dimensionality Reduction Preview Clustering K -means clustering

Clustering and Dimensionality Reduction Preview Clustering K -means clustering Mixture models Hierarchical clustering Dimensionality reduction Principal component analysis Multidimensional scaling Isomap

1.61k views • 17 slides

k -means clustering Method to automatically separate data sets into distinct groups. Clustering

k -means clustering Method to automatically separate data sets into distinct groups. Clustering example Clustering example k -means clustering algorithm 1. Start with k randomly chosen means 2. Color data points by the shortest distance to any

1.56k views • 19 slides

Data Clustering: Data Clustering: 50 Years Beyond K means 50 Years Beyond K means 50 Years

Data Clustering: Data Clustering: 50 Years Beyond K means 50 Years Beyond K means 50 Years Beyond K-means 50 Years Beyond K-means Anil K. Jain Department of Computer Science Michigan State University King-Sun Fu King-Sun Fu King-Sun

1.03k views • 39 slides

Basics of k-means clustering CLUS TERIN G METH ODS W ITH S CIP Y Shaumik Daityari Business

Basics of k-means clustering CLUS TERIN G METH ODS W ITH S CIP Y Shaumik Daityari Business Analyst Why k-means clustering? A critical drawback of hierarchical clustering: runtime K means runs signicantly faster on large datasets

691 views • 25 slides

Contents Clustering K-means Mixture of Gaussians Expectation Maximization

Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 1 Introduction to Machine Learning CMU-10701 Clustering and EM Barnabs Pczos & Aarti Singh Clustering 3 K- means

1.04k views • 67 slides

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering

Introduction to Machine Learning, Clustering and EM Barnab s P czos Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K- means clustering What is clustering?

1.32k views • 66 slides

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means

Clustering kMeans, Expectation Maximization, Self-Organizing Maps Outline K-means clustering Hierarchical clustering Incremental clustering Probability-based clustering Self-Organising Maps Classification vs. Clustering

1k views • 67 slides

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering?

Lecture 23: Spectral clustering Hierarchical clustering What is a good clustering? Aykut Erdem May 2016 Hacettepe University Last time K-Means An iterative clustering algorithm - Initialize: Pick K random points as cluster

932 views • 64 slides

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW K-means Clustering Algorithm K-means++ Initialization Algorithm Experiment Datasets Conclusion K-MEANS CLUSTERING ALGORITHM A

313 views • 14 slides

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns

Graph Clustering Graph Clustering What is clustering? What is clustering? Finding patterns in data, or grouping similar groups of data-points together into clusters . Clustering algorithms for numeric data: Lloyds K-means, EM

688 views • 19 slides

K -means Clustering Ke Chen Reading: [7.3, EA], [9.1, CMB] COMP24111 Machine Learning Outline

K -means Clustering Ke Chen Reading: [7.3, EA], [9.1, CMB] COMP24111 Machine Learning Outline Introduction K -means Algorithm Example How K -means partitions? K -means Demo Relevant Issues Application:

416 views • 22 slides

EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the

EM, K-Means, GNG 4-6-16 Reading Quiz Which of the following can be considered an instance of the EM algorithm? a) Agglomerative clustering b) Divisive clustering c) K-means clustering d) Growing neural gas EM algorithm E step:

445 views • 17 slides

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed

Clustering Reference:http://www.elet.polimi.it/upload/matteucc/Clustering/tutorial_html/ Dr Ahmed Rafea Outline Introduction Clustering Algorithms K-means Fuzzy C-means Hierarchical clustering Introduction(1)

394 views • 23 slides

1-means clustering and conductance Twan van Laarhoven Radboud University Nijmegen Open

1-means clustering and conductance Twan van Laarhoven Radboud University Nijmegen Open Universiteit Nederland November 27th, 2017 1 / 30 Outline Network community detection with conductance The relation to k -means clustering Algorithms

706 views • 52 slides

RECSM Summer School: Machine Learning for Social Sciences Session 3.3: K -Means Clustering Reto

RECSM Summer School: Machine Learning for Social Sciences Session 3.3: K -Means Clustering Reto West Department of Political Science and International Relations University of Geneva 1 Clustering Clustering Clustering refers to a set of

352 views • 17 slides

Clustering (K-Means) Clustering Readings: Matt Gormley Murphy 25.5 Bishop 12.1,

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Clustering (K-Means) Clustering Readings: Matt Gormley Murphy 25.5 Bishop

560 views • 35 slides

Lecture: k-means & mean-shift clustering Juan Carlos Niebles and Ranjay Krishna Stanford

Clustering Lecture: k-means & mean-shift clustering Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 22-Oct-2019 1 St Stanfor ord University CS 131 Roadmap Clustering Pixels Segments Images Videos Web

769 views • 55 slides

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with SSQ and the basic k-means algorithm 1.1 Discrete case 1.2 Continuous version 2. SSQ clustering for stratified survey sampling Dalenius (1950/51) 3.

587 views • 24 slides

Introduction to Machine Learning CMU-10701 Clustering and EM Barnabs Pczos & Aarti Singh

Introduction to Machine Learning CMU-10701 Clustering and EM Barnabs Pczos & Aarti Singh Contents Clustering K-means Mixture of Gaussians Expectation Maximization Variational Methods 2 Clustering 3 K- means

1.01k views • 67 slides

Clustering with k-means Introduction to Machine Learning Clustering, what? Cluster :

INTRODUCTION TO MACHINE LEARNING Clustering with k-means Introduction to Machine Learning Clustering, what? Cluster : collection of objects Similar within cluster Dissimilar between clusters Clustering : grouping objects

776 views • 50 slides

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering

Finding Clusters Types of Clustering Approaches: Linkage Based, e.g. Hierarchical Clustering Clustering by Partitioning, e.g. k-Means Density Based Clustering, e.g. DBScan Grid Based Clustering Compendium slides for Guide to Intelligent

1.32k views • 82 slides