a better k means algorithm via local search
play

A better k-means++ Algorithm via Local Search Silvio Lattanzi - PowerPoint PPT Presentation

Sep 20, 2022 •567 likes •670 views

A better k-means++ Algorithm via Local Search Silvio Lattanzi Christian Sohler Google Research Google Research ICML 2019 k-means Find a set of k centers X c C d 2 ( x, c ) ( X, C ) = min x X Constant approximation algorithms

  1. A better k-means++ Algorithm via Local Search Silvio Lattanzi Christian Sohler 
 Google Research Google Research ICML 2019

  2. k-means Find a set of k centers X c ∈ C d 2 ( x, c ) φ ( X, C ) = min x ∈ X Constant approximation algorithms are known. Goal is to design a constant approximation algorithm that is efficient, easy to implement and has good experimental results. A better k-means++ Algorithm via Local Search

  3. <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> <latexit sha1_base64="Qtv+Xu/NZDL6vUHV4Ro0OEjlzA=">AB8HicdVDLSgMxFM3UV62vqks3wSLUzZDpWNtl0Y07K9iHtEPJpJk2NDMZkoxQhn6FGxeKuPVz3Pk3ptMKnrgwuGce7n3Hj/mTGmEPqzcyura+kZ+s7C1vbO7V9w/aCuRSEJbRHAhuz5WlLOItjTnHZjSXHoc9rxJ5dzv3NPpWIiutXTmHohHkUsYARrI91dl/tcjODkdFAsIdtFbrWOYEbcWiUjteq5Cx0bZSiBJZqD4nt/KEgS0kgTjpXqOSjWXoqlZoTWaGfKBpjMsEj2jM0wiFVXpodPIMnRhnCQEhTkYaZ+n0ixaFS09A3nSHWY/Xbm4t/eb1EB3UvZVGcaBqRxaIg4VALOP8eDpmkRPOpIZhIZm6FZIwlJtpkVDAhfH0K/yftiu0g27k5KzUulnHkwRE4BmXgBpogCvQBC1AQAgewBN4tqT1aL1Yr4vWnLWcOQ/YL19AvQWj9o=</latexit> k-means++ seeding Elegant and simple algorithm Experimentally gives good results when combined with Lloyd’s algorithm. The solution is a approximation in expectation. O (log k ) David Arthur, Sergei Vassilvitskii: k-means++: the advantages of careful seeding. SODA 2007: 1027-1035 A better k-means++ Algorithm via Local Search

  4. Local search Elegant and simple algorithm It returns a constant approximation and nice experimental results. The algorithm is a bit slow. Tapas Kanungo, David M. Mount, Nathan S. Netanyahu, Christine D. Piatko, Ruth Silverman, Angela Y. Wu: 
 A local search approximation algorithm for k-means clustering. Comput. Geom. 28(2-3): 89-112 (2004) A better k-means++ Algorithm via Local Search

  5. Combining the two algorithms Elegant and simple algorithm It returns a constant approximation, it is slightly slower than k-means++ and has better experimental results. A better k-means++ Algorithm via Local Search

  6. <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> <latexit sha1_base64="CuakJu/HyZqru1PUc0cdh0lQwu4=">ACBnicbVBNS8NAEN34WetX1KMIwSLUgyUrgh6LXjxWsB/QhLZbtqlm03YnQgl5OTFv+LFgyJe/Q3e/Ddu2xy09cHA470ZuYFieAaXPfbWlpeWV1bL2UN7e2d3btvf2WjlNFWZPGIladgGgmuGRN4CBYJ1GMRIFg7WB0M/HbD0xpHst7GCfMj8hA8pBTAkbq2UeYCFU8ZkXKkIznGfYdUe5p/hgCKc9u+LW3CmcRYILUkEFGj37y+vHNI2YBCqI1l3sJuBnRAGnguVlL9UsIXREBqxrqCQR0342fSN3TozSd8JYmZLgTNXfExmJtB5HgemMCAz1vDcR/O6KYRXfsZlkgKTdLYoTIUDsTPJxOlzxSiIsSGEKm5udeiQmDzAJFc2IeD5lxdJ67yG3Rq+u6jUr4s4SugQHaMqwugS1dEtaqAmougRPaNX9GY9WS/Wu/Uxa12yipkD9AfW5w/Vw5gS</latexit> Main theoretical result Main idea is to adapt local search analysis to show that in every step with constant probability we reduce the cost of the solution by a ✓ ◆ 1 multiplicative factor 1 − 100 k A better k-means++ Algorithm via Local Search

  7. Experimental results Datasets: - RNA : 8 features from 488565 RNA input sequence pairs (Uzilov et al., 2006) - KDD-BIO : 145751 samples with 74 features measuring the match between a protein and a native sequence (KDD) - KDD-PHY : 100000 samples with 78 features representing a quantum physic task (KDD) A better k-means++ Algorithm via Local Search

  8. Experimental results KDD-BIO RNA KDD-PHY A better k-means++ Algorithm via Local Search

  9. Thanks A better k-means++ Algorithm via Local Search

Recommend

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

1 K-means clustering The K-means clustering algorithm can be seen as applying the EM algorithm to

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 18 notes: K-means and Factor Analysis Tues, 4.17 1 K-means clustering The K-means clustering algorithm can be seen as

547 views • 3 slides
Stochastic Local Search Methods Dynamic Local Search Iterated Local Search Tabu Search Marco

Stochastic Local Search Methods Dynamic Local Search Iterated Local Search Tabu Search Marco

Outline DM811 HEURISTICS AND LOCAL SEARCH ALGORITHMS FOR COMBINATORIAL OPTIMZATION 1. Stochastic Local Search Methods (Metaheuristics) Randomized Iterative Improvement Lecture 12 Attribute Based Hill Climber Stochastic Local Search Methods

590 views • 9 slides
Accelerating a local search algorithm for large instances of the independent task scheduling

Accelerating a local search algorithm for large instances of the independent task scheduling

Motivation Initial algorithm Adaptation Large instances Conclusion Accelerating a local search algorithm for large instances of the independent task scheduling problem with the GPU Frederic Pinel Johnatan Pecero Pascal Bouvry Computer

367 views • 18 slides
Local Search Distances Landscape Characteristics Fitness-Distance Correlation Ruggedness Marco

Local Search Distances Landscape Characteristics Fitness-Distance Correlation Ruggedness Marco

Outline DM811 1. Local Search, Basic Elements HEURISTICS AND LOCAL SEARCH ALGORITHMS Components and Algorithms FOR COMBINATORIAL OPTIMZATION Beyond Local Optima Computational Complexity 2. Fundamental Search Space Properties Introduction

698 views • 20 slides
K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW

K-MEANS++ OPTIMAL INITIALIZATION ALGORITHM An Improved K-means Clustering Method OVERVIEW K-means Clustering Algorithm K-means++ Initialization Algorithm Experiment Datasets Conclusion K-MEANS CLUSTERING ALGORITHM A

313 views • 14 slides
A-Star Algorithm &amp; Heaps/Priority Queues Mark Redekopp 2 A* Search Algorithm ALGORITHM

A-Star Algorithm &amp; Heaps/Priority Queues Mark Redekopp 2 A* Search Algorithm ALGORITHM

1 EE 355 Unit 14 A-Star Algorithm &amp; Heaps/Priority Queues Mark Redekopp 2 A* Search Algorithm ALGORITHM HIGHLIGHT 3 Search Methods Many systems require searching for goal states Path Planning Roomba Vacuum

887 views • 40 slides
Heuristic (Informed) search strategy Search Algorithm #2 Search SEARCH#2 1.

Heuristic (Informed) search strategy Search Algorithm #2 Search SEARCH#2 1.

Recall that the ordering of FRINGE defines the Heuristic (Informed) search strategy Search Algorithm #2 Search SEARCH#2 1. INSERT(initial-node,FRINGE) (Where we try to choose smartly) (Wh t t h tl ) 2. Repeat: a. If empty(FRINGE) then

393 views • 16 slides
Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 utzle 2

Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 utzle 2

Automatic Algorithm Configuration based on Local Search Frank Hutter 1 Holger Hoos 1 utzle 2 Thomas St 1 Department of Computer Science University of British Columbia Canada 2 IRIDIA Universit e Libre de Bruxelles Belgium Real-world

835 views • 59 slides
A Decomposition-based Local Search Algorithm for Multi-objective Sequence Dependent Setup Times

A Decomposition-based Local Search Algorithm for Multi-objective Sequence Dependent Setup Times

Basic concepts MOLS/D Experimental Study Final Considerations IEEE CEC 2018 A Decomposition-based Local Search Algorithm for Multi-objective Sequence Dependent Setup Times Permutation Flowshop Scheduling Murilo Zangari, Ademir Constantino,

297 views • 17 slides
Metaheuristics 2.3 Local Search 2.4 Simulated annealing Adrian Horga 1 2.3 Local Search 2

Metaheuristics 2.3 Local Search 2.4 Simulated annealing Adrian Horga 1 2.3 Local Search 2

Metaheuristics 2.3 Local Search 2.4 Simulated annealing Adrian Horga 1 2.3 Local Search 2 Local Search Other names: Hill climbing Descent Iterative improvement General S-Metaheuristics Old and simple method at each

620 views • 25 slides
Local Search Toolbox so far Uninformed search BFS, DFS,

Local Search Toolbox so far Uninformed search BFS, DFS,

Local Search Toolbox so far Uninformed search BFS, DFS, Itera;ve deepening DFS, Uniform cost search Heuris;c search A* Review Search:

440 views • 19 slides
k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if

k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if

k -means++ seeding Have seen that the k -means algorithm can output arbitrarily poor solutions, if started with a bad set of initial centroids k -means++ is a simple, probabilistic algorithm to compute initial centroids These centroids are

745 views • 8 slides
IDWalk : a local search algorithm combining intensification and diversification Bertrand Neveu 1 ,

IDWalk : a local search algorithm combining intensification and diversification Bertrand Neveu 1 ,

1 IDWalk : a local search algorithm combining intensification and diversification Bertrand Neveu 1 , Gilles Trombettoni 1 , Fred Glover 2 1 Equipe COPRIN CERTIS-I3S-INRIA, Sophia Antipolis, France neveu@sophia.inria.fr trombe@sophia.inria.fr 2

259 views • 21 slides
Uninformed Search (Ch. 3-3.4) 2 Announcements HW due tonight Writing 1 due next week 3 Search

Uninformed Search (Ch. 3-3.4) 2 Announcements HW due tonight Writing 1 due next week 3 Search

1 Uninformed Search (Ch. 3-3.4) 2 Announcements HW due tonight Writing 1 due next week 3 Search algorithm To search, we will build a tree with the root as the initial state Any problems with this? 6 Search algorithm We can remove

400 views • 22 slides
Local search algorithms AIMA sections 4.1,4.2 Summary Local search algorithms Hill-climbing

Local search algorithms AIMA sections 4.1,4.2 Summary Local search algorithms Hill-climbing

Local search algorithms Local search algorithms AIMA sections 4.1,4.2 Summary Local search algorithms Hill-climbing Simulated annealing Genetic algorithms (briefly) Local search in continuous spaces (very briefly) Iterative

300 views • 16 slides
X Example? In such cases, we can use local search algorithms Keep a single

X Example? In such cases, we can use local search algorithms Keep a single

Todays Class Local Search Iterative improvement methods AI Class 6 (Ch. 4.1-4.2) Hill climbing If the path to the goal Simulated annealing does not matter [we Local beam search can use] a single current node and move to

236 views • 5 slides
A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with

A history of the k-means algorithm Hans-Hermann Bock, RWTH Aachen, Allemagne 1. Clustering with SSQ and the basic k-means algorithm 1.1 Discrete case 1.2 Continuous version 2. SSQ clustering for stratified survey sampling Dalenius (1950/51) 3.

587 views • 24 slides
Cluster-based Segmentation Algorithm Why Shift What is Mean Idea K-means++ based Algorithm

Cluster-based Segmentation Algorithm Why Shift What is Mean Idea K-means++ based Algorithm

Cluster- Mean Shift http://vision.ouc.edu.cn/~zhenghaiyong CVBIOUC WangRuchen Cluster-based Segmentation Algorithm Why Shift What is Mean Idea K-means++ based Algorithm Idea K-means with Clustering

532 views • 36 slides
Artificial algorithm are widely used in computer science and especially in AI. Intelligence

Artificial algorithm are widely used in computer science and especially in AI. Intelligence

The Meaning of Search in AI COMP2240 The terms search, search space, search problem, search Artificial algorithm are widely used in computer science and especially in AI. Intelligence In this context the word

301 views • 15 slides
Machine search vs. humans in modeling or Im sorry Dave, Im afraid I cant do that

Machine search vs. humans in modeling or Im sorry Dave, Im afraid I cant do that

Machine search vs. humans in modeling or Im sorry Dave, Im afraid I cant do that Mark Sale M.D. Next Level Solutions Nov 2, 2011 Current (local) search algorithm assumption is path/starting point independence Prior

504 views • 27 slides
Image Denoising Using Two-stage Non-local Means Enming Luo advisor: Truong Nguyen March 15,

Image Denoising Using Two-stage Non-local Means Enming Luo advisor: Truong Nguyen March 15,

Image Denoising Using Two-stage Non-local Means Enming Luo advisor: Truong Nguyen March 15, 2012 Outline 2 Introduction 1 Non-local Means 2 Two-stage Non-local Means 3 Simulation Results 4 Conclusion and Future Work 5 Introduction

354 views • 31 slides
LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm

DATA MINING LECTURE 7 Clustering The k-means algorithm Hierarchical Clustering The DBSCAN algorithm Clustering Evaluation What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or

1.39k views • 110 slides
Local Search Theory Marco Chiarandini Department of Mathematics &amp; Computer Science

Local Search Theory Marco Chiarandini Department of Mathematics &amp; Computer Science

DM841 D ISCRETE O PTIMIZATION Part 2 Heuristics Local Search Theory Marco Chiarandini Department of Mathematics &amp; Computer Science University of Southern Denmark Local Search Revisited Outline Metaheuristics 1. Local Search

752 views • 37 slides
Constraint Satisfaction Problems: Local Search Alice Gao Lecture 7 Based on work by K.

Constraint Satisfaction Problems: Local Search Alice Gao Lecture 7 Based on work by K.

1/19 Constraint Satisfaction Problems: Local Search Alice Gao Lecture 7 Based on work by K. Leyton-Brown, K. Larson, and P. van Beek 2/19 Outline Learning Goals Introduction to Local Search Local Search Algorithms Hill climbing

361 views • 19 slides

More recommend