clustering unsupervised learning
play

Clustering / Unsupervised Learning The target features are not given - PowerPoint PPT Presentation

Nov 13, 2023 •286 likes •608 views

Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. D. Poole and A. Mackworth 2019 c Artificial

  1. Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 1 / 19

  2. Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. The examples are partitioned in into clusters or classes. Each class predicts feature values for the examples in the class. ◮ In hard clustering each example is placed definitively in a class. ◮ In soft clustering each example has a probability distribution over its class. Each clustering has a prediction error on the examples. The best clustering is the one that minimizes the error. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 1 / 19

  3. k -means algorithm The k -means algorithm is used for hard clustering. Inputs: training examples the number of classes, k Outputs: a prediction of a value for each feature for each class an assignment of examples to classes � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 2 / 19

  4. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  5. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . prediction � X j ( i ) for each feature X j and class i . � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  6. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . prediction � X j ( i ) for each feature X j and class i . The sum-of-squares error for class and � X j ( i ) is � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  7. k -means algorithm formalized E is the set of all examples the input features are X 1 , . . . , X n X j ( e ) is the value of feature X j for example e . there is a class for each integer i ∈ { 1 , . . . , k } . The k -means algorithm outputs function class : E → { 1 , . . . , k } . class ( e ) = i means e is in class i . prediction � X j ( i ) for each feature X j and class i . The sum-of-squares error for class and � X j ( i ) is � � 2 n � � � X j ( class ( e )) − X j ( e ) . e ∈ E j =1 Aim: find class and prediction function that minimize sum-of-squares error. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 3 / 19

  8. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  9. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is the mean value of X j for that class. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  10. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is the mean value of X j for that class. Given � X j for each j , each example can be assigned to the class that � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  11. Minimizing the error The sum-of-squares error for class and � X j ( i ) is � � 2 � � n � X j ( class ( e )) − X j ( e ) . j =1 e ∈ E Given class , the � X j that minimizes the sum-of-squares error is the mean value of X j for that class. Given � X j for each j , each example can be assigned to the class that minimizes the error for that example. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 4 / 19

  12. k -means algorithm Initially, randomly assign the examples to the classes. Repeat the following two steps: For each class i and feature X j , � e : class ( e )= i X j ( e ) � X j ( i ) ← |{ e : class ( e ) = i }| , For each example e , assign e to the class i that minimizes � � 2 � n � X j ( i ) − X j ( e ) . j =1 until the second step does not change the assignment of any example. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 5 / 19

  13. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  14. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = fs [ j , c ] / cc [ c ] � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  15. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = fs [ j , c ] / cc [ c ] class ( e ) = � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  16. k -means algorithm Sufficient statistics: cc [ c ] is the number of examples in class c , fs [ j , c ] is the sum of the values for X j ( e ) for examples in class c . then define pn ( j , c ), current estimate of � X j ( c ) pn ( j , c ) = fs [ j , c ] / cc [ c ] n � ( pn ( j , c ) − X j ( e )) 2 class ( e ) = arg min c j =1 These can be updated in one pass through the training data. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 6 / 19

  17. 1: procedure k-means ( Xs , Es , k ) Initialize fs and cc randomly (based on data) 2: def pn ( j , c ) = fs [ j , c ] / cc [ c ] 3: � n j =1 ( pn ( j , c ) − X j ( e )) 2 def class ( e ) = arg min c 4: repeat 5: fsn and ccn initialized to be all zero 6: for each example e ∈ Es do 7: c := class ( e ) 8: ccn [ c ]+ = 1 9: for each feature X j ∈ Xs do 10: fsn [ j , c ]+ = X j ( e ) 11: stable := ( fsn = fs ) and ( ccn = cc ) 12: fs := fsn 13: cc := ccn 14: until stable 15: return class , pn 16: � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 7 / 19

  18. Example Data 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 8 / 19

  19. Random Assignment to Classes 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 9 / 19

  20. Assign Each Example to Closest Mean 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 10 / 19

  21. Ressign Each Example to Closest Mean 10 8 6 4 2 0 0 2 4 6 8 10 � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 11 / 19

  22. Properties of k -means An assignment of examples to classes is stable if running both the M step and the E step does not change the assignment. This algorithm will eventually converge to a stable local minimum. Any permutation of the labels of a stable assignment is also a stable assignment. It is not guaranteed to converge to a global minimum. It is sensitive to the relative scale of the dimensions. Increasing k can always decrease error until k is the number of different examples. � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 12 / 19

  23. EM Algorithm Used for soft clustering — examples are probabilistically in classes. k -valued random variable C Model Data Probabilities ➪ X 1 X 2 X 3 X 4 P ( C ) t f t t P ( X 1 | C ) C f t t f P ( X 2 | C ) f f t t P ( X 3 | C ) X 1 X 2 X 3 X 4 · · · P ( X 4 | C ) � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 13 / 19

  24. EM Algorithm M-step X 1 X 2 X 3 X 4 C count . . . . . . P ( C ) . . . . . . . . . . . . P ( X 1 | C ) t f t t 1 0.4 P ( X 2 | C ) t f t t 2 0.1 P ( X 3 | C ) t f t t 3 0.5 P ( X 4 | C ) . . . . . . . . . . . . . . . . . . E-step � D. Poole and A. Mackworth 2019 c Artificial Intelligence, Lecture 10.2 14 / 19

Recommend

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

833 views • 26 slides
Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

1.29k views • 69 slides
Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
Clustering ! Unsupervised learning: learning without a teacher ! Only features and no labels

Clustering ! Unsupervised learning: learning without a teacher ! Only features and no labels

So far in the course Supervised learning: learning with a teacher ! You had training data which was (feature, label) pairs and the goal was to learn a mapping from features to labels Clustering ! Unsupervised learning: learning without a

164 views • 12 slides
9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert Outline Introduction to clustering K-means Bag of words (dictionary learning) Hierarchical clustering

895 views • 54 slides
Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering and nearest neighbour classifiers. Justin Pearson 1 2020 1 http://user.it.uu.se/~justin/Teaching/MachineLearning/index.html 1 / 27 Supervised

813 views • 27 slides
UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised Procedures A procedure that uses unlabeled data in its classification process. Why would we use these? Collecting and labeling large data sets can

544 views • 28 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised

Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised

+ Machine Learning and Data Mining Clustering (adapted from) Prof. Alexander Ihler Unsupervised learning Supervised learning Predict target value (y) given features (x) Unsupervised learning Understand patterns of

559 views • 27 slides
Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability Density Estimation Gaussian Mixture Models M. Magdon-Ismail CSCI 4100/6100 recap: Radial Basis Functions Nonparametric RBF Parametric k -RBF-Network

253 views • 23 slides
Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning

Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning

Machine Learning 2 DS 4420 - Spring 2020 Clustering I Byron C. Wallace Unsupervised learning So far we have reviewed some fundamentals, discussed Maximum Likelihood Estimation (MLE) for probabilistic models, and neural networks/backprop

931 views • 75 slides
Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models) Professor Liang Huang (Chaps. 15-16 of CIML) Roadmap CIML Chaps. 3, 4,5,7,11,17 ,18 so far: (large-margin) supervised learning online learning:

815 views • 54 slides
Clustering / Unsupervised Learning The target features are not given in the training examples The

Clustering / Unsupervised Learning The target features are not given in the training examples The

Clustering / Unsupervised Learning The target features are not given in the training examples The aim is to construct a natural classification that can be used to predict features of the data. D. Poole and A. Mackworth 2010 c Artificial

741 views • 19 slides
K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data

K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data

K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data points. We want to find underlying structure in the data. Examples: Identify groups of similar data points. Clustering Find a better basis

831 views • 14 slides
Unsupervised learning Clustering and Dimensionality Reduction Marta Arias marias@cs.upc.edu

Unsupervised learning Clustering and Dimensionality Reduction Marta Arias marias@cs.upc.edu

Unsupervised learning Clustering and Dimensionality Reduction Marta Arias marias@cs.upc.edu Dept. CS, UPC Fall 2018 Clustering Partition input examples into similar subsets Clustering Partition input examples into similar subsets Clustering

669 views • 48 slides
Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical

Unsupervised Learning and Clustering Owen Roberts, Zach Busser, Ganesh Sugunan Hierarchical Clustering Hierarchical Clustering Hierarchical Representations Hierarchical Algorithms Agglomeration "Nearest" Clusters Complexity

769 views • 49 slides
Clustering & Unsupervised Learning Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter

Clustering & Unsupervised Learning Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter

Clustering & Unsupervised Learning Ken Kreutz-Delgado (Nuno Vasconcelos) ECE 175A Winter 2012 UCSD Statistical Learning Goal: Given a relationship between a feature vector x and a vector y , and iid data samples ( x i ,y i ), find an

534 views • 35 slides
Unsupervised learning: latent space analysis and clustering Yifeng Tao School of Computer

Unsupervised learning: latent space analysis and clustering Yifeng Tao School of Computer

Introduction to Machine Learning Unsupervised learning: latent space analysis and clustering Yifeng Tao School of Computer Science Carnegie Mellon University Slides adapted from Tom Mitchell, David Sontag, Ziv Bar-Joseph Yifeng Tao Carnegie

931 views • 62 slides
Unsupervised Learning About this class Build a model for your data. Which datapoints

Unsupervised Learning About this class Build a model for your data. Which datapoints

Unsupervised Learning About this class Build a model for your data. Which datapoints Unsupervised learning are similar? k -means Clustering Nowadays lots of work on using unlabeled data to improve the performance of supervised learn-

450 views • 6 slides
Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction

Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction

Clustering CS294 Practical Machine Learning Junming Yin 10/09/06 Outline Introduction Unsupervised learning What is clustering? Application Dissimilarity (similarity) of objects Clustering algorithm K-means, VQ,

649 views • 62 slides

More recommend