DM825 Introduction to Machine Learning Lecture 13 Unsupervised Learning Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark
k -means Outline EM Algorithm 1. k -means 2. Expectation Maximization Algorithm 2
k -means Outline EM Algorithm 1. k -means 2. Expectation Maximization Algorithm 3
k -means k -means EM Algorithm x m } and no y i we want to cluster the data Given { � x 1 , . . . , � Initialize cluster centroids randomly µ 1 , . . . , µ k ∈ R n repeat for i = 1 . . . m do c i ← arg min l � x i − µ l � 2 ; // assign for l = 1 . . . k do m I { c i = l } x i � i =1 µ l ← I { c i = l } ; // move m � i =1 until convergence ; k is a parameter i =1 � x i − µ c i � 2 µ ) = � m Optimization of the distortion function J ( � c, � k -means ≡ coordinate descent on J : solve in � c, � µ by changing one variable while keeping the others fixed. Each probability solved optimally. J ( � c, � µ ) is non convex hence local optimality issues Convergence guaranteed by decreasing J . 4
k -means EM Algorithm 5
Recommend
More recommend