Formulation of the . . . Why µ p in Fuzzy Clustering? Our Explanation Proof Kehinde Akinola, Ahnaf Farhan, and Vladik Kreinovich Home Page Title Page University of Texas at El Paso El Paso, TX 79968, USA ◭◭ ◮◮ kaakinola@miners.utep.edu, ◭ ◮ afarhan@miners.utep.edu, Page 1 of 8 vladik@utep.edu Go Back Full Screen Close Quit
1. Formulation of the Problem Formulation of the . . . • One of the main algorithms for clustering n given Our Explanation d -dimensional points: Proof – selects K “typical” values c k and – selects assignments k ( i ) for each i from 1 to n Home Page – so as to minimize the sum Title Page � ( x i − c k ( i ) ) 2 . ◭◭ ◮◮ i ◭ ◮ • This minimization is usually done iteratively. Page 2 of 8 • First, we pick c k and assign each point x i to the cluster k whose representative c k is the closest to x i . Go Back Full Screen • Then, we freeze k ( i ) and select new typical represen- tatives c k by minimizing the objective function. Close • This leads to c k being an average of all the points x i Quit assigned to the k -th cluster.
2. Formulation of the Problem (cont-d) Formulation of the . . . • Then, the procedure repeats again and again – until Our Explanation the process converges. Proof • In practice, for some objects, we cannot definitely as- sign them to a single cluster. Home Page • In such cases, it is reasonable to assign, to each object i , Title Page – degrees µ ik of belongs to different clusters k , ◭◭ ◮◮ – so that � µ ik = 1. ◭ ◮ k • In this case, it seems reasonable to take each term Page 3 of 8 ( x i − c k ) 2 with the weight µ ik . Go Back • In other words, it seems reasonable to find the values Full Screen µ ik and c k by minimizing the expression Close � µ ik · ( x i − c k ) 2 . Quit i,k
3. Formulation of the Problem (cont-d) Formulation of the . . . • It seems reasonable to minimize Our Explanation � µ ik · ( x i − c k ) 2 . Proof i,k • However, this expression is linear in µ ik . Home Page • It is known that the minimum of a linear function un- Title Page der linear constraints is always at a vertex. ◭◭ ◮◮ • Thus, the minimum is attained when one value µ ik is ◭ ◮ 1 and the rest are 0s. Page 4 of 8 • We want to come up with truly fuzzy clustering, with Go Back 0 < µ ik < 1 for some i and k . Full Screen • Thus, we need to replace the factor µ ik with a non- linear expression f ( µ ik ). Close f ( µ ik ) · ( x i − c k ) 2 . • Then, we minimize the expression � Quit i,k
4. Formulation of the Problem (cont-d) Formulation of the . . . • We minimize the expression Our Explanation Proof � f ( µ ik ) · ( x i − c k ) 2 . i,k • In practice, the functions f ( µ ) = µ p works the best. Home Page Title Page • Why? ◭◭ ◮◮ ◭ ◮ Page 5 of 8 Go Back Full Screen Close Quit
5. Our Explanation Formulation of the . . . • The weights µ ik are normalized so that their sum is 1. Our Explanation • So, if we delete some clusters or add more clusters, we Proof need to re-normalize these values. • A usual way to do it is to multiply them by a normal- Home Page ization constant c . Title Page • It is therefore reasonable to require that: ◭◭ ◮◮ – the relative quality of different clustering ideas ◭ ◮ – not change is we simply re-scale. Page 6 of 8 • This implies, e.g., that: Go Back – if f ( µ 1 ) · v 1 = f ( µ 2 ) · v 2 , Full Screen – then after re-scaling µ i → c · µ i , we should have Close f ( c · µ 1 ) · v 1 = f ( c · µ 2 ) · v 2 . Quit • We show that this condition implies that f ( µ ) = µ p .
6. Our Explanation (cont-d) Formulation of the . . . • Indeed, f ( c · µ 2 ) = f ( µ 2 ) f ( c · µ 1 ) = v 1 Our Explanation f ( µ 1 ) . v 2 Proof = f ( c · µ 1 ) = f ( c · µ 2 ) def • Thus r for all µ 1 and µ 2 . f ( µ 1 ) f ( µ 2 ) Home Page • So, the ratio r does not depend on µ : r = r ( c ), and Title Page f ( c · µ ) = r ( c ) · f ( µ ) . ◭◭ ◮◮ ◭ ◮ • It is known that the only continuous solutions of this functional equations are f ( µ ) = C · µ p . Page 7 of 8 • Minimization is not affected if we divide the objective Go Back function by C and get f ( µ ) = µ p . Full Screen Close Quit
7. Proof Formulation of the . . . • We can solve the equation f ( c · µ ) = r ( c ) · f ( µ ) when Our Explanation f ( µ ) is differentiable. Proof • Indeed, since f ( µ ) is differentiable, the ratio r ( c ) = f ( c · µ ) is also differentiable. Home Page f ( µ ) Title Page • If we differentiate both sides of the equation with re- spect to c , we get µ · f ′ ( c · µ ) = r ′ ( c ) · f ( µ ). ◭◭ ◮◮ • For c = 1, we get µ · d f def ◭ ◮ dµ = p · f , where p = r ′ (1). Page 8 of 8 • If we move all the terms containing f to one side and Go Back all others to another, we get d f = p · dµ f µ . Full Screen • Integrating, we get ln( f ) = p · ln( µ ) + c 1 . Close • If we apply exp to both sides, we get f ( µ ) = C · µ p , Quit where C = exp( c 1 ) .
Recommend
More recommend
