Algorithms with provable guarantees for clustering problems Ola - PowerPoint PPT Presentation

Algorithms with provable guarantees for clustering problems Ola Svensson

Where to place rescue centers? Build k centers so as to minimize sum of travel distances

Where to place rescue centers? optimize some objective Build k centers so as to minimize sum of travel distances

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( ) decrease distance for 3 clients increase distance for 6 clients

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( ) decrease distance for 3 clients decrease distance for 6 clients increase distance for 6 clients increase distance for 3 clients

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( )

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( ) CENTER: Open point/facility on real line so as to minimize max distance over all clients ( )

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( ) CENTER: Open point/facility on real line so as to minimize max distance over all clients ( )

Median and Center MEDIAN: Open point/facility on real line so as to minimize sum of distances from clients ( ) CENTER: Open point/facility on real line so as to minimize max distance over all clients ( ) x x

K-Median and K-Center K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( )

K-Median and K-Center K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( ) K-CENTER: Open k points/facilities in a metric space so as to minimize max distance over all clients ( )

K-Median and K-Center K-MEDIAN: Open k points/facilities in a metric space so as to minimize sum of distances from clients ( ) K-CENTER: Open k points/facilities in a metric space so as to minimize max distance over all clients ( )

Mathematical formulation of objective functions

Mathematical formulation of objective functions General Problem parameterized by 𝒒 ≥ 𝟐 : Find a set 𝑻 of k points/facilities in a metric space so as to minimize 𝟐/𝒒 𝒆 𝒌, 𝑻 𝒒 𝒌 𝒅𝒎𝒋𝒇𝒐𝒖

Mathematical formulation of objective functions General Problem parameterized by 𝒒 ≥ 𝟐 : Find a set 𝑻 of k points/facilities in a metric space so as to minimize 𝟐/𝒒 𝒆 𝒌, 𝑻 𝒒 𝒌 𝒅𝒎𝒋𝒇𝒐𝒖 Distance from client j to closest facility in S K-MEDIAN: 𝒒 = 𝟐 K-CENTER: 𝒒 = ∞ Actually, 𝑘 𝑑𝑚𝑗𝑓𝑜𝑢 𝑒 𝑘, 𝑇 2 and Euclidean metric K-MEANS: 𝒒 = 𝟑

Facility Location Facility Location: Open facilities in a metric space so as to minimize sum of distances from clients + opening costs

ALL THESE PROBLEMS ARE INTRACTABLE (NP-HARD) IN THE WORST CASE

Solving intractable problems • Heuristics • good for “typical” instances • bad instances do not happen too often 16384 4096 ! 1024 Sweden has only 9 256 million inhabitants 64 Dantzig, Fulkerson, and Johnson solve a 49- 16 city instance to optimality ≈ 360 persons/city 4 Applegate, Bixby, Chvatal, Cook, and Helsgaun solve a 1 24978-city instance 50's 70's 80's 90's 00's

Solving intractable problems Approximation Algorithms • • Perhaps we can efficiently find a reasonably good solution? Approximation Ratio: worst case over all instances α =1 is an exact polynomial time algorithm • α =1.01 then algorithm finds a solution with at most 1% higher cost •

GOAL: Complete understanding of worst case behavior

State of the Art Approximation Hardness 1.488 1.463 Facility Location [Li’11] [Guha & Khuller’98] 2 2 K-Center [Gonzales’85, Hochbaum & Shmoys’85 ] [Hsu & Nemhauser’79] 2.67 1+2/e K-Median [Byrka et al’15] [Jain et al.’02] 9 1.0013 K-Means [Kanungo et al’2004] [Lee. Schmidt, Wright’15] Even better: Approximation algorithms (can be) achieved by standard LP relaxations and techniques transfer between problems

A 2-APPROXIMATION ALGORITHM FOR K-CENTER

Open any point Greedy K-Center For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points

Open any point Greedy K-Center For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points

Open any point Greedy K-Center For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points

Open any point Greedy K-Center For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Consider optimal solution and corresponding Voronoi diagram

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1: We opened up one point in each cell

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1: We opened up one point in each cell

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1: We opened up one point in each cell ≤ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1: We opened up one point in each cell ≤ 𝑃𝑄𝑈 ≤ 2 ⋅ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈 In this case any client is connected within distance ≤ 𝟑 ⋅ 𝑷𝑸𝑼

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1I: We did not open up one point in each cell

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1I: We opened up two points in a single cell

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1I: We opened up two points in a single cell ≤ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈

Open any point Analysis For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Case 1I: We opened up two points in a single cell ≤ 2 ⋅ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈 ≤ 𝑃𝑄𝑈 Also in this case any client is connected within distance ≤ 𝟑 ⋅ 𝑷𝑸𝑼

Open any point For 𝑗 = 2, … , 𝑙 Open point farthest away from already opened points Gonzales, Hochbaum & Shmoys’85 THEOREM: The above greedy algorithm is a 2-approximation for k-Center

ALGORITHMS FOR FACILITY LOCATION AND K-MEDIAN

LINEAR PROGRAMMING RELAXATION

LP Relaxation for Facility Location LINEAR PROGRAM: • y i takes value 1 if i is opened and 0 otherwise • x ij takes value 1 if j is connected to i and 0 otherwise

LP Relaxation for Facility Location LINEAR PROGRAM: • y i takes value 1 if i is opened and 0 otherwise • x ij takes value 1 if j is connected to i and 0 otherwise connection cost opening cost minimize 𝑗∈𝐺 𝑔 𝑗 𝑧 𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒 𝑗𝑘 𝑦 𝑗𝑘 subject to 𝑗∈𝐺 𝑦 𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦 𝑗𝑘 ≤ 𝑧 𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦 𝑗𝑘 , 𝑧 𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷

LP Relaxation for Facility Location LINEAR PROGRAM: • y i takes value 1 if i is opened and 0 otherwise • x ij takes value 1 if j is connected to i and 0 otherwise minimize 𝑗∈𝐺 𝑔 𝑗 𝑧 𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒 𝑗𝑘 𝑦 𝑗𝑘 Every client is connected subject to 𝑗∈𝐺 𝑦 𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦 𝑗𝑘 ≤ 𝑧 𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦 𝑗𝑘 , 𝑧 𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷

LP Relaxation for Facility Location LINEAR PROGRAM: • y i takes value 1 if i is opened and 0 otherwise • x ij takes value 1 if j is connected to i and 0 otherwise minimize 𝑗∈𝐺 𝑔 𝑗 𝑧 𝑗 + 𝑗∈𝐺,𝑘∈𝐷 𝑒 𝑗𝑘 𝑦 𝑗𝑘 Clients connected to open facilities subject to 𝑗∈𝐺 𝑦 𝑗𝑘 = 1 𝑘 ∈ 𝐷 𝑦 𝑗𝑘 ≤ 𝑧 𝑗 i ∈ 𝐺, 𝑘 ∈ 𝐷 𝑦 𝑗𝑘 , 𝑧 𝑗 ∈ [0,1] i ∈ 𝐺, 𝑘 ∈ 𝐷