Al Algo gori rithms thms fo for r in inst stan ance ce-st - PowerPoint PPT Presentation

Al Algo gori rithms thms fo for r in inst stan ance ce-st stable able an and pe d pert rtur urbat bation ion-res resili ilient ent pr prob oblems ems Haris Angelidakis, TTIC Konstantin Makarychev, Northwestern Yury Makarychev, TTIC Aravindan Vijayaraghavan, Northwestern QTW on Beyond Worst-Case Analysis Northwestern, May 24, 2017

Motivation • Practice: Need to solve clustering and combinatorial optimization problems. • Theory: • Many problems are NP-hard. Cannot solve them exactly. • Design approximation algorithms for worst case. Can we get better algorithms for real-world instances than for worst-case instances?

Motivation • Answer: Yes! When we solve problems that arise in practice, we often get much better approximation than it is theoretically possible for worst case instances. • Want to design algorithms with provable performance guarantees for solving real-world instances.

Motivation • Need a model for real-world instances. • Many different models have been proposed. • It’s unrealistic that one model will capture all instances that arise in different applications.

This work • Assumption: instances are stable/perturbation- resilient • Consider several problems: • 𝑙 -means • 𝑙 -median • Max Cut • Multiway Cut • Get exact polynomial-time algorithms

𝑙 -means and 𝑙 -median Given a set of points 𝑌 , distance 𝑒(⋅,⋅) on 𝑌 , and 𝑙 Partition 𝑌 into 𝑙 clusters 𝐷 1 , … , 𝐷 𝑙 and find a “center” 𝑑 𝑗 in each 𝐷 𝑗 so as to minimize 𝑙 ( 𝑙 -median) ෍ ෍ 𝑒(𝑣, 𝑑 𝑗 ) 𝑗=1 𝑣∈𝐷 𝑗 𝑙 𝑒 𝑣, 𝑑 𝑗 2 ෍ ෍ ( 𝑙 -means) 𝑗=1 𝑣∈𝐷 𝑗

Multiway Cut Given • a graph 𝐻 = (𝑊, 𝐹, 𝑥) • a set of terminals 𝑢 1 , … , 𝑢 𝑙 𝑢 2 𝑢 4 𝑢 1 𝑢 3 Find a partition of 𝑊 into sets 𝑇 1 , … , 𝑇 𝑙 that minimizes the weight of cut edges s.t. 𝑢 𝑗 ∈ 𝑇 𝑗 .

Instance-stability & perturbation- resilience ➢ Consider an instance ℐ of an optimization or clustering problem. ➢ ℐ′ is a 𝛿 -perturbation of ℐ if it can be obtained from ℐ by “perturbing the parameters” — multiplying each parameter by a number from 1 to 𝛿 . • 𝑥 𝑓 ≤ 𝑥 ′ 𝑓 ≤ 𝛿 ⋅ 𝑥 𝑓 • 𝑒(𝑣, 𝑤) ≤ 𝑒 ′ 𝑣, 𝑤 ≤ 𝛿 ⋅ 𝑒(𝑣, 𝑤)

Instance-stability & perturbation- resilience An instance ℐ of an optimization or clustering problem is perturbation-resilient/instance-stable if the optimal solution remains the same when we perturb the instance: every γ -perturbation ℐ′ has the same optimal solution as ℐ

Instance-stability & perturbation- resilience Every γ -perturbation ℐ′ has the same optimal solution as ℐ • In practice, we are interested in solving instances where the optimal solution “stands out” among all solutions [Bilu, Linial] • Objective function is an approximation to the “true” objective function. • “Practically interesting instance” ⇒ it is stable

Results

History Instance-stability & perturbation-resilience was introduced in discrete optimization: by Bilu and Linial `10 in clustering: by Awasthi, Blum, and Sheffet `12

Results (clustering) [Awasthi, Blum, Sheffet 𝒍 -center, 𝛿 ≥ 3 𝒍 -means, `12] 𝒍 -median 𝒍 -center, [Balcan, Liang `13] 𝛿 ≥ 1 + 2 𝒍 -means, 𝒍 -median [Balcan, Haghtalab, sym. /asym. 𝛿 ≥ 2 𝒍 -center White `16] 𝒍 -means, 𝛿 ≥ 2 [AMM `17] 𝒍 -median

Results (optimization) [Bilu, Linial `10] 𝛿 ≥ 𝑑𝑜 Max Cut [Bilu, Daniely, Linial, 𝛿 ≥ 𝑑 𝑜 Max Cut Saks `13] [MMV `13] 𝛿 ≥ 𝑑 log 𝑜 log log 𝑜 Max Cut 𝛿 ≥ 4 [MMV `13] Multiway 𝛿 ≥ 2 − 2/𝑙 [AMM `17] Multiway

Results (optimization) Our algorithms are robust. • Find the optimal solution, if the instance is stable. • Find an optimal solution or detects that the instance is not stable, otherwise. • Never output an incorrect answer. Solve weakly stable instances. Assume that when we perturb the instance • the optimal solution changes only slightly, or • there is a core that changes only slightly.

Hardness results for center-based obejctives [Balcan, Haghtalab, White `16] No polynomial-time algorithm for (2 − 𝜁) -perturbation-resilient instances of 𝑙 -center ( 𝑂𝑄 ≠ 𝑆𝑄) . [Ben-David, Reyzin `14] No polynomial-time algorithm for instances of 𝑙 -means, 𝑙 -median, 𝑙 -center satisfying (2 − 𝜁) -center proximity property ( 𝑄 ≠ 𝑂𝑄) .

Hardness results for optimization problems Set Cover, Vertex Cover, Min 2-Horn Deletion There is no robust algorithm for 𝑃(𝑜 1−𝜁 ) -stable instances unless P = NP [AMM `17]. Provide evidence that [MMV `13, AMM `17] • No robust algorithm for Max Cut when 𝛿 < 𝑃 log 𝑜 log log 𝑜 • Multiway cut is hard when 𝛿 < 4 1 𝑙 . 3 − 𝑃

Algorithm for Clustering Problems

Center proximity property [Awasthi, Blum, Sheffet `12] A clustering 𝐷 1 , …, 𝐷 𝑙 with centers 𝑑 1 , …, 𝑑 𝑙 satisfies the center proximity property if for every 𝑞 ∈ 𝐷 𝑗 : 𝑒 𝑞, 𝑑 > 𝛿 𝑒 𝑞, 𝑑 𝑗 𝑘 𝑞 𝑑 𝑑 𝑗 𝑘 𝐷 𝑗 𝐷 𝑘

Plan [AMM `17] i. 𝛿 -perturbation resilience ⇒ 𝛿 -center proximity ii. 2-center proximity ⇒ each cluster is a subtree of the MST iii. use single-linkage + DP to find 𝐷 1 , … , 𝐷 𝑙

Perturbation resilience ⇒ center proximity Perturbation resilience: the optimal clustering doesn’t change when we perturb the distances. 𝑒 𝑣, 𝑤 /𝛿 ≤ 𝑒 ′ 𝑣, 𝑤 ≤ 𝑒(𝑣, 𝑤) [ABS `12] 𝑒′(⋅,⋅) doesn’t have to be a metric [AMM `17] 𝑒′(⋅,⋅) is a metric Metric perturbation resilience is a more natural notion.

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Assume center proximity doesn’t hold. Then 𝑒 𝑞, 𝑑 𝑘 ≤ 𝛿 𝑒 𝑞, 𝑑 𝑗 . 𝑞 𝑑 𝑑 𝑗 𝑘 𝐷 𝑗 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Assume center proximity doesn’t hold. • Let 𝑒 ′ 𝑞, 𝑑 𝑘 = 𝑒 𝑞, 𝑑 𝑗 ≥ 𝛿 −1 𝑒(𝑞, 𝑑 𝑘 ) . • Don’t change other distances. • Consider the shortest-path closure. 𝑞 𝑑 𝑑 𝑗 𝑘 𝐷 𝑗 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Assume center proximity doesn’t hold. • Let 𝑒 ′ 𝑞, 𝑑 𝑘 = 𝑒 𝑞, 𝑑 𝑗 ≥ 𝛿 −1 𝑒(𝑞, 𝑑 𝑘 ) . • Don’t change other distances. • Consider the shortest-path closure. 𝑞 𝑑 𝑑 𝑗 𝑘 𝐷 𝑗 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Assume center proximity doesn’t hold. • Let 𝑒 ′ 𝑞, 𝑑 𝑘 = 𝑒 𝑞, 𝑑 𝑗 ≥ 𝛿 −1 𝑒(𝑞, 𝑑 𝑘 ) . • Don’t change other distances. • Consider the shortest-path closure. This is a 𝛿 -perturbation. 𝑞 𝑑 𝑑 𝑗 𝑘 𝐷 𝑗 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Distances inside clusters 𝑫 𝒋 and 𝑫 𝒌 don’t change. Consider 𝑣, 𝑤 ∈ 𝐷 𝑗 . 𝑒 𝑣, 𝑤 , 𝑒 ′ 𝑣, 𝑤 = min 𝑒 𝑣, 𝑞 + 𝑒′ 𝑞, 𝑑 𝑘 + 𝑒 𝑑 𝑘 , 𝑤 𝑣 𝑞 𝑑 𝑑 𝑗 𝑘 𝑤 𝐷 𝑗 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Distances inside clusters 𝑫 𝒋 and 𝑫 𝒌 don’t change. Consider 𝑣, 𝑤 ∈ 𝐷 𝑗 . 𝑒 𝑣, 𝑤 , 𝑒 ′ 𝑣, 𝑤 = min 𝑒 𝑣, 𝑞 + 𝑒′ 𝑞, 𝑑 𝑘 + 𝑒 𝑑 𝑘 , 𝑤 𝑣 𝑞 𝑑 𝑑 𝑗 𝑘 𝑤 𝐷 𝑗 𝐷 𝑘

Perturbation resilience ⇒ center proximity [ABS `12, AMM `17] Since the instance is 𝛿 -stable, 𝐷 1 , … , 𝐷 𝑙 must be the unique optimal solution for distance 𝑒′ . Still, 𝑑 𝑗 and 𝑑 𝑘 are optimal centers for 𝐷 𝑗 and 𝐷 𝑘 . 𝑒 ′ 𝑞, 𝑑 𝑗 = 𝑒 ′ 𝑞, 𝑑 𝑘 ⇒ can move 𝑞 from 𝐷 𝑗 to 𝐷 𝑘 𝑞 𝑑 𝑘 𝑑 𝑗 𝐷 𝑗 𝐷 𝑘

Each cluster is a subtree of MST [ABS `12] 2-center proximity ⇒ every 𝑣 ∈ 𝐷 𝑗 is closer to 𝑑 𝑗 than to any 𝑤 ∉ 𝐷 𝑗 Assume the path from 𝑣 ∈ 𝐷 𝑗 to 𝑑 𝑗 in MST, leaves 𝐷 𝑗 . 𝑤 𝑑 𝑗 𝑣

Each cluster is a subtree of MST [ABS `12] 2-center proximity ⇒ every 𝑣 ∈ 𝐷 𝑗 is closer to 𝑑 𝑗 than to any 𝑤 ∉ 𝐷 𝑗 Assume the path from 𝑣 ∈ 𝐷 𝑗 to 𝑑 𝑗 in MST, leaves 𝐷 𝑗 . 𝑤 𝑑 𝑗 𝑣

Dynamic programming algorithm Root MST at some 𝑠 . 𝑈 𝑣 is the subtree rooted at 𝑣 . cost 𝑣 (𝑘, 𝑑) : the cost of the partitioning of 𝑈 𝑣 • into 𝑘 clusters (subtrees) • so that 𝑑 is the center of the cluster containing 𝑣 . 𝑠 𝑑 𝑣 𝑈(𝑣)

Dynamic programming algorithm Fill out the DP table bottom-up. Example: 𝑙 -median, 𝑣 has 2 children 𝑣 1 and 𝑣 2 . 𝑣 𝑣 1 𝑣 2 𝑈(𝑣)

Dynamic programming algorithm Fill out the DP table bottom-up. Example: 𝑙 -median, 𝑣 has 2 children 𝑣 1 and 𝑣 2 . 𝑣 𝑈(𝑣)

Dynamic programming algorithm Fill out the DP table bottom-up. Example: 𝑙 -median, 𝑣 has 2 children 𝑣 1 and 𝑣 2 . 𝑣 𝑈(𝑣)