Convex Combinatorial Optimization ORSIS Prize 2005 Shmuel Onn and - PowerPoint PPT Presentation

Convex Combinatorial Optimization ORSIS Prize 2005 Shmuel Onn and Uri Rothblum Technion – Israel Institute of Technology http://ie.technion.ac.il/~onn Supported in part by ISF – Israel Science Foundation

Linear Combinatorial Optimization (LCO) 2 N of subsets of N = {1,…,n} and real ⊆ LCO: Given family F ∑ weighting w:N , find F F F of maximum weight w(F)= w(j) , ∈ ∈ F j Example: Spanning Trees 1 e.g. n=6, graph G=K 4 gives the family F = {{1,2,4}, {1,2,5}, … {3,5,6}, {4,5,6}} F 5 4 2 3 6 3) Shmuel Onn

Linear Combinatorial Optimization (LCO) 2 N of subsets of N = {1,…,n} and real ⊆ LCO: Given family F ∑ weighting w:N , find F F F of maximum weight w(F)= w(j) , ∈ ∈ F j Example: Spanning Trees 3 e.g. n=6, graph G=K 4 gives the family F = {{1,2,4}, {1,2,5}, … {3,5,6}, {4,5,6}} F -1 0 Now consider weighting w:{1,…,6} 1 2 -2 3) Shmuel Onn

Linear Combinatorial Optimization (LCO) 2 N of subsets of N = {1,…,n} and real ⊆ LCO: Given family F ∑ weighting w:N , find F F F of maximum weight w(F)= w(j) , ∈ ∈ F j Example: Spanning Trees 3 e.g. n=6, graph G=K 4 gives the family F = {{1,2,4}, {1,2,5}, … {3,5,6}, {4,5,6}} F -1 0 Now consider weighting w:{1,…,6} . 1 2 -2 3) The maximum weight tree is easily obtained by the greedy algorithm Shmuel Onn

Convex Combinatorial Optimization (CCO) 2 N , vectorial weighting w:N CCO: Given family F ⊆ , and , d convex functional c: , find F F F of maximum value c(w(F)). ∈ d Example: Spanning Trees (3 -2) Consider again n=6, the graph G=K 4 , d=2, weighting w and convex function (-1 2) (0 1) 2 2 2 2 c: defined by c(x) = |x| = x 1 + x 2 (1 0) (2 -1) The objective value of the optimal (-2 3) tree F* is c(w(F*)) = c((-3 6)) = 9 + 36 = 45 where w(F*) = (0 1) + (-1 2) + (-2 3) = (-3 6) Shmuel Onn

Some bad news, good news, and questions 2 N of subsets of N = {1,…,n} and real ⊆ LCO: Given family F F ∑ weighting w:N , find F F F of maximum weight w(F)= w(j) , ∈ ∈ F j 2 N , vectorial weighting w:N CCO: Given family F , and ⊆ , d convex functional c: , find F F F of maximum value c(w(F)). ∈ d Very broad expressive power Generally intractable even for d=1 For variable d, intractable even for F F =2 N We provide broad polynomial time solvabe setup Approximation algorithms ? Classification of “edge-well-behaved” families F F ? Convex minimization ? Shmuel Onn

Application 1: Matroids Here F is the family of independent sets or bases of a matroid over N presented by a membership oracle. In particular, includes the spanning trees of before. Corollary 1: CCO over matroids is polynomially solvable (studied by Hassin and Tamir, Onn) Shmuel Onn

Application 2: Positive Semidefinite Quadratic Assignment Quadratic Assignment: Given n x n matrix M, find vector x {0,1} n maximizing x T Mx ∈ Special Case (yet NP-hard): M positive semidefinite of rank d, so M = W T W with W given d x n matrix Formulation as CCO: F = 2 N is the family of all subsets w(j) = W j is the j-th column of W 2 + … + x d d c: is defined by c(x) = |x| 2 = x 1 2 Corollary 2: PSD Quadratic Assignment is polynomially solvable (studied by Allemand, Fukuda, Liebling and Steiner) Shmuel Onn

Application 3: Partitioning Problems Partition m items, each evaluated by k criteria, to p players, to maximize social utility which is a convex function of the sums of values of items each player gets The data for the problem consists of: Criteria-item table given as k x m matrix A Restrictions on the number of items each player gets kxp Convex functional on k x p matrices c: Shmuel Onn

Example and demonstration of the utility computation: Consider m=6 items, k=2 criteria, p=3 players The criteria -item matrix is: items criteria Each player gets 2 items The convex functional on k x p matrices is c(X) = ∑ X ij 3 The matrix of a partition such as π = (34, 56, 12) is: players criteria The social utility of π is c(A π ) = 244432 Shmuel Onn

All 90 partitions π of items {1, …,6} To 3 players where each player gets 2 items Shmuel Onn

All 90 corresponding 2x3 partition matrices A π on which the convex c is to be evaluated Shmuel Onn

The optimal partition is among 30 special extremal partitions, which are directly enumerable by our CCO algorithm. π = (34, 56, 12) players criteria c(A π ) = 244432 Shmuel Onn

Formulation as CCO: Define n = mp, d = kp The ground set is N = { (i,j) : i = 1, …, m, j = 1, …, p } Each partition π = ( π 1 , …, π p ) is encoded as F π = { (i,j) : i π j }; ∈ 2 N consists of all F π of partitions π obyeing restrictions F ⊆ kxp d ≅ The weight function w:N is given by j A i w(i,j) = A i 0 0 ⊗ 1 j = i=1, …, m, j=1, …, p k p The convex functional c on k x p matrices is as given. Corollary 3: Partitioning problems are polynomially solvable (studied by Aviran, Granot, Barnes, Hoffman, Hwang, Onn, Rothblum, and more) Shmuel Onn

Application 4: Minimum Variance Clustering Given m points v 1 , …, v m in k , group them into p (balanced) clusters so as to minimize the sum of cluster variances . P=3, k=3, m large Formulation as CCO: Special case of partitioning, hence CCO problem with n=mp, d=kp, weighting w of elements by k x p matrices, and convex function c assigning to each k x p matrix X its Euclidean norm c(X) = ∑ X ij 2 Corollary 4: Minimum variance clustering is polynomially solvable (numerous applications in the analysis of statistical data and other areas) Shmuel Onn

The Geometric Approach to Combinatorial Optimization Denote by 1 j the j-th unit vector in R n . ∑ ⊆ For a subset F N denote by 1 F = 1 j its indicator. ∈ F j 2 N is the 0-1 polytope ⊆ The family polytope of F F F = conv { 1 F : F F P F ∈ F } The linear and convex combinatorial optimization linear and convex combinatorial optimization The F of P F problems now ask for the best vertex problems now ask for the best vertex of P Shmuel Onn

The Gr Ö tschel – Lovász – Schrijver Theory F is facet-well-behaved then linear combinatorial optimization (LCO) If P F over the family F F is solvable in polynomial time. The proof makes use of: � Ellipsoid method (equivalence of separation and optimization) Shmuel Onn

Our Work F is edge-well-behaved then convex combinatorial optimization (CCO) If P F over the family F F is solvable in strongly polynomial time. Shmuel Onn

Our Work F is edge-well-behaved then convex combinatorial optimization (CCO) If P F over the family F F is solvable in strongly polynomial time. The proof makes use of: � Zonotope construction � � Equivalence of augmentation and optimization (via scaling and diophantine approximation) Shmuel Onn

The Main Theorem A family F F is edge-guaranteed if it comes with a set of vectors E = {e 1 , …, e m } R n containing every edge direction of the polytope P F F . ⊆ Typically, have edge-well-behaved classes C = ( F F n ) of families having “uniform” edge-direction sets E n = {e 1 , …, e m(n) } with m(n) polynomial in n. 2 N , vectorial weighting w:N R d , Now, recall CCO: Given family F ⊆ , and convex functional c:R d R, find F F F of maximum value c(w(F)). ∈ Theorem: Fix Theorem: Fix d d. Then . Then convex combinatorial optimization (CCO) is strongly polynomially time solvable over any edge-guaranteed family F F given by a membership oracle and convex c given by an evaluation oracle. Shmuel Onn

Back to the Applications 1. Matroids over n elements: edge-well-behaved with with E n = { 1 i - 1 j : 1 ≤ i <j ≤ n } so m(n) = O(n 2 ) 2. PSD quadratic assignment over n x n matrices: edge-well-behaved with E n = { 1 i : 1 ≤ i ≤ n } so m(n) = n with 3. Partitioning n items to p players: edge-well-behaved with with E n the set of all n x p circuit matrices so m(n) = O(n p ) 4. Minimum variance clustering: same as partitioning, m(n) = O(n p ) Corollary: All are strongly polynomially time solvable Shmuel Onn

Proof ingredient 1: zonotope refinement and construction Lemma 1: If E = {e 1 , …, e m } contains all edge directions of a polytope P then the zonotope Z = [-1, 1] e 1 + … + [-1, 1] e m is a refinement of P. a 1 a a 6 a 6 a a a 1 a 1 6 6 1 a 5 a a 5 a 5 5 a 2 a 2 P Z 1 e 1 e 2 e 2 e a 2 a 2 E E 3 e 3 e a 3 a 3 a 4 a a 3 a 4 3 a 4 a 4 Lemma 2: In R d , the zonotope Z can be constructed from E = {e 1 , …, e m } along with a vector a i in the cone of every vertex in O(m d-1 ) operations. (Edelsbrunner, Gritzmann, Orourk, Seidel, Sharir, Sturmfels, … ) Shmuel Onn

Proof ingredient 2: membership augmentation optimization Consider LCO with weight w over family F F edge-guaranteed by E Lemma 3: Membership Augmentation Proof: F in F F can be improved if and only if there is an edge direction e in E such that w ● e > 0 and 1 F + e = 1 G for some G in F F . Lemma 4: Augmentation (linear) Optimization Proof: Schulz-Weismantel-Ziegler and Gr Ö tschel–Lovász using scaling ideas going back to Edmonds-Karp. Lemma 5: Polynomial time LCO Strongly polynomial time LCO Proof: Frank-Tárdos show that using Diophantine approximation can replace w by w’ of bit size depending polynomially only on n. Shmuel Onn