Conic Optimization: Relaxing at the Cutting Edge Miguel F . Anjos - PowerPoint PPT Presentation

Background Integrated IPM New Relaxations Conclusion Conic Optimization: Relaxing at the Cutting Edge Miguel F . Anjos Professor and Canada Research Chair Various parts are joint work with E. Adams (Poly Mtl), A. Engau (U. Colorado-Denver), F . Rendl and A. Wiegele (Klagenfurt), and A. Vannelli (U. Guelph) CanaDAM 2013 – Memorial U., NL, Canada – June 11, 2013

Background Integrated IPM New Relaxations Conclusion Outline Background 1 Integrated IPM 2 New Relaxations 3 Conclusion 4

Background Integrated IPM New Relaxations Conclusion Background

Background Integrated IPM New Relaxations Conclusion Conic Optimization Conic optimization refers to the problem of optimizing a linear (or possibly convex quadratic) function over the intersection of an affine space and a (pointed) closed convex cone: b T y � c , x � (P) inf (D) sup m s.t. � a i , x � = b i , i = 1 , . . . , m s.t. � y i a i + s = c i = 1 s ∈ K ∗ x ∈ K where the dual cone K ∗ is defined as K ∗ := { y ∈ ℜ n : � x , y � ≥ 0 ∀ x ∈ K} . If K = R n + then we have linear programming (LP) If K = S n + then we have semidefinite programming (SDP) � If K = SOC n = { x ∈ R n + 1 : x 0 ≥ x 2 1 + . . . + x 2 n } then we have second-order cone programming (SOCP) These cones are self-dual: K = K ∗ .

Background Integrated IPM New Relaxations Conclusion The SOC and the psd Cone The SOC constraint x 2 0 − x 2 1 − . . . − x 2 n ≥ 0 , x 0 ≥ 0 is equivalent to the positive semidefinite constraint   x 0 x 1 x 0 x 2     x 0 x 3 � 0    .  ... .   .   · · · x 1 x 2 x 3 x 0 where � 0 denotes positive semidefiniteness. Hence, SOCP is a special case of SDP , and LP is a special case of SOCP .

Background Integrated IPM New Relaxations Conclusion Why Conic Optimization? Conic optimization problems share many of the advantageous properties of LP , including: an elegant and powerful duality theory, and polynomial-time solvability using interior-point methods (IPMs) – but with a major caveat: an IPM requires a self-concordant barrier function for the cone underlying the feasible set. Although such a function (the Universal Barrier Function) exists for general convex sets, it is very hard to compute in general. However, efficient self-concordant barriers exist for symmetric cones.

Background Integrated IPM New Relaxations Conclusion Symmetric Cones Symmetric cones arise from direct products of the following five types of cones: second-order cones symmetric psd matrices over the reals (psd cone) Hermitian psd matrices over the complex numbers (can be expressed as a psd cone of twice the size); Hermitian psd matrices over the quaternions (can be expressed as a psd cone of four times the size); One exceptional 27-dimensional cone (3 × 3 Hermitian psd matrices over the octonions). Thus, S n + is (basically) the most general class of symmetric cones.

Background Integrated IPM New Relaxations Conclusion The Max-Cut Problem Given a graph G = ( V , E ) and weights w ij for all edges ( i , j ) ∈ E , find an edge-cut of maximum weight, i.e. find a set S ⊆ V s.t. the sum of the weights of the edges with one end in S and the other in V \ S is maximum. We assume wlog that w ii = 0 for all i ∈ V , and that G is complete (assign w ij = 0 if edge ij �∈ E ).

Background Integrated IPM New Relaxations Conclusion Standard Integer LP Formulation n n � � max w ij y ij i = 1 j = i + 1 s.t. y ij + y ik + y jk ≤ 2 , 1 ≤ i < j < k ≤ n y ij − y ik − y jk ≤ 0 , 1 ≤ i < j ≤ n , k � = i , j y ij ∈ { 0 , 1 } , 1 ≤ i < j ≤ n where � 1 if edge ij is cut y ij = 0 otherwise , y ij = y ji , and w ij denotes the weight of edge ij . This formulation is the basis for a highly successful branch-and-cut algorithm for solving spin glass problems in physics (Liers, Jünger, Reinelt and Rinaldi (2005)). The solver can be accessed online at the Spin Glass Server: http://www.informatik.uni-koeln.de/spinglass/

Background Integrated IPM New Relaxations Conclusion Quadratic Formulation of Max-Cut Whereas the ILP formulation is edge-based, we use a node-based quadratic formulation. Let the vector v ∈ {− 1 , + 1 } n represent any cut in the graph via the interpretation that the sets { i | v i = + 1 } and { i | v i = − 1 } specify the partition. Then max-cut may be formulated as: � 1 − v i v j n n � � � max w ij 2 i = 1 j = i + 1 v 2 s.t. i = 1 , i = 1 , . . . , n .

Background Integrated IPM New Relaxations Conclusion The Basic Semidefinite Relaxation of Max-Cut Consider the change of variable X = vv T , v ∈ {± 1 } n . Then X ij = v i v j and max-cut is equivalent to max Q • X s.t. diag ( X ) = e rank ( X ) = 1 X � 0 , where Q = 1 4 ( Diag ( We ) − W ) . Removing the rank constraint, we obtain the basic SDP relaxation of max-cut.

Background Integrated IPM New Relaxations Conclusion Goemans and Williamson (1995): 0.878-approximation algorithm Theorem If w ij ≥ 0 for all edges ij, then max-cut opt value SDP relax opt value ≥ α ξ 2 where α := min 1 − cos ξ ≈ 0 . 87856 . π 0 ≤ ξ ≤ π This result is much stronger than any similar result known for linear optimization relaxations.

Background Integrated IPM New Relaxations Conclusion Framework for a Practical Cutting-Plane Algorithm Solve initial relaxation Q • X max s.t. diag ( X ) = e X � 0 Find valid inequalities Triangle Inequalities: X ij + X ik + X jk ≥ − 1 repeat − X ij + X ik + X jk ≥ − 1 X ij − X ik + X jk ≥ − 1 Add inequalities to the conic relaxation X ij + X ik − X jk ≥ − 1 and resolve

Background Integrated IPM New Relaxations Conclusion Selected Extensions The basic SDP relaxation, augmented with selected inequalities, is a key ingredient of the max-cut solver Biqmac (Rendl, Rinaldi and Wiegele (2007)): http://biqmac.uni-klu.ac.at/ This basic relaxation of max-cut is also the basis for successful solution approaches to other problems, including: Max- k -cut problems (Ghaddar, A. and Liers (2007); A., Ghaddar, Hupp, Liers, Wiegele (2013)) Min-bisection problems (Armbruster, Helmberg, Fügenschuh and Martin (2011)) Single-row facility layout problems

Background Integrated IPM New Relaxations Conclusion Single-Row Facility-Layout Problem (SRFLP) Problem Data: n one-dimensional facilities with positive lengths ℓ i , i = 1 , . . . , n c ij pairwise interaction costs Decision Variables: permutation π Problem Objective: minimize the total weighted sum of the center-to-center distances � c ij ( 1 2 ℓ i + D π ( i , j ) + 1 � 1 � min 2 ℓ j ) = 2 c ij ( ℓ i + ℓ j ) + c ij D π ( i , j ) π ∈ Π i < j i < j i < j where D π ( i , j ) is the sum of the lengths of the facilities between i and j .

Background Integrated IPM New Relaxations Conclusion Binary Quadratic Model (A., Kennings, Vannelli (2005)) � − 1 if facility i is placed to the left of facility j R ij = 1 if facility i is placed to the right of facility j Facility k is between i and j if and only if R ki R kj = − 1 so that ( 1 − R ki R kj ) � D ( i , j ) = l k 2 k � = i , j If R ik = R kj , then R ij = R ik which gives the necessary constraint R ik R kj − R ik R ij − R ij R kj = − 1 for all triples i < k < j   c ij � � � � min const − l k R ki R kj − l k R ik R kj + l k R ik R jk  2 i < j k < i i < k < j k > j s.t. R ik R kj − R ik R ij − R ij R kj = − 1 and R 2 ij = 1 for all i < k < j

Background Integrated IPM New Relaxations Conclusion SDP Relaxation The decision variable is X = xx T where x = ( R 12 , . . . , R ( n − 1 ) n ) T ∈ R ( n 2 )   c ij � � � � min const − l k X ki , kj − l k X ik , kj + l k X ik , jk  2 i < j k < i i < k < j k > j s.t. X ij , jk − X ij , ik + X ik , jk = − 1 , diag ( X ) = e , X � 0 A cutting-plane algorithm using triangle inequalities can be applied and solve instances with up to 30 facilities to global optimality (A. and Vannelli (2008)). We can reduce the inequalities by summing over k � � � X ij , jk − X ij , ik + X ik , jk = − ( n − 2 ) k � = i , j In this way, bounds for instances with up to 100 facilities can be obtained (A. and Yen (2009)).

Background Integrated IPM New Relaxations Conclusion Integrated Interior-Point Method with Cutting Planes

Background Integrated IPM New Relaxations Conclusion Recall: Cutting-Plane Algorithm Framework Solve initial relaxation min C • X Find valid inequalities s.t. A ( X ) = b X � 0 repeat Add and/or remove inequalities and resolve This is challenging in practice as the number of inequalities increases.