BiqCrunch: a semidefinite-based solver for binary quadratic problems - PowerPoint PPT Presentation

BiqCrunch: a semidefinite-based solver for binary quadratic problems Nathan Krislock PIMS Postdoctoral Fellow, Computer Science University of British Columbia, Canada Pacific Institute for the Mathematical Sciences Canadian Discrete and Algorithmic Mathematics Conference Memorial University of Newfoundland June 10–13, 2013 Joint work with J´ erˆ ome Malick (CNRS) and Fr´ ed´ eric Roupin (Universit´ e Paris 13) Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 1 / 22

BiqCrunch Binary Quadratic Problems BiqCrunch is a branch & bound solver for binary quadratic problems: z T A 0 z + b T maximize 0 z + c 0 z T A i z + b T subject to i z = c i , i ∈ { 1 , . . . , m E } z T A j z + b T j z ≤ c j , j ∈ { 1 , . . . , m I } z ∈ { 0 , 1 } n examples of binary quadratic problems: ◮ Max-Cut / Unconstrained Binary Quadratic ◮ Maximum k -Cluster ◮ Maximum Independent Set / Quadratic Stable Set ◮ Quadratic Knapsack ◮ Quadratic Assignment ◮ . . . Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 2 / 22

BiqCrunch BiqCrunch uses our improved semidefinite bounding procedure written in C and Fortran and uses: ◮ L-BFGS-B, a library for quasi-Newton bound-constrained optimization ◮ BOB, a branch-and-bound framework uses our BC (BiqCrunch) format – is very similar to the SDPA format available as an online solver Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 3 / 22

BiqCrunch Vector form z T A 0 z + b T maximize 0 z + c 0 z T A i z + b T subject to i z = c i , i ∈ { 1 , . . . , m E } z T A j z + b T j z ≤ c j , j ∈ { 1 , . . . , m I } z ∈ { 0 , 1 } n Matrix form maximize � Q 0 , Z � � Q i , Z � = c i , i ∈ { 1 , . . . , m E } subject to � Q j , Z � ≤ c j , j ∈ { 1 , . . . , m I } � zz T � z z ∈ { 0 , 1 } n Z = , z T 1 � A 0 � A i � A j 1 1 1 � � � 2 b 0 2 b i 2 b j Q 0 = , Q i = , Q j = 1 2 b T 1 1 2 b T 2 b T 0 c 0 0 0 i j Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 4 / 22

Max-Cut Max-Cut � 1 − x i x j � � maximize w ij 2 ij ∈ E x ∈ {− 1 , 1 } n subject to G = ( V , E ) ( n = | V | ) Equivalent formulation x T Qx maximize (MC) x ∈ {− 1 , 1 } n subject to Q = 1 4 L ∈ S n ( L is the Laplacian matrix of G ) Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 6 / 22

Max-Cut Max-Cut � 1 − x i x j � � maximize w ij 2 ij ∈ E x ∈ {− 1 , 1 } n subject to G = ( V , E ) ( n = | V | ) Equivalent formulation x T Qx maximize (MC) x ∈ {− 1 , 1 } n subject to Q = 1 4 L ∈ S n ( L is the Laplacian matrix of G ) Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 6 / 22

Max-Cut Max-Cut x T Qx maximize (MC) x ∈ {− 1 , 1 } n subject to X = xx T , x ∈ {− 1 , 1 } n ⇐ ⇒ diag( X ) = e , X � 0, rank( X ) = 1 maximize � Q , X � (MC) subject to diag( X ) = e , X � 0 rank( X ) = 1 � Q , X � = � ij Q ij X ij = trace( QX ) diag( X ) = ( X 11 , . . . , X nn ) T , e = (1 , . . . , 1) T X � 0 : X is symmetric and positive semidefinite Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 7 / 22

Semidefinite Relaxation of Max-Cut Semidefinite relaxation � Q , X � maximize (SDP) subject to diag( X ) = e , X � 0 gives an upper bound on Max-Cut: (MC) ≤ (SDP) tight bounds needed for B&B methods to solve Max-Cut efficiently Goemans and Williamson (1995): (SDP) < 1 . 14(MC) can be solved very efficiently by a primal-dual interior-point method bound is too weak to be used to solve Max-Cut efficiently, so we add some triangle inequalities Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 8 / 22

Semidefinite Relaxation of Max-Cut Enhancing the SDP bound � n � There are 4 triangle inequalities (for 1 ≤ i < j < k ≤ n ): 3 X ij + X ik + X jk ≥ − 1 , − X ij + X ik − X jk ≥ − 1 X ij − X ik − X jk ≥ − 1 , − X ij − X ik + X jk ≥ − 1 Choosing a subset I of the inequalities, we have � Q , X � maximize (SDP I ) subject to diag( X ) = e , X � 0 A I ( X ) ≥ − e can greatly enhance the bound: (MC) ≤ (SDP I ) ≤ (SDP) Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 9 / 22

Our Semidefinite Bounds A simple fact � X � 2 F ≤ n 2 diag( X ) = e , X � 0 = ⇒ Our semidefinite bounds � n 2 − � X � 2 � � Q , X � + α maximize F 2 (SDP α I ) subject to diag( X ) = e , X � 0 A I ( X ) ≥ − e for α ≥ 0: (SDP I ) ≤ (SDP α I ) smaller α gives tighter upper bounds: I ) < (SDP α ′ α < α ′ = ⇒ (SDP α I ) lim α → 0 (SDP α I ) = (SDP I ) Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 10 / 22

Our Semidefinite Bounds Dual function F α I ( y , z ) The Lagrangian is ( y ∈ R n , z ∈ R |I| + ): L ( X ; y , z ) := � Q , X � + α � n 2 − � X � 2 � F 2 + � y , e − diag( X ) � + � z , e + A I ( X ) � The dual function is: X � 0 L ( X ; y , z ) F α I ( y , z ) := max = 1 2 F + e T y + e T z + α � � 2 n 2 � X I ( y , z ) � � 2 α � � � Q − Diag( y ) + A ∗ where X I ( y , z ) := I ( z ) + Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 11 / 22

Our Semidefinite Bounds Dual function F α I ( y , z ) The Lagrangian is ( y ∈ R n , z ∈ R |I| + ): I ( z ) , X � − α 2 � X � 2 L ( X ; y , z ) := � Q − Diag( y ) + A ∗ F + e T y + e T z + α 2 n 2 The dual function is: F α I ( y , z ) := max X � 0 L ( X ; y , z ) = 1 2 F + e T y + e T z + α � � 2 n 2 � X I ( y , z ) � � 2 α � � � Q − Diag( y ) + A ∗ where X I ( y , z ) := I ( z ) + Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 11 / 22

Our Semidefinite Bounds Generic semidefinite relaxation maximize � Q , X � subject to A X ≤ a (SDP) B X = b X � 0 diag( X ) = e constraints included in the equality constraints B X = b Generic semidefinite bounds F α ( y , z ) = 1 F + b T y + a T z + α 2 � � 2 n 2 � X ( y , z ) � � 2 α � � � Q − B ∗ ( y ) + A ∗ ( z ) X ( y , z ) = + Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 12 / 22

Our Semidefinite Bounds Computing the best bound F α I ( y , z ) minimize I ( y , z ) F α (DSDP α I ) subject to z ≥ 0 (DSDP α I ) is a smooth convex optimization problem � 1 � ∇ y F α I ( y , z ) = e − diag α X I ( y , z ) � 1 � ∇ z F α I ( y , z ) = e + A I α X I ( y , z ) can be solved with a quasi-Newton method like L-BFGS-B ill-conditioned for very small α Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 13 / 22

Efficiently Computing the Bounds Algorithm: Improved semidefinite bounding procedure Given: Initial vectors and inequalities: y 0 = 0 ∈ R n , I 0 = ∅ , z 0 = 0 ∈ R 0 Scalars: α 1 , ε 1 > 0 Parameters: 0 < θ ≤ 1 and 0 < ρ ≤ 1 For k = 1 , 2 , . . . do: 1 Starting from ( y k − 1 , z k − 1 ), compute ( y k , ˆ z k ) such that �� � � � � � ∇ y F α k � ∇ z F α k � I k − 1 ( y k , ˆ z k ) I k − 1 ( y k , ˆ z k ) < ε k max ∞ , � � � � � � � − ∞ 2 Update I , α , ε : ◮ Add / remove triangle inequalities to get I k and z k ◮ Reduce α : α k +1 ← θα k ◮ Reduce ε : ε k +1 ← ρε k Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 14 / 22

Efficiently Computing the Bounds Convergence Theorem If: ( X k , y k , z k , I k ) generated by Algorithm, where X k := α k X I k ( y k , z k ) 1 α k → 0 and ε k → 0 ( ¯ z , ¯ I ) is an accumulation point of ( X k , y k , z k , I k ) X , ¯ y , ¯ Then: ( ¯ X , ¯ y , ¯ z ) is a primal-dual optimal solution for (SDP ¯ I ) the bounds converge to (SDP ¯ I ): k →∞ F α k I k ( y k , z k ) = (SDP ¯ lim I ) Furthermore, if all violated inequalities are added (in the limit), we have k →∞ F α k I k ( y k , z k ) = (SDP I all ) lim Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 15 / 22

Comparing to Biq Mac Our bounds: F α I ( y , z ) 2 � � � � 1 Q − Diag( y ) + A ∗ 2 n 2 F + e T y + e T z + α I ( y , z ) = I ( z ) F α � � 2 α � � + F α I is convex and smooth evaluate F α I ( y , z ) and ∇ F α I ( y , z ) by computing a partial eigenvalue decomposition minimize F α I ( y , z ) by a quasi-Newton method Biq Mac bounds: θ I ( z ) θ I ( z ) = e T z � Q + A ∗ I ( z ) , X � + minimize subject to diag( X ) = e , X � 0 θ I is convex and nonsmooth evaluate θ I ( z ) and φ ∈ ∂θ I ( z ) by solving an SDP minimize θ I ( z ) by a bundle method Nathan Krislock (PIMS, UBC) BiqCrunch: SDP-based solver for BQP CanaDAM 2013 16 / 22