Lift-and-project hierarchies for combinatorial problems Monique - PowerPoint PPT Presentation

Lift-and-project hierarchies for combinatorial problems Monique Laurent CWI, Amsterdam & Tilburg University MAP 2012, Konstanz September 19, 2012

Typical combinatorial optimization problem: max c T x s.t. Ax ≤ b , x ∈ { 0 , 1 } n LP relaxation: P := { x ∈ R n | Ax ≤ b } Integral polytope to be found: P I := conv ( P ∩ { 0 , 1 } n ) Goal: Procedure to construct a tighter, tractable relaxation P ′ such that P I ⊆ P ′ ⊆ P leading to P I after finitely many iterations.

Cutting planes atal closure of P = { x ∈ R n : Ax ≤ b } : Gomory-Chv´ P ′ = { x | u T Ax ≤ ⌊ u T b ⌋ ∀ u ≥ 0 with u T A integer } . P ′ is a polyhedron. P I is found after finitely many iterations. [Chv´ atal 1973] O ( n 2 log n ) iterations suffice if P ⊆ [0 , 1] n . [Eisenbrand-Schulz 1999] But optimization over P ′ is hard! [Eisenbrand 1999]

This talk: Lift-and-project methods We present several techniques to construct a hierarchy of LP/SDP relaxations: P ⊇ P 1 ⊇ . . . ⊇ P n = P I . � Balas-Ceria-Cornu´ ejols hierarchy [1993] LP � Lov´ asz-Schrijver N / N + operators [1991] LP / SDP � Sherali-Adams hierarchy [1990] LP � Lasserre hierarchy [2001] SDP Common feature: One can optimize in polynomial time over P t for any fixed t . Comparison: SA ⊆ LS ⊆ BCC ⊆ SA ∩ LS + Las

Great interest recently in such hierarchies: Polyhedral combinatorics: How many rounds are needed to find P I ? Which valid inequalities are satisfied after t rounds? New tractable instances? Proof systems: Use hierarchies as a model to generate inequalities and show e.g. P I = ∅ . Complexity theory: What is the integrality gap after t rounds? Can one use the hierarchy to get improved tractable approximations? Link to hardness of the problem? Common background for the hierarchies: Moment theory and sums of squares of polynomials.

Plan of the lecture Balas-Ceria-Cornu´ ejols, Lov´ sz-Schrijver, Sherali-Adams constructions. Full lifting and moment matrices Lasserre hierarchy Application to matchings, stable sets, knapsack, max-cut Copositive hierarchy

Some notation P = { x ∈ R n : Ax ≤ b } Homogenize P to the cone: P = { ( x 0 , x ) ∈ R n +1 : bx 0 − Ax ≥ 0 } ˜ = { y ∈ R n +1 : g ℓ T y ≥ 0 ( ℓ = 1 , · · · , m ) } a T writing Ax ≤ b ℓ x ≤ b ℓ ( ℓ = 1 , · · · , m ) as � b ℓ � and setting g ℓ = . − a ℓ

Lift-and-project strategy 1. Generate new constraints: Multiply the system Ax ≤ b by products of the constraints x i ≥ 0 and 1 − x i ≥ 0. � Polynomial system in x . 2. Linearize (and lift ) by introducing new variables y I for i ∈ I x i and setting x 2 products � i = x i . � Linear system in ( x , y ). 3. Project back on the x -variable space. � LP relaxation P ′ satisfying P I ⊆ P ′ ⊆ P . The methods vary in the choice of the multipliers and of iterating.

The Balas-Ceria-Cornu´ ejols construction 1. Multiply the system Ax ≤ b by x 1 and 1 − x 1 : x 1 ( b − Ax ) ≥ 0 , (1 − x 1 )( b − Ax ) ≥ 0 2. Linearize: Set y i = x 1 x i , identify y 1 = x 1 and get the lift : M 1 = { ( x , y ) : y 1 = x 1 , bx 1 − Ay ≥ 0 , b (1 − x 1 ) − A ( x − y ) ≥ 0 } 3. Project M 1 back to the x -subspace and get P 1 such that P I ⊆ P 1 ⊆ P . 4. Iterate: use variable x 2 starting from P 1 and get P 12 , etc. Lemma P 1 = conv ( P ∩ { x : x 1 = 0 , 1 } ) . x 1 + (1 − x 1 ) x − y y Pf: “ ⊆ ”: Write x ∈ P 1 as x = x 1 1 − x 1 . “ ⊇ ”: x ∈ P ∩ { x : x 1 = 0 , 1 } = ⇒ ( x , x 1 x ) ∈ M 1 = ⇒ x ∈ P 1 . Corollary Find P I after n steps.

The Lov´ asz-Schrijver construction: N -operator 1. Multiply Ax ≤ b by x i , 1 − x i ∀ i ∈ [ n ] and get the system: T � 1 � � 1 � ( b ℓ − a T ℓ x ) x i = g T e i ≥ 0 ∀ ℓ, ℓ x x T � 1 � � 1 � ( b ℓ − a T ℓ x )(1 − x i ) = g T ( e 0 − e i ) ≥ 0 ∀ ℓ. ℓ x x T � 1 � � 1 � 2. Linearize: The new matrix variable Y = belongs to x x M ( P ) = { Y ∈ S n +1 | Y 0 i = Y ii , Ye i , Y ( e 0 − e i ) ∈ ˜ P ∀ i ∈ [ n ] } , 3. Project: � � 1 � � x ∈ R n | ∃ Y ∈ M ( P ) s.t. N ( P ) = = Ye 0 x

The Lov´ asz-Schrijver construction: N + -operator 1. Multiply Ax ≤ b by x i , 1 − x i ∀ i ∈ [ n ] and get the system: T � 1 � � 1 � ( b ℓ − a T ℓ x ) x i = g T e i ≥ 0 ∀ ℓ, ℓ x x T � 1 � � 1 � ( b ℓ − a T ℓ x )(1 − x i ) = g T ( e 0 − e i ) ≥ 0 ∀ ℓ. ℓ x x T � 1 � � 1 � 2. Linearize: The new matrix variable Y = belongs to x x M ( P ) = { Y ∈ S n +1 | Y 0 i = Y ii , Ye i , Y ( e 0 − e i ) ∈ ˜ P ∀ i ∈ [ n ] } , M + ( P ) = M ( P ) ∩ S + n +1 . 3. Project: � � 1 � � x ∈ R n | ∃ Y ∈ M + ( P ) s.t. N + ( P ) = = Ye 0 x

Properties of the N - and N + -operators + ( P ) = N + ( N t − 1 0. Iterate: N t ( P ) = N ( N t − 1 ( P )), N t ( P )). + 1. P I ⊆ N + ( P ) ⊆ N ( P ) ⊆ P . � 2. N ( P ) ⊆ conv ( P ∩ { x | x i = 0 , 1 } ). i ∈ [ n ] 3. N n ( P ) = P I . 4. If one can optimize in polynomial time over P , then the same holds for N t ( P ) and for N t + ( P ) for any fixed t . Example For the ℓ 1 -ball centered at e / 2: � � x ∈ R V | � i ∈ V \ I (1 − x i ) ≥ 1 i ∈ I x i + � P = 2 ∀ I ⊆ V , 2 e ∈ N n − 1 P I = ∅ , but 1 ( P ). + Hence, n iterations of the N + operator are needed to find P I .

Application to stable sets P = FR ( G ) = { x ∈ R V + | x i + x j ≤ 1 ( ij ∈ E ) } P I = STAB ( G ): stable set polytope of G = ( V , E ). 1. Y ∈ M ( FR ( G )) = ⇒ y ij = 0 for all edges ij ∈ E . 2. The clique inequality : � i ∈ Q x i ≤ 1 is valid for N + ( FR ( G )), but its N -rank is | Q | − 2. � SDP helps! i ∈ V ( C ) x i ≤ | C |− 1 3. The odd circuit inequalities : � 2 are valid for N ( FR ( G )) and they determine it exactly . n 4. α ( G ) − 2 ≤ N -rank ≤ n − α ( G ) − 1 . 5. N + -rank ≤ α ( G ) [tight for G = line graph of K 2 p +1 ]

The Sherali-Adams construction 1. New polynomial constraints: • x I (1 − x ) W \ I ( b − Ax ) ≥ 0 for I ⊆ W with | W | = t . • x I (1 − x ) U \ I ≥ 0 for I ⊆ U with | U | = t + 1. 2. Linearize & lift: Introduce new variables y U for all U ∈ P t +1 ( V ), setting y i = x i ( x 2 i = x i ). 3. Project back on x -variables space and get SA t ( P ). Lemma SA 1 ( P ) = N ( P ) . SA t ( P ) ⊆ N t ( P ) .

Full lifting �� � y x = ∈ { 0 , 1 } P ( V ) x ∈ { 0 , 1 } n x i � i ∈ I I ⊆ V y x = (1 , x 1 , .., x n , x 1 x 2 , .., x n − 1 x n , .., � x i ) i ∈ V   Y = y x ( y x ) T = � � x i x j �  i ∈ I j ∈ J I , J ⊆ V If x ∈ P ∩ { 0 , 1 } n then Y = y x ( y x ) T satisfies: 1. Y ( ∅ , ∅ ) = 1 . 2. Y ( I , J ) depends only on I ∪ J � moment matrix 3. Y � 0 . 4. g ℓ ( x ) Y � 0 � localizing moment matrix These conditions characterize conv ( y x : x ∈ P ∩ { 0 , 1 } n ), thus P I .

Full lifting via moment matrices Definition Given y ∈ R P ( V ) define: 1. The moment matrix M V ( y ) = ( y I ∪ J ) I , J ∈P ( V ) . 2. The shifted vector g ∗ y = ( y I + � i g i y I ∪{ i } ) I ∈P ( V ) . [linearize g ( x ) y x = ( g ( x ) x I ) I ] 3. The localizing moment matrix M V ( g ∗ y ). Theorem 1. conv ( y x ( y x ) T : x ∈ P ∩ { 0 , 1 } ) is equal to ∆ P = { y ∈ R P ( V ) : y ∅ = 1 , M V ( y ) � 0 , M V ( g ℓ ∗ y ) � 0 ∀ ℓ } . 2. P I is the projection of ∆ P . 3. ∆ P is a polytope.

Proof Definition Let Z be the matrix with columns y x for x ∈ { 0 , 1 } n . Recall: ∆ P = { y ∈ R P ( V ) : y ∅ = 1 , M V ( y ) � 0 , M V ( g ℓ ∗ y ) � 0 ∀ ℓ } . Lemma = { y ∈ R P ( V ) : y ∅ = 1 , Z − 1 y ≥ 0 , ( Z − 1 y ) J = 0 if χ J �∈ P } ∆ P = conv ( y x : x ∈ P ∩ { 0 , 1 } n ) . Proof: M V ( y ) = Z diag( Z − 1 y ) Z T . 1. Z diagonalizes M V ( y ): ⇒ Z − 1 y ≥ 0. Thus: M V ( y ) � 0 ⇐ ⇒ ( Z − 1 y ) J g ℓ ( χ J ) ≥ 0 for all J . 2. M V ( g ℓ ∗ y ) � 0 ⇐

Case n = 2 Z is the 0 / 1 matrix indexed by P ( V ) with Z − 1 ( I , J ) = ( − 1) | J \ I | if I ⊆ J, 0 otherwise. Z ( I , J ) = 1 , ∅ 1 2 12 ∅ 1 2 12     ∅ 1 1 1 1 ∅ 1 − 1 − 1 1 1 0 1 0 1 1 0 1 0 − 1 Z − 1 =     Z = �     2 0 0 1 1 2 0 0 1 − 1     12 0 0 0 1 12 0 0 0 1    y 0 y 1 y 2 y 12 y ∅ − y 1 − y 2 + y 12 ≥ 0   y 1 − y 12 ≥ 0 y 1 y 1 y 12 y 12    M V ( y ) =  � 0 ⇐ ⇒   y 2 y 12 y 2 y 12 y 2 − y 12 ≥ 0    y 12 ≥ 0 y 12 y 12 y 12 y 12 

Example    y ∅ y 1 y 2 y 12 y ∅ − y 1 − y 2 + y 12 ≥ 0   y 1 − y 12 ≥ 0 y 1 y 1 y 12 y 12    M V ( y ) =  � 0 ⇐ ⇒   y 2 y 12 y 2 y 12 y 2 − y 12 ≥ 0    y 12 ≥ 0 y 12 y 12 y 12 y 12  Consider � � ( x 1 , x 2 ) : g ( x ) = 3 P = 2 − x 1 − x 2 ≥ 0 . ( g ∗ y ) ∅ = 3 2 y ∅ − y 1 − y 2 , ( g ∗ y ) 1 = 3 2 y 1 − y 1 − y 12 = 1 2 y 1 − y 12 , ( g ∗ y ) 2 = 1 2 y 2 − y 12 , ( g ∗ y ) 12 = 3 2 y 12 − y 12 − y 12 = − 1 2 y 12 . ( g ∗ y ) ∅ − ( g ∗ y ) 1 − ( g ∗ y ) 2 + ( g ∗ y ) 12 = 3 2( y ∅ − y 1 − y 2 ) . M V ( y ) , M V ( g ∗ y ) � 0 ⇐ ⇒ y 12 = 0 , y 1 , y 2 ≥ 0 , y ∅ − y 1 − y 2 ≥ 0 .