Liege University: Francqui Chair 2011-2012 Lecture 4: Nonlinear - PowerPoint PPT Presentation

Liege University: Francqui Chair 2011-2012 Lecture 4: Nonlinear analysis of combinatorial problems Yurii Nesterov, CORE/INMA (UCL) March 16, 2012 Yu. Nesterov () Nonlinear analysis of combinatorial problems 1/24 March 16, 2012 1 / 24

Outline 1 Boolean quadratic problem 2 Simple bounds 3 SDP-relaxation and its quality 4 General constraints 5 Generating functions of integer sets 6 Knapsack volume 7 Fast computations 8 Further extensions Yu. Nesterov () Nonlinear analysis of combinatorial problems 2/24 March 16, 2012 2 / 24

Boolean quadratic problem Let Q = Q T be an ( n × n )-matrix. find f ∗ ( Q ) ≡ max x {� Qx , x � : x i = ± 1 , i = 1 . . . n } . Maximization: Minimization: find f ∗ ( Q ) ≡ min x {� Qx , x � : x i = ± 1 , i = 1 . . . n } . Clearly f ∗ ( − Q ) = − f ∗ ( Q ). Trivial Properties Both problems are NP-hard. They can have up to 2 n local extremums. Very often we are happy with approximate solutions Yu. Nesterov () Nonlinear analysis of combinatorial problems 3/24 March 16, 2012 3 / 24

Simple bounds: Eigenvalues For any x ∈ R n with x i = ± 1, we have � x � 2 = n . Upper bound. Therefore, f ∗ ( Q ) ≤ � x � 2 = n � Qx , x � = n · λ max ( Q ). max Lower bounds. 1. If Q � 0, then f ∗ ( Q ) = max | x i |≤ 1 � Qx , x � ≥ � x � 2 =1 � Qx , x � = λ max ( Q ). max 2. Consider random x with Prob ( x i = 1) = Prob ( x i = − 1) = 1 2 . Then n f ∗ ( Q ) ≥ E x ( � Qx , x � ) = � Q i , j E x ( x i x j ) i , j =1 n � = Q i , i = Trace ( Q ). i =1 Example: Q = ee T , Trace ( Q ) = λ max ( Q ) = n . In both cases, relative quality is n . Yu. Nesterov () Nonlinear analysis of combinatorial problems 4/24 March 16, 2012 4 / 24

Polyhedral bound For Boolean x ∈ R n , we have n | Q i , j | def � Qx , x � = � Q i , j x i x j ≤ � = � Q � 1 . i , j =1 i , j How good is it? Random hyperplane technique. (Krivine 70’s, Goemans, Williamson 95) Let us fix V ∈ M n . Consider the random vector ξ = sgn [ V T u ] with random u ∈ R n , uniformly distributed on unit sphere. ([ · ] denotes component-wise operations.) � v i , v j � Lemma1: E ( ξ i ξ j ) = 2 π arcsin � v i �·� v j � . Lemma 2: For X � 0, we have arcsin[ X ] � X . 6 [ X ] 3 + 3 40 [ X ] 5 + . . . � X . Proof: arcsin[ X ] = X + 1 Yu. Nesterov () Nonlinear analysis of combinatorial problems 5/24 March 16, 2012 5 / 24

Quality of polyhedral bound ( Q � 0) Let Q = V T V (this means that Q i , j = � v i , v j � ). Then n � � Q ( i , j ) arcsin Q ( i , j ) def 2 2 f ∗ ( Q ) ≥ E ( � Q ξ, ξ � ) = √ � = π ρ . π Q ( i , i ) Q ( j , j ) i , j =1 Denote D = diag ( Q ) − 1 / 2 . Then ρ ≥ � Q , DQD � M . Denote S 1 = � Q , I n � M , S 2 = � | Q i , j | . Then S 1 + S 2 = � Q � 1 . Thus, i � = j ( Q i , j ) 2 S 2 √ √ � Q , DQD � M = S 1 + � Q i , i Q j , j ≥ S 1 + 2 � Q i , i Q j , j i � = j i � = j S 2 S 2 S 2 = S 1 + 2 ≥ S 1 + nS 1 − S 1 = � Q � 1 − S 2 + 2 ( n − 1)( � Q � 1 − S 2 ) . 2 � n √ � 2 � Q i , i − S 1 i =1 1 The minimum is attained for S 2 = � Q � 1 · (1 − √ n ). Thus, 2 � Q � 1 ≥ f ∗ ( Q ) ≥ � Q , DQD � M ≥ 1+ √ n � Q � 1 . It is better than the eigenvalue bound! Yu. Nesterov () Nonlinear analysis of combinatorial problems 6/24 March 16, 2012 6 / 24

SDP-bounds: Primal Relaxation (Lov´ asz) For X , Y ∈ M n , we have � XY , Z � M = � X , ZY T � M = � Y , X T Z � M . Denote 1 k n : (1 k n ) j = ± 1 , j = 1 . . . n , k = 1 . . . 2 n . Then � Q 1 k n , 1 k n � = � Q , 1 k n (1 k n ) T � M . Therefore f ∗ ( Q ) = max X ∈P n � Q , X � M , def = Conv { 1 k n (1 k n ) T , k = 1 . . . 2 n } . where P n Note that: The complete description of P n is not known. For X ∈ P n we have: X � 0, and d ( X ) = 1 n . Thus, f ∗ ( Q ) ≤ max {� Q , X � M : X � 0 , d ( X ) = 1 n } . Yu. Nesterov () Nonlinear analysis of combinatorial problems 7/24 March 16, 2012 7 / 24

Dual Relaxation (Shor) Problem: f ∗ ( Q ) = max x {� Qx , x � : x 2 i = 1 , i = 1 . . . n } . n ξ i (1 − ( x i ) 2 ). Its Lagrangian is L ( x , ξ ) = � Qx , x � + � Therefore i =1 f ∗ ( Q ) = max min ξ L ( x , ξ ) ≤ min ξ max L ( x , ξ ) x x ξ {� 1 n , ξ � : Q � D ( ξ ) } def = s ∗ ( Q ). = min Note: Both relaxations give exactly the same upper bound: s ∗ ( Q ) = min ξ max X � 0 {� 1 n , ξ � + � X , Q − D ( ξ ) � M } . = max X � 0 min ξ {� 1 n − D ( X ) , ξ � + � X , Q � M } . = max X � 0 {� X , Q � M : d ( X ) = 1 n } . Any hope? (Looks as an attempt to approximate Q by D ( ξ ).) Yu. Nesterov () Nonlinear analysis of combinatorial problems 8/24 March 16, 2012 8 / 24

Trigonometric form of Quadratic Boolean Problem We have seen that f ∗ ( Q ) ≥ 2 π arcsin[ V T V ] with d ( V T V ) = 1 n . Let us show that 2 f ∗ ( Q ) = max π � Q , arcsin[ V T V ] � M . � v i � =1 Let x ∗ be the global solution. Proof: Choose arbitrary a , � a � = 1. Define v i = a if x ∗ i = 1, and v i = − a otherwise. Then V T V = x ∗ ( x ∗ ) T and 2 π arcsin[ V T V ] = x ∗ ( x ∗ ) T . Since { X = V T V : d ( X ) = 1 n } ≡ { X � 0 : d ( X ) = 1 n } , we get � 2 f ∗ ( Q ) = max � π � Q , arcsin[ X ] � M : d ( X ) = 1 n . X � 0 Corollary: s ∗ ( Q ) ≥ f ∗ ( Q ) ≥ 2 π s ∗ ( Q ). Relative accuracy does not depend on dimension! Yu. Nesterov () Nonlinear analysis of combinatorial problems 9/24 March 16, 2012 9 / 24

General constraints on squared variables Consider two problems: φ ∗ = max {� Qx , x � : [ x ] 2 ∈ F} , φ ∗ = min {� Qx , x � : [ x ] 2 ∈ F} , where F is a bounded closed convex set. Trigonometric form: φ ∗ = max { 2 π � D ( d ) QD ( d ) , arcsin[ X ] � : X � 0 , d ( X ) = 1 n , d ≥ 0 , [ d ] 2 ∈ F} , φ ∗ = min { 2 π � D ( d ) QD ( d ) , arcsin[ X ] � : X � 0 , d ( X ) = 1 n , d ≥ 0 , [ d ] 2 ∈ F} . Relaxations: Define the support function ξ ( u ) = max {� u , v � : v ∈ F} , and ψ ∗ = min { ξ ( u ) : D ( u ) � Q } , ψ ∗ = max {− ξ ( u ) : Q + D ( u ) � 0 } , τ ∗ = ξ ( d ( Q )), τ ∗ = − ξ ( − d ( Q )). Simple relations: ψ ∗ ≤ φ ∗ ≤ τ ∗ ≤ τ ∗ ≤ φ ∗ ≤ ψ ∗ . Yu. Nesterov () Nonlinear analysis of combinatorial problems 10/24 March 16, 2012 10 / 24

Main result Denote ψ ( α ) = αψ ∗ + (1 − α ) ψ ∗ , and β ∗ = ψ ∗ − τ ∗ ψ ∗ − ψ ∗ , β ∗ = τ ∗ − ψ ∗ ψ ∗ − ψ ∗ . Theorem. 1. Let α ∗ = max { 2 π ω ( β ∗ ) , 1 − β ∗ } , and α ∗ = min { 1 − 2 π ω ( β ∗ ) , β ∗ } , √ ( ≥ 1 + 1 1 − α 2 2 α 2 ). where ω ( α ) = α arcsin( α ) + ψ ∗ ≤ φ ∗ ≤ ψ ( α ∗ ) ≤ ψ ( α ∗ ) ≤ φ ∗ ≤ ψ ∗ . Then 2. 0 ≤ φ ∗ − ψ ( α ∗ ) ≤ 24 49 . φ ∗ − φ ∗ α = α ∗ (2 − α ∗ ) − α ∗ Then | φ ∗ − ψ (¯ α ) | ≤ 12 3. Define ¯ 1+ α ∗ − 2 α ∗ . 37 . φ ∗ − φ ∗ Yu. Nesterov () Nonlinear analysis of combinatorial problems 11/24 March 16, 2012 11 / 24

Main limitation: Absence of linear constraints Example. Let β > 0. Consider the problem φ ∗ = max x {� Qx , x � : [ x ] 2 = 1 n , � c , x � = β } , x {� Qx , x � : [ x ] 2 = 1 n , � c , x � = β } . φ ∗ = min Natural relaxation: ψ ∗ = max X {� Q , X � : d ( X ) = 1 n , X � 0 , � Xc , c � = β 2 } , X {� Q , X � : d ( X ) = 1 n , X � 0 , � Xc , c � = β 2 } . ψ ∗ = min Denote by v any vector with [ v ] 2 = 1 n . Assumptions: 1. There exists a unique v ∗ such that � c , v ∗ � = β . 2. There exist v − and v + such that 0 < � c , v − � < β < � c , v + � . Note: in this case φ ∗ = φ ∗ (unique feasible solution). Yu. Nesterov () Nonlinear analysis of combinatorial problems 12/24 March 16, 2012 12 / 24

Consider the polytope P n = Conv { V i = v i v T i , i = 1 , . . . , 2 n } . Lemma. Any V i is an extreme point of P n . Any pair V i , V j is connected by an edge. Note: 1. In view of our assumption ∃ ˜ V ∈ P n : ˜ � ˜ V = α v − v T − + (1 − α ) v + v T V c , c � = β 2 . + , α ∈ (0 , 1) , 2. P n ⊂ { X : d ( X ) = 1 n , X � 0 } . Conclusion: We can choose Q : ψ ∗ > φ ∗ . Since ψ ∗ ≤ φ ∗ , the relative accuracy of ψ ∗ is + ∞ . Reason of the troubles: We intersect edges of P n . This cannot happen if β = 0 . Yu. Nesterov () Nonlinear analysis of combinatorial problems 13/24 March 16, 2012 13 / 24

Further developments Boolean quadratic optimization with m homogeneous linear equality constraints (accuracy O (ln m )). Quadratic maximization with quadratic inequality constraints (accuracy O (ln m )). Main bottleneck: absence of cheap relaxations. Yu. Nesterov () Nonlinear analysis of combinatorial problems 14/24 March 16, 2012 14 / 24

Generating functions of integer sets 1. Primal generating functions. x α , For set S ⊂ Z n , define f ( S , x ) = � α ∈ S n where x α = x α i � i . i =1 f ( S , 1 n ) = N ( S ), the integer volume of S . Can be used for counting problems. Sometimes have short representation . Example: S = { x ∈ Z : x ≥ 0 } . Then 1 f ( S , x ) = 1 − x . Yu. Nesterov () Nonlinear analysis of combinatorial problems 15/24 March 16, 2012 15 / 24