Finding Infimum Point with Respect to the Second Order Cone In - PowerPoint PPT Presentation

Finding Infimum Point with Respect to the Second Order Cone In honor of Don Goldfarb Marta Cavaleiro and Farid Alizadeh Management Science and Information Systems Department Rutgers University U.S. and Mexico Workshop on Optimization and its Applications Huatulco, Mexico January 2018

The Dual Simplex Method for a Special SOCP Problem The Simplex Method has been extended to convex Quadratic Programming decades ago (Franke-Wolfe 55) (Goldfarb-Idnani 83) gave a practical dual algorithm (our research is inspired partly by their work) The simplex method can be extended to a large class of LP-Type problems (Matousek, Sharir, Welzl 96) Competitiveness and contrast to Interior Point Methods

Simplex vs Interior point methods, why simplex? Reminder: For linear optimization: ◮ Interior point (IP) methods usually have to solve a full-fledged linear system per iteration, but have a small number of iterations ◮ In the simplex method a low rank update of a previously solved system must be found, but the number of iterations is large ◮ IP methods are better for parallel implementation, and sparse systems ◮ Simplex is better for warm-start, and for cases where constraints arrive in a stream ◮ Dual simplex is also generally more suitable for branch and bound and similar procedures A Similar situation exists for problems more general than linear optimization

Infimum with respect to the Second-Order cone � � x = ( x 0 ; x ) ∈ R d : � x � 2 ≤ x 0 Let Q be the second-order cone Q := We define the infimum of a set of points P = { p 1 , . . . , p m } ⊂ R d with respect to Q as: � � Inf Q ( P ) := max x 0 = � e 0 , x � x s.t. x � Q p i , i = 1, ..., m with e 0 = ( 1, 0, . . . , 0 ) ⊤ . Fig 1. Example in R 3 with 6 points. It does not seem that this problem is a QP.

Equivalence to the Smallest Enclosing Ball of Balls Lemma B ( c 1 , r 1 ) ⊆ B ( c 2 , r 2 ) iff � c 2 − c 1 � ≤ r 2 − r 1 . Fig 2. Consider a SOC with vertex at each p i . Fig 3. View from the top.

The smallest ball containing a set of balls: � � max x 0 = radius x s.t. � p i − x � ≤ p i0 − x 0 , i = 1, ..., m � x 0 � � � p i0 But � p i − x i � ≤ p i0 − x 0 � Q ⇔ x p i The smallest enclosing ball of balls is an “LP-type” problem (Matou˘ sek, Sharir & Welzl (1996)) Previous work: Megiddo (1989); Welzl (1991); Chazelle and Matou˘ sek (1996); B˘ adoiu et al. (2002); Fischer and G¨ artner (2003); Kumar et al. (2003); Zhou et al (2005).

Duality and Complementary Slackness Complementary slackness: � p i − x, y i � = 0, for i = 1, . . . , m. Dual problem: with x and y i , i = 1, ..., m , be the optimal primal m � and dual solutions, respectively. min � p i , y i � y ◮ if x ≺ Q p i then y i = 0 ; i = 1 m ◮ if y i ≻ Q 0 then x = p i (which can happen at � s.t. y i = e 0 most once); i = 1 ◮ if p i − x ∈ ∂ Q and y i ∈ ∂ Q then y i � Q 0, i = 1, . . . , m y i0 ( p − x ) + ( p i0 − x 0 ) y i = 0 y i0 y i = p i0 − x 0 ( x − p i ) . ⇔ Theorem x is the optimal solution to the primal problem iff x � Q p i , i = 1, ..., m , and x ∈ conv ( p i : � p i − x � = p i0 − x 0 ) .

Fig 4. View from the top. The center is in the convex hull of points on the boundary, so it is optimal

The concept of basis Based on the concept for LP-type problems Matou˘ sek, Sharir & Welzl (1996) ◮ Let P be the set of all points, and P 1 ⊆ P ◮ Define w ( P 1 ) = Inf Q ( P 1 ) B ′ ⊂ B . ◮ A subset B ⊆ P 1 is a basis if w ( B ′ ) > w ( B ) for all ◮ A basis contains at least 2 points and at most d affinely independent points ◮ B ⊆ P 1 is a basis for Inf Q ( P 1 ) problem if B is affinely independent, and where the optimal x satisfies x ∈ ri conv ( B ) , with B = { p i : p i ∈ B } ◮ The points on a basis B reside on the boundary ∂ ( Q + x )

Given a basis, how to find x? � p i − x � 2 − ( p i0 − x 0 ) 2 = � p 1 − x � 2 − ( p 10 − x 0 ) 2 , ∀ p i ∈ B \ { p 1 } and x ∈ aff ( B ) � � B T � � � b + x 0 c � 2 −( p 10 − x 0 ) 2 = 0 � p 1 − A − 1 w ( x 0 ) � � x = and N T N T p 1 � �� � � �� � A w ( x 0 ) with N a basis for Null ( Sub ( B ∪ { p ∗ } )) , B = 2 � � p 1 − p 1 , ..., p | B | − p 1 , � p 1 � 2 − p 2 10 − � p 2 � 2 + p 2    p 10 − p 20  20 . .   . . c = 2  and b =   . .     � 2 + p 2  � p 1 � 2 − p 2 p 10 − p | B | 0 � � 10 − � p | B | | B | 0

The dual variables given a basic solution A basic solution corresponds to a dual feasible solution. Consider x , the solution to Inf Q ( B ) , with B ⊆ P 1 a basis. We know that: � � x ∈ conv ( { p i : p i ∈ B } ) so ∃ α i ≥ 0 s.t. x = α i = 1, α i p i , p i ∈B i and α i ’s are unique. The corresponding dual variables are: • y i for i : p i ∈ B is such that: α i ( p i0 − x 0 ) y i0 y i0 = and y i = ( p i − x ) , � j α j ( p j0 − x 0 ) p i0 − x 0 • y i = 0 for i : p i �∈ B , which are feasible for the dual problem and satisfy the complementary slackness conditions.

A Dual Simplex Algorithm Based on Dearing and Zeck’s dual algorithm (2009) 0. Initialization: It starts with x , the solution Inf Q ( B ) for some basis B (it is easy to find a basis for a set of two points). 1. Check optimality: If x is primal feasible, then x is the optimal solution to Inf Q ( P ) . Else pick p ∗ primal infeasible. 2. Solve Inf Q ( B ∪ { p ∗ } ) : Move x ”towards” the feasibility of p ∗ , such that the following invariants are maintained: ◮ The corresponding dual solution is always feasible. ◮ Complementary slackness is satisfied, that is, the primal constraints corresponding to the basis are binding. At the end, we have a new basis for the problem Inf Q ( B ∪ { p ∗ } ) , which is obtained by possibly having to remove some points from the old basis, and by adding p ∗ . A new iteration then starts

Movement along a curve The “curve” is parametrized by t is as follows ◮ � p i − x ( t ) � = p i0 − x 0 ( t ) for all p i ∈ B � B ∪ { p ∗ } � ◮ x ( t ) ∈ aff And the search is restricted to the polyhedron � � x 0 � � � B ∪ { p ∗ } � C = | x ∈ conv x ダ 二 二 二 二 二二 二 二 三 垄 目 コ 蜘 . 三

Two scenarios are possible ◮ By moving along this curve, we reduce x 0 enough to make p ∗ become feasible and at ∂ Q , and x new ∈ C . In this case the pivot is complete and � p ∗ � B new = B ∪ ◮ Or before p ∗ is absorbed into Q , the curve hits the wall of C . In this case one of the points p i whose dual variable y i is about to become infeasible must leave the basis: B ′ B \ { p i } where y i = 0 ← new B ′ new ∪ { p ∗ } � � C new conv ← The curve will now move in the affine space spanned by C new This may have to be repeated several times before p ∗ becomes feasible (Similar to Goldfarb & Idnani for QP)

The curve x ( t ) x moves along the curve ∆ x ( t ) : R → R d − 1 which has the following properties: ◮ Primal constraints of B are binding (complementary slackness is kept): � p i − ( x + ∆ x ( t )) � − p i0 = � p 1 − ( x + ∆ x ( t )) � − p 10 , p i ∈ B \ { p 1 } � B T ( x ( t ) + ∆ x ( t )) = b + x 0 ( t ) c � p 1 − ( x ( t ) + ∆ x ( t )) � 2 = ( p 10 − x 0 ( t )) 2 ◮ Dual feasibility of � m i = 1 y i ( t ) = e 0 is kept: x + ∆ x ( t ) ∈ aff ( B ∪ { p ∗ } ) � N T ( x ( t ) + ∆ x ( t )) = N T p ∗ N is a basis for Null ( Sub ( B ∪ { p ∗ } )) . ◮ We wish to move towards feasibility of p ∗ .

Fig 4. ∆ x ( t ) moving in C .

What if ∆ x ( t ) = 0 ? This happens when x is the only point such that the primal constraints are binding for the points in B , that is | B | = d . When this happens, a point needs to be removed from the basis: ◮ p k ∈ B such that x ∈ conv ( { p j : p j ∈ B \ { p k } ∪ { p ∗ } ) This rule ensures that the dual variables corresponding to x (which are now different from before) are still dual feasible.

The dual variables for x + ∆ x ( t ) � � x + ∆ x ( t ) ∈ aff ( B ∪ { p ∗ } ) so ∃ α j s.t. x + ∆ x ( t ) = α j p j , α j = 1. p j ∈B∪ { p ∗ } p j ∈B∪ { p ∗ } The corresponding dual variables are α i ( p i0 − x 0 ( t )) y i0 p i0 − x 0 ( p i −( x + ∆ x ( t ))) , i : p i ∈ B∪ { p ∗ } � y i0 ( t ) = y i ( t ) = p j ∈B∪ { p ∗ } α j ( p j0 − x 0 ( t )) , y i ( t ) = 0, i : p i �∈ B ∪ { p ∗ } � m and these always satisfy y i ( t ) = e 0 for all t . i = 1 If α i < 0 then y i ≻ Q 0 , so y i becomes dual infeasible. This tells us how far we can move along ∆ x ( t ) : until we hit one face of conv ( B ∪ { p ∗ } ) .