How This Talk Came to Be... January 9, 2003 Were thinking of - PowerPoint PPT Presentation

A Triangular-Based Branch and Bound Method for Nonconvex Quadratic Programming and the Computational Grid J EFF L INDEROTH Industrial and Systems Engineering Lehigh University jtl3@lehigh.edu A RGONNE G LOBAL O PTIMIZATION T HEORY I NSTITUTE S EPTEMBER 9, 2003

How This Talk Came to Be... January 9, 2003 ‘‘We’re thinking of writing an NSF proposal. What are you working on these days, Jeff?’’ ‘‘I have begun preliminary work on a branch-and-bound method for a global optimization problem that relies on (convex) quadratic relaxations. Having a simple API to be able to build the nonlinear relaxations on the fly during the branch-and-bound process would be something very useful for this problem’’

The Fundamental Theorem of email Theorem 1. Mentioning a topic off-handedly in email about a subject you are planning on pursuing in research does not make you an expert in the �eld. Theorem 2. Mentioning by email that you �have begun preliminary work� on a subject doesn't mean that you will have anything useful to say about that subject in nine months Proofs. (By picture) Q . E . D .

Jeff's Main Summer Activities Golf Feeding New Son Jacob ⋆ Parallel B&B for (non)convex QCQP not a top summer priority

Outline • Nonconvex Quadratic Programming ⋄ Relaxations with convex/concave envelopes of bilinear functions ⋄ Formulae for envelopes over triangles ⋄ Why triangles are good ⋄ How not to solve the resulting relaxations • The Computational Grid ⋄ Brief Introduction ⋄ Branch-and-bound on the Computational Grid ⋄ The Quadratic Assignment Problem ⋄ Special challenges for branch-and-bound methods for non-discrete problems on the computational grid

(Nonconvex) QCQP x ∈ℜ n q 0 ( x ) min subject to q k ( x ) ≥ ∀ k ∈ I b k q k ( x ) = b k ∀ k ∈ E ≤ x u ≥ x l where q k ( x ) = ( c k ) T x + x T Q k x ∀ k ∈ { 0 ∪ I ∪ E} • l and u are �nite • q k ( x ) could be convex, concave, or nonconvex

Caution • I'm certainly not an expert in this area. • I do know that these problems are very hard from a computational standpoint ⋄ QCQP generalizes integer programming and lots of other hard problems. ⋄ Problems with a tens of variables (or tens of quadratic terms) are about the limit of what can be solved ⋆ Solving these problems may require a large amount of computing resources�The computational grid!

Solving QCQP • Popular (best?) method is to use convex and concave envelopes. • Consider quadratic term x i x j , for ( x i , x j ) ∈ Ω ≡ [ l i , u i ] × [ l j , u j ] . ⋄ x i x j ≥ max { l i x j + l j x i − l i l j , u i x j + u j x i − u i u j } ⋄ x i x j ≤ min { l i x j + u j x i − l i u j , u i x j + l j x i − u i l j } • These functions are (resp.) the convex and concave envelope of the function x i x j over [ l i , u i ] × [ l j , u j ] . (McCormick '76, Al-Khayyal and Falk, '83) • vex Ω ( f ) � Convex Envelope of f over Ω �Pointwise supremum of convex underestimators of f over Ω . • cav Ω ( f ) � Concave Envelope of f over Ω �Pointwise in�mum of concave overestimators of f over Ω .

(LP) Relaxation of QCQP X X X n n n c 0 Q 0 min i x i + ij z ij l ≤ x ≤ u i =1 i =1 j =1 subject to X X X n n n c k Q k i x i + ij z ij ≥ b k ∀ k ∈ I i =1 i =1 j =1 X X X n n n c k Q k i x i + ij z ij = b k ∀ k ∈ E i =1 i =1 j =1 z ij − l i x j − l j x i + l i l j 0 ∀ i = 1 , . . . , n, j = 1 , . . . , n ≥ z ij − u i x j − u j x i + u i u j ≥ 0 ∀ i = 1 , . . . , n, j = 1 , . . . , n z ij − l i x j − u j x i + l i u j 0 ∀ i = 1 , . . . , n, j = 1 , . . . , n ≤ z ij − u i x j − l j x i + u i l j 0 ∀ i = 1 , . . . , n, j = 1 , . . . , n ≤

Worth 1000 Words?�Part I x*y 1 0.8 0.6 0.4 0.2 0 1 0.8 0.6 0 0.2 0.4 0.4 0.2 0.6 0.8 1 0 x i x j

Worth 1000 Words?�Part II max(0,x+y-1) (min(x,y)) 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 1 1 0.8 0.8 0.6 0 0.6 0.2 0 0.4 0.4 0.2 0.4 0.6 0.2 0.4 0.2 0.8 0.6 1 0 0.8 1 0 vex ( x i x j ) cav ( x i x j )

Branching • In LP relaxation, z ij = x i x j ∀ x i , x j ∈ ∂ Ω . • If z ij � = x i x j , we branch. Two suggested branching schemes I II I II IV III

Triangle-Based Branching • I'd like to propose a triangular-based branching scheme... C B D A • In order to do this, we need formulae for cav A,B,C,D ( x i x j ) and vex A,B,C,D ( x i x j )

Concave Envelope Formulae Ω ∩ { ( x i , x j ) | x j − u j ≤ l j − u j Let AB = ( x i + l i ) } u i − l i Ω ∩ { ( x i , x j ) | x j − u j ≥ l j − u j Let CD = ( x i + l i ) } u i − l i  if x i = l i , x j = l j l i l j  Cav AB ( x i x j ) = c 0 + c i x i + c j x j + c ij x i x j + c i 2 x 2 i + c j 2 x 2 Otherwise j  d 0 + d i x i + d j x j  if x i = u i , x j = u j u i u j  Cav CD ( x i x j ) = c 0 + c i x i + c j x j + c ij x i x j + c i 2 x 2 i + c j 2 x 2 Otherwise j  d 0 + d i x i + d j x j

Messy De�nitions for Completeness Coef. cav AB cav CD − l 2 i l 2 u 2 i u 2 j + l i l j u i u j j − l i l j u i u j c 0 − l i l j u j − l j u i u j + 2 l 2 − 2 u 2 j u i + l j u i u j + l i l j u j c i j l i − l i l j u i − l i u i u j + 2 l 2 − 2 u 2 c j i l j i u j + l i u i u j + l i l j u i c ij u i u j − l i l j u i u j − l i l j l j u j − l 2 u 2 j − l j u j c i 2 j l i u i − l 2 u 2 i − l i u i c j 2 i − l i u j − u i l j + 2 l i l j − 2 u i u j + l i u j + l j u i d 0 u j − l j u j − l j d i u i − l i u i − l i d j

Now Vex • You can likewise derive formulae for vex BC ( x i x j ) and vex AD ( x i x j ) • I won't bore you with the formulae. For Ω = [0 , 1] × [0 , 1] , x 2 vex BC ( x i x j ) = i x i − x j + 1 x 2 vex AD ( x i x j ) = j − x i + x j + 1

cav Pics (x*y)/(x+y) (1-2*x-2*y+x*y+x*x+y*y)/(x+y-2) 0.25 1 0.2 0.8 0.15 0.6 0.1 0.4 0.05 0.2 0 0 1 1 0.8 0.8 0.6 0 0.6 0.2 0 0.4 0.4 0.2 0.4 0.2 0.6 0.4 0.8 0.6 0.2 1 0 0.8 1 0 cav AB ( x i x j ) cav CD ( x i x j )

This Just In... • Recall, I said I was not an expert... • The convex envelope formulae appeared implictly in [Sherali and Alameddine '90]. • They said they were planning on developing an algorithm using these results, but I don't think they ever did. ⋆ I claim that this would be a very good idea.

Why Triangles Are Good • Just like integer programming (and maybe even more so), a relaxation is good if it is tight. ⋆ In this case, we can explicity calculate a meaningful measure of relaxation goodness ( η ) over an arbitrary region Γ . � ( cav Γ ( x i x j ) − vex Γ ( x i x j )) dx i dx j . η Γ = Γ

Branching Schemes • For Example: ( x i , x j ) ∈ [0 , 2] × [0 , 2] . Consider two branching schemes... C II III B D I A IV η [0 , 2] × [0 , 2] = 8 / 3 η Rectangle = η I + η II + η III + η IV = 2 / 3 η Triangle = η A + η B + η C + η D = 4 / 9 • A branch-and-bound algorithm based on triangular subdivisions may be quite good!

Barriers to Triangular B&B Algorithm • How to (easily, at least for prototyping purposes) interface B&B C++ driver code with existing NLP software to solve relaxations? ⋆ COIN to the rescue! ⋄ NLPAPI (a very recent addition to COIN) is a C API to NLP software. • Lancelot • IPOPT�Very, very, very recently (like three days ago) • This is great, but there is a more fundamental barrier to using NLP in a B&B algorithm...

NLP Stinks! • NLP is quite slow. ⋄ This is largely a function of NLPAPI/Lancelot ⋄ The entire problem is built from scratch every time, writing out SIF �les, before calling Lancelot • NLP is sometimes wrong(!?!?!) • The envelope functions are not differentiable everywhere on the boundary. • They have the �wrong� curvature outside of the region of interest • NLP sometime says, �I don't think your problem has a feasible solution, but I'm not too sure.�

It's Probably My Fault • NLP doesn't stink. I just couldn't resist putting up that slide. • It's the wrong hammer for the job. • The envelope functions I presented have a second-order cone representation. ⋄ Thanks go to Kurt Anstreicher for making me believe that there really was a SOC representation of the envelope functions ⋄ Thanks go to Masakazu Muramatsu for showing me how these things work.

Ice Cream Cone (Symmetric Cone) Programming min { c T x | Ax = b, x ∈ K} • K ⊂ ℜ n is a symmetric cone • Quadratic cone in ℜ n :   � n �    x ∈ ℜ n : x 1 ≥ � K n � x 2 q = � i  i =2 • SOCP has a nice duality theory � It can tell me (with con�dence) that a problem is infeasible • SOCP solvers are robust • I think it should reasonable to embed a SOCP (or even an SDP) solver into a branch and bound algorithm.

SOC Representation (Example) • Imagine Ω = [0 , 1] × [0 , 1] • Restrict ( x i , x j ) ∈ B ≡ { ( x i , x j ) | x i ≤ x j , x i + x j ≤ 1 } x 2 x i − x j +1 , z ij ≤ xy ⇒ z ij ≥ i x + y   z ij + 1 − x j + x i x 2 i  ∈ K 3   z ij ≥ x i − x j + 1 , ( x i , x j ) ∈ B ⇔ 2 x i   q  z ij − 1 + x j − x i   2 x i + x j − z ij xy  ∈ K 3   z ij ≤ x + y , ( x i , x j ) ∈ B ⇔ 2 x i   q  − x j − z ij

Wake Up! • I am going to start talking about �The Grid��Probably a more interesting topic