CONVEXIFICATION AND GLOBAL OPTIMIZATION Nick Sahinidis University - PowerPoint PPT Presentation

CONVEXIFICATION AND GLOBAL OPTIMIZATION Nick Sahinidis University of Illinois at Urbana-Champaign Chemical and Biomolecular Engineering

MIXED-INTEGER NONLINEAR PROGRAMMING (P) min f ( x, y ) Objective Function s.t. g ( x, y ) ≤ 0 Constraints x ∈ R n Continuous Variables y ∈ Z p Integrality Restrictions Challenges: Multimodal Objective • Objective f ( x ) Integrality • Feasible Space Nonconvex Constraints • f ( x ) Convex Objective Projected Objective Nonconvex Feasible Space

MINLP ALGORITHMS • Branch-and-Bound • Our approach – Bound problem over successively – Branch-and-Reduce refined partitions » Ryoo and Sahinidis, 1995, 1996 » Falk and Soland, 1969 » Shectman and Sahinidis, 1998 » McCormick, 1976 – Constraint Propagation & Duality- Based Reduction • Convexification » Ryoo and Sahinidis, 1995, 1996 – Outer-approximate with increasingly tighter convex programs » Tawarmalani and Sahinidis, 2002 – Tuy, 1964 – Convexification – Sherali and Adams, 1994 » Tawarmalani and Sahinidis, • Decomposition 2001, 2002 – Project out some variables by • Tawarmalani, M. and N. V. solving subproblem Sahinidis, Convexification and » Duran and Grossmann, 1986 Global Optimization in » Visweswaran and Floudas, 1990 Continuous and Mixed-Integer Nonlinear Programming, Kluwer Academic Publishers, Nov. 2002.

BRANCH-AND-BOUND Objective Objective P P U R R L L Variable Variable a. Lower Bounding b. Upper Bounding Objective P R R2 U L R1 R R1 R2 Fathom Subdivide Variable c. Domain Subdivision d. Search Tree

FACTORABLE FUNCTIONS (McCormick, 1976) Definition: Factorable functions are recursive compositions of sums and products of functions of single variables. � exp( xy + z ln w ) z 3 Example: f ( x, y, z, w ) = x 1 = xy f x 2 = ln( w ) � �� � x 5 x 3 = zx 2 � �� � x 6 x 3 ���� � �� � x 4 = x 1 + x 3 � � 0 . 5 z 3 exp( xy + z ln w ) ���� ���� x 5 = exp( x 4 ) x 2 x 1 � �� � x 6 = z 3 x 4 � �� � x 7 = x 5 x 6 x 7 f = √ x 7

RATIO: THE FACTORABLE RELAXATION z ≥ x/y y L ≤ y ≤ y U x / y y U x L ≤ x ≤ x U x L x y y L x U zy ≥ x z ≥ x/y cross-multiplying y L ≤ y ≤ y U x L ≤ x ≤ x U x L /y U ≤ z ≤ x U /y L y L ≤ y ≤ y U x L ≤ x ≤ x U Relaxing z ≥ ( xy U − yx L + x L y U ) /y U 2 zy − ( z − x L /y U )( y − y U ) ≥ x Simplifying z ≥ ( xy L − yx U + x U y L ) /y L 2 zy − ( z − x U /y L )( y − y L ) ≥ x y L ≤ y ≤ y U y L ≤ y ≤ y U x L ≤ x ≤ x U x L ≤ x ≤ x U

TIGHT RELAXATIONS f ( x ) f ( x ) Concave Concave envelope over-estimator Convex envelope Convex under-estimator x x f ( x ) x Convex/concave envelopes often finitely generated

CONVEX EXTENSIONS OF L.S.C. FUNCTIONS Definition: A function f ( x ) is a convex extension of g ( x ) : C �→ R restricted to X ⊆ C if • f ( x ) is convex on conv ( X ), • f ( x ) = g ( x ) for all x ∈ X . Example: The Univariate Case • f(x) is a convex extension of g(x) restricted g(x) l m to { l, n, o, q } q f(x) • Convex extension of g(x) restricted to { l, m, n p n, o, p, q } cannot be constructed o x (0,0)

THE GENERATING SET OF A FUNCTION Definition: The generating set of the epigraph of a function g ( x ) over a compact convex set C is defined as � � � ��� � � G epi C ( g ) = x � ( x, y ) ∈ vert epi conv g ( x ) , � where vert( · ) is the set of extreme points of ( · ). Examples: g ( x ) = − x 2 g ( x ) = xy x − x 2 xy x Convex Envelope y G epi G epi [0 , 6] ( g ) = { 0 } ∪ { 6 } [1 , 4] 2 ( g ) = { 1 , 1 } ∪ { 1 , 4 } ∪ { 4 , 1 } ∪ { 4 , 4 }

TWO-STEP CONVEX ENVELOPE CONSTRUCTION 1. Identify generating set • Key result: A point in set X is not in the generating set if it is not in the generating set over a neighborhood of X that contains it 2. Use disjunctive programming techniques to construct epigraph over the generating set • Rockafellar (1970) • Balas (1974)

IDENTIFYING THE GENERATING SET Characterization: x 0 �∈ G epi C ( g ) if and only if there exists X ⊆ C and x 0 �∈ G epi X ( g ). Example I: X is linear joining ( x L , y 0 ) and ( x U , y 0 ) � � � � x ∈ { x L , x U } G epi ( x/y ) = ( x, y ) x y x y Example II: X is ǫ neighborhood of ( x 0 , y 0 ) � � � � x ∈ { x L , x U } G epi ( x 2 y 2 ) = ( x, y ) ∪ x 2 y 2 � � � � y ∈ { y L , y U } ( x, y ) U y x L x y L x U y

CONVEX ENVELOPE OF x/y Second Order Cone Representation: √ � 2(1 − λ ) � �� x L � � � ≤ z p + y p � � z p − y p � √ � �� � x U 2 λ � � � ≤ z − z p + y − y p � � z − z p − y + y p � y p ≥ y L (1 − λ ) , y p ≥ y − y U λ y p ≤ y U (1 − λ ) , y p ≤ y − y L λ x = (1 − λ ) x L + λx U z p , u, v ≥ 0 , z c − z p ≥ 0 0 ≤ λ ≤ 1 Comparison of Tightness: 14 . 2 4 40 0 . 5 0 y 0 x 0 . 1 3 . 4 0 . 5 x 0 4 0 . 1 y 0 . 5 0 . 10 . 1 x 4 0 . 1 y 0 . 1 Ratio: x/y x/y − Envelope x/y − Factorable Maximum Gap: Envelope and Factorable Relaxation: � � x U , y L + y L ( y U − y L )( x U y U − x L y L ) Point: x U y U 2 − x L y L 2 x U ( y U − y L ) 2 ( x U y U − x L y L ) 2 Gap: y L y U (2 x U y U − x L y L − x U y L )( x U y U 2 − x L y L 2 )

ENVELOPES OF MULTILINEAR FUNCTIONS • Multilinear function over a box p ∑ ∏ t = − ∞ < ≤ ≤ < +∞ = M ( x ,..., x ) a x , L x U , i 1 , , n K 1 n t i i i i = t i 1 • Generating set  ∏  n   vert [ L , U ]  i i   = i 1 • Polyhedral convex encloser follows trivially from polyhedral representation theorems

FURTHER APPLICATIONS M ( x 1 , x 2 , · · · x n ) / ( y a 1 1 y a 2 2 . . . y a m m ) where M ( · ) is a multilinear expression y 1 , . . . , y m � = 0 a 1 , . . . , a m ≥ 0 Example: ( x 1 x 2 + x 3 x 2 ) / ( y 1 y 2 y 3 ) n k � � a ij y j f ( x ) i i =1 j = − p where f is concave a ij ≥ 0 for i = 1 , . . . , n ; j = − p, . . . , k y i > 0 Example: x/y + 3 x + 4 xy + 2 xy 2

PRODUCT DISAGGREGATION Consider the function: n n ∑ ∑ φ = + + + ( x ; y , , y ) a a y x b x b y K 1 n 0 k k 0 k k = = k 1 k 1 Let n Π = × L U L U H [ x , x ] [ y , y ] k k = k 1 Then n ∑ convenv φ = + + + a a y x b H 0 k k 0 = k 1 n ∑ convenv ( b y x ) L U L U k k × [ y , y ] [ x , x ] k k = k 1 Disaggregated formulations are tighter

POOLING: p FORMULATION y 11 ≤ 3% S ≤ 2 . 5% S x 11 Blend X $6 y 21 $9 Pool ≤ 1% S X ≤ 100 x 21 $16 y 12 ≤ 2% S ≤ 1 . 5% S x 12 Blend Y $10 y 22 $15 Y ≤ 200 X -revenue Y -revenue cost � �� � � �� � � �� � min 6 x 11 + 16 x 21 + 10 x 12 − 9( y 11 + y 21 ) − 15( y 12 + y 22 ) q = 3 x 11 + x 21 s.t. Sulfur Mass Balance y 11 + y 12 x 11 + x 21 = y 11 + y 12 Mass balance x 12 = y 21 + y 22 qy 11 + 2 y 21 ≤ 2 . 5 y 11 + y 21 Quality Requirements qy 12 + 2 y 22 ≤ 1 . 5 y 12 + y 22 y 11 + y 21 ≤ 100 Demands y 12 + y 22 ≤ 200 Haverly 1978

POOLING: q FORMULATION ≤ 3% S $6 y 11 ≤ 2 . 5% S Blend X y 11 q 11 + y 12 q 11 z 31 $9 Pool X ≤ 100 ≤ 1% S $16 y 11 q 21 + y 12 q 21 y 12 ≤ 2% S ≤ 1 . 5% S Blend Y $10 z 32 $15 Y ≤ 200 cost � �� � min 6 ( y 11 q 11 + y 12 q 11 ) + 16 ( y 11 q 21 + y 12 q 21 ) + 10 ( z 31 + z 32 ) X -revenue Y -revenue � �� � � �� � 9( y 11 + y 21 ) − 15( x 12 + x 22 ) − s.t. q 11 + q 21 = 1 Mass Balance − 0 . 5 z 31 + 3 y 11 q 11 + y 11 q 21 ≤ 2 . 5 y 11 Quality Requirements 0 . 5 z 32 + 3 y 12 q 11 + y 12 q 21 ≤ 1 . 5 y 12 y 11 + z 31 ≤ 100 Demands y 12 + z 32 ≤ 200 Ben-Tal et al. 1994

POOLING: pq FORMULATION ≤ 3% S $6 y 11 ≤ 2 . 5% S Blend X y 11 q 11 + y 12 q 11 z 31 $9 Pool X ≤ 100 ≤ 1% S $16 y 11 q 21 + y 12 q 21 y 12 ≤ 2% S ≤ 1 . 5% S Blend Y $10 z 32 $15 Y ≤ 200 cost � �� � min 6 ( y 11 q 11 + y 12 q 11 ) + 16 ( y 11 q 21 + y 12 q 21 ) + 10 ( z 31 + z 32 ) X -revenue Y -revenue � �� � � �� � 9( y 11 + y 21 ) − 15( x 12 + x 22 ) − s.t. q 11 + q 21 = 1 Mass Balance − 0 . 5 z 31 + 3 y 11 q 11 + y 11 q 21 ≤ 2 . 5 y 11 Quality Requirements 0 . 5 z 32 + 3 y 12 q 11 + y 12 q 21 ≤ 1 . 5 y 12 y 11 + z 31 ≤ 100 Demands y 12 + z 32 ≤ 200 y 11 q 11 + y 11 q 21 = y 11 Convexification y 12 q 11 + y 12 q 21 = y 12 Constraints Proof relies on Convex Extensions

PROOF VIA CONVEX EXTENSIONS ≤ 3% S $6 y 11 ≤ 2 . 5% S Blend X y 11 q 11 + y 12 q 11 z 31 $9 Pool X ≤ 100 ≤ 1% S $16 y 11 q 21 + y 12 q 21 y 12 ≤ 2% S ≤ 1 . 5% S Blend Y $10 z 32 $15 Y ≤ 200 With Convexification Constraints, the convex envelope of I � C ik q il y lj i =1 over I � q il = 1 i =1 q il ∈ [0 , 1] y lj ∈ [ y L lj , y U lj ] is included. In the example, the convex envelopes of 3 q 11 y 11 + q 21 y 11 and 3 q 11 y 12 + q 21 y 12 over q 11 + q 12 = 1 q 11 , q 12 ∈ [0 , 1] y 11 ∈ [0 , 100] , y 12 ∈ [0 , 200] are generated in this way.

OUTER APPROXIMATION Motivation: • Convex NLP solvers are not as robust as LP solvers • Linear programs can be solved efficiently Outer-Approximation: Convex Functions are underestimated by tangent lines φ ( x ) x x U x L