Optimizing Convex Functions over Non-Convex Domains Dan Bienstock - PowerPoint PPT Presentation

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x ∈ R d − int ( P ) } Sketch proof of (a) . Suppose γ T x − q ≤ δ is valid and holds with equality at some feasible point y . If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the i th facet of P . Using Farkas’s Lemma, we can show that δ = � y � 2 − α b i γ = 2 y − α a i and for some α ≥ 0. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x ∈ R d − int ( P ) } Sketch proof of (a) . Suppose γ T x − q ≤ δ is valid and holds with equality at some feasible point y . If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the i th facet of P . Using Farkas’s Lemma, we can show that δ = � y � 2 − α b i γ = 2 y − α a i and for some α ≥ 0. This means the inequality is a (partially) lifted first-order inequality. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x ∈ R d − int ( P ) } Sketch proof of (a) . Suppose γ T x − q ≤ δ is valid and holds with equality at some feasible point y . If y is strictly feasible, the inequality is just a first-order inequality. So assume y is on the i th facet of P . Using Farkas’s Lemma, we can show that δ = � y � 2 − α b i γ = 2 y − α a i and for some α ≥ 0. This means the inequality is a (partially) lifted first-order inequality. (“Partially” since α may not be as large as possible) Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 9 / 33

Deriving the Lifting Coefficient We want the lifted inequality from a point y on the i th facet of P . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

Deriving the Lifting Coefficient We want the lifted inequality from a point y on the i th facet of P . Amount of violation at point ( x , � x � 2 ) is: � x � 2 − (2 y − α a i ) T x − α b i + � y � 2 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

Deriving the Lifting Coefficient We want the lifted inequality from a point y on the i th facet of P . Amount of violation at point ( x , � x � 2 ) is: � x � 2 − (2 y − α a i ) T x − α b i + � y � 2 For the lifted inequality to be valid, this must be non-negative for all feasible x . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

Deriving the Lifting Coefficient We want the lifted inequality from a point y on the i th facet of P . Amount of violation at point ( x , � x � 2 ) is: � x � 2 − (2 y − α a i ) T x − α b i + � y � 2 For the lifted inequality to be valid, this must be non-negative for all feasible x . If the lifted inequality is as strong as possible, the violation will be 0 at some point x on another facet of P . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 10 / 33

Deriving the Lifting Coefficient We find a lifting coefficient for every facet other than the i th . Taking the smallest one gives a valid lifted inequality. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

Deriving the Lifting Coefficient We find a lifting coefficient for every facet other than the i th . Taking the smallest one gives a valid lifted inequality. To find the lifting coefficient for j th facet, we find α j satisfying: � x � 2 − (2 y − α j a i ) T x − α j b i + � y � 2 � � minimize: = 0 a T subject to: j x = b j Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

Deriving the Lifting Coefficient We find a lifting coefficient for every facet other than the i th . Taking the smallest one gives a valid lifted inequality. To find the lifting coefficient for j th facet, we find α j satisfying: � x � 2 − (2 y − α j a i ) T x − α j b i + � y � 2 � � minimize: = 0 a T subject to: j x = b j We can solve this QP in closed form to obtain 2( b j − a T j y ) α j = � a i �� a j � − a T i a j Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

Deriving the Lifting Coefficient We find a lifting coefficient for every facet other than the i th . Taking the smallest one gives a valid lifted inequality. To find the lifting coefficient for j th facet, we find α j satisfying: � x � 2 − (2 y − α j a i ) T x − α j b i + � y � 2 � � minimize: = 0 a T subject to: j x = b j We can solve this QP in closed form to obtain 2( b j − a T j y ) α j = � a i �� a j � − a T i a j Final lifting coefficient is: min j � = i α j Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 11 / 33

Finding the Strongest Cut We have derived an expression for the lifting coefficient α : 2( b j − a T j y ) α = min � a i �� a j � − a T i a j j � = i Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 12 / 33

Finding the Strongest Cut We have derived an expression for the lifting coefficient α : 2( b j − a T j y ) α = min � a i �� a j � − a T i a j j � = i Given a point ˆ x ∈ int ( P ), solving m convex QPs of the following form gives the strongest cut at ˆ x : −� y � 2 + ˆ x T y + α ( b i − a T maximize: i ˆ x ) a T subject to: i y = b i a T j y ≤ b j ∀ j � = i 2( b j − a T j y ) α ≤ ∀ j � = i � a i �� a j �− a T i a j Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 12 / 33

Disjunctive Approach P = { x ∈ R d : a T i x ≤ b i , 1 ≤ i ≤ m } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

Disjunctive Approach P = { x ∈ R d : a T i x ≤ b i , 1 ≤ i ≤ m } S = conv ( Q 1 ∪ Q 2 ∪ . . . ∪ Q m ) , where � ( x , q ) ∈ R d +1 : � x � 2 ≤ q , a T � Q i = i x ≥ b i . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

Disjunctive Approach P = { x ∈ R d : a T i x ≤ b i , 1 ≤ i ≤ m } S = conv ( Q 1 ∪ Q 2 ∪ . . . ∪ Q m ) , where � ( x , q ) ∈ R d +1 : � x � 2 ≤ q , a T � Q i = i x ≥ b i . Ceria and Soares (1999) ⇒ min { q : (ˆ x , q ) ∈ S } is an SOCP Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

Disjunctive Approach P = { x ∈ R d : a T i x ≤ b i , 1 ≤ i ≤ m } S = conv ( Q 1 ∪ Q 2 ∪ . . . ∪ Q m ) , where � ( x , q ) ∈ R d +1 : � x � 2 ≤ q , a T � Q i = i x ≥ b i . Ceria and Soares (1999) ⇒ min { q : (ˆ x , q ) ∈ S } is an SOCP with O ( md ) variables and m conic constraints Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

Disjunctive Approach P = { x ∈ R d : a T i x ≤ b i , 1 ≤ i ≤ m } S = conv ( Q 1 ∪ Q 2 ∪ . . . ∪ Q m ) , where � ( x , q ) ∈ R d +1 : � x � 2 ≤ q , a T � Q i = i x ≥ b i . Ceria and Soares (1999) ⇒ min { q : (ˆ x , q ) ∈ S } is an SOCP with O ( md ) variables and m conic constraints Separation: solve SOCP, use SOCP “Farkas Lemma”, get linear cut Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

Disjunctive Approach P = { x ∈ R d : a T i x ≤ b i , 1 ≤ i ≤ m } S = conv ( Q 1 ∪ Q 2 ∪ . . . ∪ Q m ) , where � ( x , q ) ∈ R d +1 : � x � 2 ≤ q , a T � Q i = i x ≥ b i . Ceria and Soares (1999) ⇒ min { q : (ˆ x , q ) ∈ S } is an SOCP with O ( md ) variables and m conic constraints Separation: solve SOCP, use SOCP “Farkas Lemma”, get linear cut Generally harder than m QPs with d + 1 variables, 2 m linear constraints each Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 13 / 33

Baby Example A simple example: d = 1 and P = [ − 1 , 2]. Feasible region is blue . Weak relaxation gives optimal point of (0 , 0). 8 6 q 4 2 0 ● −3 −2 −1 0 1 2 3 x Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 14 / 33

Baby Example First-order inequality at x = − 1 is: q ≥ − 2 x − 1 8 6 q 4 2 ● 0 ● −3 −2 −1 0 1 2 3 x Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 15 / 33

Baby Example Lifted first-order inequality at x = − 1 is: q ≥ x + 2. In this case, a single cut gives the convex hull of feasible region. 8 6 q 4 2 0 ● −3 −2 −1 0 1 2 3 x Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 16 / 33

Geometrical characterization When does a point x � 2 � � ˆ x , � ˆ violate a lifted first-order inequality q ≥ 2 y T x − � y � 2 − α ( a T i x − b i ) ? Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

Geometrical characterization When does a point x � 2 � � ˆ x , � ˆ violate a lifted first-order inequality q ≥ 2 y T x − � y � 2 − α ( a T i x − b i ) ? It does, if and only if: x � 2 − 2 y T ˆ x < − � y � 2 + α b i x + α a T � ˆ i ˆ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

Geometrical characterization When does a point x � 2 � � ˆ x , � ˆ violate a lifted first-order inequality q ≥ 2 y T x − � y � 2 − α ( a T i x − b i ) ? It does, if and only if: x � 2 − 2 y T ˆ x < − � y � 2 + α b i x + α a T � ˆ i ˆ This describes the interior of a ball , Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

Geometrical characterization When does a point x � 2 � � ˆ x , � ˆ violate a lifted first-order inequality q ≥ 2 y T x − � y � 2 − α ( a T i x − b i ) ? It does, if and only if: x � 2 − 2 y T ˆ x < − � y � 2 + α b i x + α a T � ˆ i ˆ This describes the interior of a ball , which must be contained in int ( P ) Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 17 / 33

Geometrical characterization int(P) cut−off region y Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 18 / 33

Geometrical characterization int(P) cut−off region y Characterization: x ∈ R d − int ( P ) iff, for each ball B ( µ, R ) ⊆ P , Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 18 / 33

Geometrical characterization int(P) cut−off region y Characterization: x ∈ R d − int ( P ) iff, for each ball B ( µ, R ) ⊆ P , � x − µ � 2 ≥ R 2 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 18 / 33

Geometrical characterization - Example 1 3 2 1 ● 0 −1 −2 −3 −4 −2 0 2 4 Separating (0 , 0). Excluded region in red . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 19 / 33

Geometrical characterization - Example 2 2 1 0 ● ● −1 −2 −6 −4 −2 0 2 4 6 Separating (0 , 0). Excluded region in red . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 20 / 33

Ellipsoidal Case For A � 0 , polynomially separable linear inequality description for: conv { ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

Ellipsoidal Case For A � 0 , polynomially separable linear inequality description for: conv { ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } S µ ρ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

Ellipsoidal Case For A � 0 , polynomially separable linear inequality description for: conv { ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } S µ ρ q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S q − 2 µ T ˆ x + µ T µ ) so as to maximize ρ − (ˆ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

Ellipsoidal Case For A � 0 , polynomially separable linear inequality description for: conv { ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } S µ ρ q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S x − µ � 2 − ˆ q − 2 µ T ˆ x + µ T µ ) = ρ − � ˆ x � 2 so as to maximize ρ − (ˆ q + � ˆ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 21 / 33

S-Lemma (Yakubovich, 1971) Let f and g be quadratic functions on R d , and suppose that there is ¯ x with g (¯ x ) < 0. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 22 / 33

S-Lemma (Yakubovich, 1971) Let f and g be quadratic functions on R d , and suppose that there is ¯ x with g (¯ x ) < 0. Then: f ( x ) ≥ 0 whenever g ( x ) ≤ 0 if and only if There is τ ≥ 0 such that f ( x ) + τ g ( x ) ≥ 0 for all x ∈ R d Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 22 / 33

S-Lemma (Yakubovich, 1971) Let f and g be quadratic functions on R d , and suppose that there is ¯ x with g (¯ x ) < 0. Then: f ( x ) ≥ 0 whenever g ( x ) ≤ 0 if and only if There is τ ≥ 0 such that f ( x ) + τ g ( x ) ≥ 0 for all x ∈ R d Proof relies on convexity of { ( f ( x ) , g ( x )) : x ∈ R d } for homogenous f and g (Dines 1941). Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 22 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } ⇒ B( µ, √ ρ ) ⊆ R d − S is SOCP-representable → Fix ρ > 0: Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } ⇒ B( µ, √ ρ ) ⊆ R d − S is SOCP-representable → Fix ρ > 0: � x � 2 − ρ ≥ 0 , ∀ x s.t. ( x + µ ) T A ( x + µ ) − 2 c T ( x + µ ) + b ≥ 0 } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } ⇒ B( µ, √ ρ ) ⊆ R d − S is SOCP-representable → Fix ρ > 0: � x � 2 − ρ ≥ 0 , ∀ x s.t. ( x + µ ) T A ( x + µ ) − 2 c T ( x + µ ) + b ≥ 0 } iff (S-Lemma) there exists τ > 0 such that, for all x ∈ R d � � I − 1 x + 2 τ ( c T − µ T A ) x + 1 x T τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ A → A = UΛU T (eigenspace), y . = U T x , v . = U T (c − A µ ) Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } ⇒ B( µ, √ ρ ) ⊆ R d − S is SOCP-representable → Fix ρ > 0: � x � 2 − ρ ≥ 0 , ∀ x s.t. ( x + µ ) T A ( x + µ ) − 2 c T ( x + µ ) + b ≥ 0 } iff (S-Lemma) there exists τ > 0 such that, for all x ∈ R d � � I − 1 x + 2 τ ( c T − µ T A ) x + 1 x T τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ A → A = UΛU T (eigenspace), y . = U T x , v . = U T (c − A µ ) ⇒ ∀ y ∈ R d � � I − 1 y + 2 τ v T y + 1 y T τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ Λ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

{ ( x , q ) ∈ R d +1 : � x � 2 ≤ q , x T Ax − 2 c T x + b ≥ 0 } ⇒ B( µ, √ ρ ) ⊆ R d − S is SOCP-representable → Fix ρ > 0: � x � 2 − ρ ≥ 0 , ∀ x s.t. ( x + µ ) T A ( x + µ ) − 2 c T ( x + µ ) + b ≥ 0 } iff (S-Lemma) there exists τ > 0 such that, for all x ∈ R d � � I − 1 x + 2 τ ( c T − µ T A ) x + 1 x T τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ A → A = UΛU T (eigenspace), y . = U T x , v . = U T (c − A µ ) ⇒ ∀ y ∈ R d � � I − 1 y + 2 τ v T y + 1 y T τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ Λ  I − 1 1  τ Λ τ v     � 0       τ v T 1 τ ( − µ T A µ + 2 c T µ − b ) − ρ 1 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 23 / 33

I − 1 y + 2 τ v T y + 1 y T � � τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ Λ ∀ y Using Schur Complement gives the equivalent conditions: v 2 � d − 1 1 − λ j /τ + 1 τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 j τ 2 j =1 τ ≥ λ max ( A ) Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 24 / 33

I − 1 y + 2 τ v T y + 1 y T � � τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ Λ ∀ y Using Schur Complement gives the equivalent conditions: v 2 � d − 1 1 − λ j /τ + 1 τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 j τ 2 j =1 τ ≥ λ max ( A ) The first inequality is equivalent to: d v 2 � j − µ T A µ + 2 c T µ − b − ρτ ≥ 0 − τ − λ j j =1 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 24 / 33

I − 1 y + 2 τ v T y + 1 y T � � τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 τ Λ ∀ y Using Schur Complement gives the equivalent conditions: v 2 � d − 1 1 − λ j /τ + 1 τ ( − µ T A µ + 2 c T µ − b ) − ρ ≥ 0 j τ 2 j =1 τ ≥ λ max ( A ) The first inequality is equivalent to: d v 2 � j − µ T A µ + 2 c T µ − b − ρτ ≥ 0 − τ − λ j j =1 which is SOCP-representable. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 24 / 33

q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S q − 2 µ T ˆ x + µ T µ ) so as to maximize ρ − (ˆ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S x − µ � 2 − ˆ q − 2 µ T ˆ x + µ T µ ) = ρ − � ˆ x � 2 so as to maximize ρ − (ˆ q + � ˆ Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S x − µ � 2 − ˆ q − 2 µ T ˆ x + µ T µ ) = ρ − � ˆ x � 2 so as to maximize ρ − (ˆ q + � ˆ Given ρ > 0, . x − µ � 2 − ˆ x � 2 ∆( ρ ) = max ρ − � ˆ q + � ˆ µ s.t. B ( µ, √ ρ ) ⊆ R d − S Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S x − µ � 2 − ˆ q − 2 µ T ˆ x + µ T µ ) = ρ − � ˆ x � 2 so as to maximize ρ − (ˆ q + � ˆ Given ρ > 0, . x − µ � 2 − ˆ x � 2 ∆( ρ ) = max ρ − � ˆ q + � ˆ µ s.t. B ( µ, √ ρ ) ⊆ R d − S Separation problem: max ρ> 0 ∆( ρ ) Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

q ) ∈ R d +1 , Separation problem: given(ˆ x , ˆ Find a ball B( µ, √ ρ ) ⊆ R d − S x − µ � 2 − ˆ q − 2 µ T ˆ x + µ T µ ) = ρ − � ˆ x � 2 so as to maximize ρ − (ˆ q + � ˆ Given ρ > 0, . x − µ � 2 − ˆ x � 2 ∆( ρ ) = max ρ − � ˆ q + � ˆ µ s.t. B ( µ, √ ρ ) ⊆ R d − S Separation problem: max ρ> 0 ∆( ρ ) Lemma: ∆( ρ ) is a concave function of ρ . Separation problem can be solved using a series of SOCPs. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 25 / 33

Indefinite Quadratics Next we consider an objective of the form x T Mx + v T x + c min where M is symmetric but indefinite. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 26 / 33

Indefinite Quadratics Next we consider an objective of the form x T Mx + v T x + c min where M is symmetric but indefinite. By changing coordinates and lifting we obtain the equivalent problem: min q − p + v T x + c d d � � x 2 λ i x 2 s.t. q ≥ i , p ≤ i i =1 i =1 where λ i > 1 for each i . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 26 / 33

Indefinite Quadratics Define d P = { ( x , p ) ∈ R d × R : p ≥ � λ i x 2 i } i =1 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 27 / 33

Indefinite Quadratics Define d P = { ( x , p ) ∈ R d × R : p ≥ � λ i x 2 i } i =1 The problem can then be stated as: min q − p + v T x + c d ( x , p ) ∈ R d +1 − int ( P ) � x 2 s.t. q ≥ i , i =1 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 27 / 33

Indefinite Quadratics Fix β = max i λ i , and for µ ∈ R d and φ ∈ R , define M ( β, µ, φ ) = { ( x , p ) ∈ R d +1 : p ≥ β � x − µ � 2 + φ } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 28 / 33

Indefinite Quadratics Fix β = max i λ i , and for µ ∈ R d and φ ∈ R , define M ( β, µ, φ ) = { ( x , p ) ∈ R d +1 : p ≥ β � x − µ � 2 + φ } This is a paraboloid in the ( x , p ) space. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 28 / 33

Indefinite Quadratics Fix β = max i λ i , and for µ ∈ R d and φ ∈ R , define M ( β, µ, φ ) = { ( x , p ) ∈ R d +1 : p ≥ β � x − µ � 2 + φ } This is a paraboloid in the ( x , p ) space. Then ( x , p ) ∈ R d +1 − int ( P ) if and only if ( x , p ) ∈ R d +1 − int ( M ( β, µ, φ )) for all ( β, µ, φ ) with M ( β, µ, φ ) ⊆ P . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 28 / 33

Indefinite Quadratics - Separation q ) ∈ R d × R × R , the separation problem can be formulated as: Given (ˆ x , ˆ p , ˆ q − 2 βµ T ˆ x + β � µ � 2 + φ min β ˆ s.t. M ( β, µ, φ ) ⊆ P Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 29 / 33

Indefinite Quadratics - Separation q ) ∈ R d × R × R , the separation problem can be formulated as: Given (ˆ x , ˆ p , ˆ q − 2 βµ T ˆ x + β � µ � 2 + φ min β ˆ s.t. M ( β, µ, φ ) ⊆ P If ( µ ∗ , φ ∗ ) is optimal for this problem, then we get the cut β q − 2 βµ ∗ T x + β � µ ∗ � 2 + φ ∗ − p ≥ 0 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 29 / 33

Indefinite Quadratics - Separation We can reformulate the separation problem as: βµ T ˆ x + � µ � 2 + φ � min β ˆ q − 2 � s.t. M ( β, µ/ β, φ ) ⊆ P Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 30 / 33

Indefinite Quadratics - Separation We can reformulate the separation problem as: βµ T ˆ x + � µ � 2 + φ � min β ˆ q − 2 � s.t. M ( β, µ/ β, φ ) ⊆ P But M ( β, µ/ √ β, φ ) ⊆ P iff � d � β � 2 + φ − � � λ i x 2 min β � x − µ/ ≥ 0 i x ∈ R d i =1 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 30 / 33

Indefinite Quadratics - Separation We need to enforce the constraint � d � β � 2 + φ − � � λ i x 2 min β � x − µ/ ≥ 0 i x ∈ R d i =1 Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

Indefinite Quadratics - Separation We need to enforce the constraint � d � β � 2 + φ − � � λ i x 2 min β � x − µ/ ≥ 0 i x ∈ R d i =1 The optimal solution to the minimization problem is any point ¯ x where √ β x i = µ i for i s.t. λ i � = β ¯ β − λ i Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

Indefinite Quadratics - Separation We need to enforce the constraint � d � β � 2 + φ − � � λ i x 2 min β � x − µ/ ≥ 0 i x ∈ R d i =1 The optimal solution to the minimization problem is any point ¯ x where √ β x i = µ i for i s.t. λ i � = β ¯ β − λ i Plugging ¯ x into the objective value and simplifying results in the constraint: µ 2 i λ i � φ ≥ β − λ i { i : λ i � = β } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

Indefinite Quadratics - Separation We need to enforce the constraint � d � β � 2 + φ − � � λ i x 2 min β � x − µ/ ≥ 0 i x ∈ R d i =1 The optimal solution to the minimization problem is any point ¯ x where √ β x i = µ i for i s.t. λ i � = β ¯ β − λ i Plugging ¯ x into the objective value and simplifying results in the constraint: µ 2 i λ i � φ ≥ β − λ i { i : λ i � = β } which is SOCP-representable. Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 31 / 33

Indefinite Quadratics - Separation The final separation problem is the SOCP: βµ T ˆ x + � µ � 2 + φ � min β ˆ q − 2 d µ 2 i λ i � s.t. φ ≥ β − λ i { i : λ i � = β } Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 32 / 33

Application: the pooling problem Example: � min 2 x j y j + . . . j s.t. . . . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33

Application: the pooling problem Example: � min 2 x j y j + . . . j s.t. . . . j ( x j + y j ) 2 − � j ( x 2 j + y 2 But: 2 � = � j ) j x j y j Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33

Application: the pooling problem Example: � min 2 x j y j + . . . j s.t. . . . j ( x j + y j ) 2 − � j ( x 2 j + y 2 But: 2 � = � j ) j x j y j General case where α > 0: � u 2 � α j u 2 min − j j j j s.t. . . . Dan Bienstock, Alex Michalka (Columbia) Convex obj non-convex domain Aussois 2012 33 / 33