A Semidefinite Relaxation Scheme for Multivariate Quartic Polynomial Optimization With Quadratic Constraints Zhi-Quan Luo Department of Electrical and Computer Engineering University of Minnesota Shuzhong Zhang Department of Systems Engineering and Engineering Management Chinese University of Hong Kong 2009 Lunteran MOR Conference, The Netherland 13 January 2009
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Talk Outline • Quartic optimization: motivation • What is SDP/SOS relaxation? • Approximation bounds 1
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Quartic Optimization Maximization form � maximize f ( x ) = a ijkℓ x i x j x k x ℓ (1) 1 ≤ i,j,k,ℓ ≤ n x T A i x ≤ 1 , i = 1 , ..., m, subject to or the minimization form � minimize f ( x ) = a ijkℓ x i x j x k x ℓ (2) 1 ≤ i,j,k,ℓ ≤ n x T A i x ≥ 1 , i = 1 , ..., m, subject to where A i ∈ R n × ( n +1) / 2 : positive semidefinite, i = 1 , ..., m . • f max and f min denote the optimal values of (1) and (2) respectively. • To ensure f min and f max exist, we assume throughout that � m i A i ≻ 0 . 2
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Quartic Optimization: Motivation Quartic optimization problems arise in various engineering applications • Sensor localization: let A and S denote the anchor nodes and sensor nodes respectively � 2 � 2 � x i − x j � 2 − d 2 � x i − s j � 2 − d 2 � � � � minimize + ij ij i,j ∈S i ∈S ,j ∈A ⇒ Quartic minimization (Known: NP-hard; constant factor approximation is also hard ) • Digital communication: blind channel equalization of constant modulus signals x ( t ) = Hs ( t ) + n ( t ) where H is unknown, the components of s ( t ) are constant ( | s i ( t ) | = 1 , ∀ i ) A channel equalizer g can be found by ( | g T x ( t ) | 2 − 1) 2 , � ⇒ Quartic minimization minimize t 3
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang • Signal processing: independent component analysis (ICA) x = Hs , H full column rank, unknown ⋆ s is independent, high 4-th Kurtosis, non-Gaussian sources; x : measurement, unknown linear mixture of s ⋆ Goal: Find G such that Gx is a permutation of s ⋆ Gx is separate, independent ⇔ the 4-th order Kurtosis of Gx is high ⇒ maximize the 4-th order Kurtosis of Gx (fourth order polynomial of G ) subject to ball constraint (power constraint) ⇒ ball-constrained homogeneous quartic maximization 4
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Quartic Optimization: Complexity • The quartic polynomial optimization problems (1)–(2) are nonconvex , NP-hard ⇒ consider polynomial time relaxation procedures that can deliver provably high quality approximate solutions (for special subclasses of quartic optimization problems). Approximation Ratio • ˆ x is a c - factor approximation of quartic minimization problem (2) if f min ≤ f (ˆ x ) ≤ cf min with c independent of problem data. (Therefore, f min = 0 ⇔ f (ˆ x ) = 0 .) • Weaker notion: (1 − ǫ ) -approximation of quartic minimization problem (2) if x ) − f min ≤ (1 − ǫ )( f max − f min ) f (ˆ with ǫ independent of problem data. • Similarly for quartic maximization problem. 5
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang SDP/SOS Relaxation • the sum-of-squares (SOS) technique ⋆ represent each nonnegative polynomial as a sum of squares of some other polynomials a given degree ⋆ Alternatively, use matrix lifting 1 x i � � X := x i x j 1 x i x i x j x i x j x k · · · x i x j x k . . . ⋆ Under the lifting, each polynomial inequality is relaxed to a convex , linear matrix inequality • approximate (arbitrarily well) by a hierarchy of SDPs with increasing size • difficulty: the size of the resulting SDPs in the hierarchy grows exponentially fast 6
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang SDP/SOS Relaxation • The most effective use of SDP relaxation so far has been for the quadratic optimization problems whereby only the first level relaxation in the SOS hierarchy is used. ⋆ difficulty: cannot provide arbitrarily tight approximation in general ⋆ does lead to provably high quality approximate solution for certain type of quadratic optimization problems (e.g., Max-Cut) • Question: find a provably good first level SOS approximation of some quartic optimization problems (1)–(2)? 7
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang SDP Relaxation of Nonconvex Quadratic Optimization Problem • focus here on a specific class of problems: general QCQPs • vast range of applications... the generic QCQP can be written: x T A 0 x + r 0 minimize x T A i x + r i ≤ 0 , subject to i = 1 , . . . , m • if all A i are p.s.d., convex problem, • here, we suppose at least one A i not p.s.d. 8
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Convex Relaxation Using a fundamental observation: X := xx T X ij = x i x j X � 0 , rank( X ) = 1 , ⇔ ⇔ and noting x T A i x = Tr ( XA i ) , the original QCQP: minimize f ( x ) = x T A 0 x + r 0 x T A i x + r i ≤ 0 , subject to i = 1 , . . . , m can be rewritten: minimize g ( X ) = Tr ( XA 0 ) + r 0 subject to Tr ( XA i ) + r i ≤ 0 , i = 1 , . . . , m X � 0 , rank( X ) = 1 the only nonconvex constraint is now rank( X ) = 1 ... 9
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Convex Relaxation: Semidefinite Relaxation • we can directly relax this last constraint, i.e. drop the nonconvex rank( X ) = 1 to keep only X � 0 • the resulting program gives a lower bound on the optimal value g ( X ) = T r ( XA 0 ) + r 0 minimize subject to Tr ( XA i ) + r i ≤ 0 , i = 1 , . . . , m ⇒ SDP X � 0 How to Generate a Feasible Solution? Let X ∗ be the optimal solution of • pick x as a Gaussian variable with x ∼ N (0 , X ∗ ) • Since Tr ( X ∗ A i ) + r i = E[ x T A i x + r i ] , x will solve the QCQP “on average” over this distribution 10
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Generate a Feasible Solution In other words, SDP is equivalent to E[ x T A 0 x + r 0 ] minimize E[ x T A i x + r i ] ≤ 0 , subject to i = 1 , . . . , m a good feasible point can then be obtained by sampling enough x . . . Two observations: • SDP finds the convariance matrix used in sampling • The relaxed function g ( X ) satisfies ⋆ Consistency: g ( X ) = f ( x ) when X = xx T ⋆ Compatibility: g ( X ) = E ( f ( x )) when x ∼ N (0 , X ) Key question: • how good is the approximate solution x ? • can we bound f ( x ) /f ∗ by a constant? 11
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang Summary of Existing Results Assume • A i , ¯ A i � 0 , i = 0 , 1 , 2 , ..., m • B j �� 0 indefinite, j = 0 , 1 , 2 , ..., d R , d = 1 or R or C , d ≥ 2 R , d = 0 C , d = 0 , 1 min w H A 0 w Θ( m 2 ) ∞ Θ( m ) s . t . w H A i w ≥ 1 , w H B j w ≥ 1 max w H B 0 w Θ(log − 1 m ) Θ(log − 1 m ) ∞ s . t . w H A i w ≤ 1 , w H B j w ≤ 1 w H A i w max min Θ( m 2 ) w H ¯ Θ( m ) N.A. A i w + σ 2 1 ≤ i ≤ m s.t. � w � 2 ≤ P Blue: NRT’99 , Red: LSTZ’06, CLC’07, HLNZ’07 12
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang SDP Relaxation for Quartic Optimization Consider the first level SOS hierarchy so that x i x j �→ X ij , X � 0 . Under this mapping, each quartic term is mapped, non-uniquely , to a quadratic term, e.g., X 12 X 34 x 1 x 2 x 3 x 4 �→ X 13 X 24 X 14 X 23 • Which one should we use? • Should we choose a convex combination of the three choices? • Does it matter? 13
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang It Matters! Consider the following quartic optimization problem in R 4 : f ( x ) = ( x 1 x 2 ) 2 minimize (3) x 2 x 2 1 ≥ 1 , 2 ≥ 1 . subject to Under the matrix lifting transformation X = xx T , (3) is relaxed to g ( X ) = X 2 minimize 12 subject to X 11 ≥ 1 , X 22 ≥ 1 , X � 0 . • It can be checked ⋆ f min = 1 ⋆ g min = g ( I ) = 0 since X = I is a feasible solution. • This shows that the approximation ratio is unbounded! f min = ∞ . (4) g min 14
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang It Matters! • On the other hand, consider the symmetric mapping x i x j x ℓ x m �→ 1 3( X ij X ℓm + X iℓ X jm + X im X jℓ ) . Under this mapping, the quartic objective function f ( x ) = x 2 1 x 2 2 is relaxed to h ( x ) = 1 3( X 11 X 22 + 2 X 2 12 ) . • Let h min := minimize h ( X ) subject to X 11 ≥ 1 , X 22 ≥ 1 , X � 0 . • Notice that h min = h ( I ) = 1 3 , implying f min = 1 = 3 , 1 h min 3 which is indeed finite. 15
SDP Relaxation for Quartic Optimization Zhi-Quan Luo & Shuzhong Zhang SDP Relaxation for Quartic Optimization • Suppose g ( X ) is a quadratic function to be used as a relaxation of the quartic function f ( x ) . Then g ( X ) should satisfy � a ijkℓ x i x j x k x ℓ , whenever X = xx T . consistency property: g ( X ) = f ( x ) = 1 ≤ i,j,k,ℓ ≤ n • There are many quadratic functions g ( X ) satisfying this property, e.g. X ij X kℓ x i x j x k x ℓ �→ X ik X jℓ X iℓ X jk • Which one should we pick? Goal: pick one that ensures good approximation of quartic problem (1). 16
Recommend
More recommend
