Composite L q (0 < q < 1) Minimization over Polyhedron Ya-Feng Liu State Key Laboratory of Scientific/Engineering Computing, Institute of Computational Mathematics and Scientific/Engineering Computing, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China Email: yafliu@lsec.cc.ac.cn Joint Work with Shiqian Ma, Yu-Hong Dai, and Shuzhong Zhang 2014 Workshop on Optimization for Modern Computation Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 1 / 51
Polyhedral Constrained Composite L q Minimization Polyhedral constrained composite L q (0 < q < 1) minimization problem F ( x ) := � max { b − Ax , 0 }� q min q + h ( x ) x ∈ R N (1) s.t. x ∈ X . - A = [ a 1 , a 2 , ..., a M ] T ∈ R M × N , b = [ b 1 , b 2 , ..., b M ] T ∈ R M ; - h ( x ) : continuously differentiable satisfying �∇ h ( x ) − ∇ h ( y ) � 2 ≤ L h � x − y � 2 , ∀ x , y ∈ X ; - X ⊆ R N : a general polyhedral set. Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 2 / 51
Two Extremes � max { b − Ax , 0 }� q min q + h ( x ) x ∈ R N s.t. x ∈ X . - as q → 0 , the above L q minimization problem approaches min � max { b − Ax , 0 }� 0 + h ( x ) x ∈ R N s.t. x ∈ X . - as q → 1 , the above L q minimization problem approaches min � max { b − Ax , 0 }� 1 + h ( x ) x ∈ R N s.t. x ∈ X . Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 3 / 51
Outline Motivated Applications Related Works Exact Recovery Computational Complexity Optimality Conditions Algorithmic Framework & Analysis Simulation Results (NOT Covered) Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 4 / 51
Two Motivated Applications Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 5 / 51
Wireless Communications SINR at receiver k in the K -link SISO interference channel: g kk p k SINR k := ≥ γ k , k = 1 , 2 , ..., K � g kj p j + η k j � = k ¯ p k ≥ p k ≥ 0 , k = 1 , 2 , ..., K - p k : transmission power at transmitter k - g kj ≥ 0 : channel gain from transmitter j to receiver k - η k > 0 : noise power of link k - γ k > 0 : SINR target of link k - ¯ p k > 0 : power budget at transmitter k Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 6 / 51
Joint Power and Admission Control Infeasibility issues of the linear system SINR k ≥ γ k , ¯ p k ≥ p k ≥ 0 , k = 1 , 2 , . . . , K - mutual interference among different links - individual power budget constraints Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 7 / 51
Joint Power and Admission Control Infeasibility issues of the linear system SINR k ≥ γ k , ¯ p k ≥ p k ≥ 0 , k = 1 , 2 , . . . , K - mutual interference among different links - individual power budget constraints The admission control is necessary to determine the connections to be rejected. Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 7 / 51
Joint Power and Admission Control Infeasibility issues of the linear system SINR k ≥ γ k , ¯ p k ≥ p k ≥ 0 , k = 1 , 2 , . . . , K - mutual interference among different links - individual power budget constraints The admission control is necessary to determine the connections to be rejected. Joint power and admission control (JPAC): - the admitted links should be satisfied with their required SINR targets - the number of admitted (removed) links should be maximized (minimized) - the total transmission power to support the admitted links should be minimized Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 7 / 51
Normalized Channel Two equivalent equations: p k ⇔ 0 ≤ x k := p k - power constraint: 0 ≤ p k ≤ ¯ p k ≤ 1 ¯ g kk p k x k - SINR constraint: ≥ γ k ⇔ ≥ 1 γ k g kj ¯ p k x j + γ k η k p j � g kj p j + η k � g kk ¯ g kk ¯ p k j � = k j � = k Normalized channel: � γ 1 η 1 � T p 1 , γ 2 η 2 p 2 , · · · , γ K η K - noise vector b = > 0 g 11 ¯ g 22 ¯ g KK ¯ p K � T � p 1 p 1 , p 2 p 2 , · · · , p K - power allocation vector x = ¯ ¯ ¯ p K - channel gain matrix A with its ( k , j )-th entry − γ k g kj ¯ p j if k � = j ; p k , g kk ¯ a kj = 1 , if k = j . Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 8 / 51
Composite L q Minimization Formulation Simple to check g kk p k ≥ γ k ⇐ ⇒ ( b − Ax ) k ≤ 0 � g kj p j + η k j � = k Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 9 / 51
Composite L q Minimization Formulation Simple to check g kk p k ≥ γ k ⇐ ⇒ ( b − Ax ) k ≤ 0 � g kj p j + η k j � = k The JPAC problem can be formulated as [L.-Dai-Luo, 2013] � max { b − Ax , 0 }� q p T x min q + ρ ¯ x (2) s.t. 0 ≤ x ≤ e . Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 9 / 51
Support Vector Machine: Linearly Separable Data � M s m ∈ R N − 1 , y m ∈ R � Given a database m =1 , where s m is called example and y m is the label associated with s m . s = [ s T , 1] T ∈ R N s T x with ˆ Find a linear discriminant function ℓ ( s ) = ˆ - all data are correctly classified - the margin of the hyperplane ℓ that separates the two classes is maximized If the data are linearly separable, the above problem can be formulated as N − 1 1 � x 2 min n 2 x n =1 s T s.t. y m ˆ m x ≥ 1 , m = 1 , 2 , . . . , M . Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 10 / 51
Support Vector Machine: Not Linearly Separable Data Data are often NOT linearly separable in practice, and thus the above problem is not feasible. For the not linearly separable data, we can solve the following model instead: M N − 1 � q + ρ � � s T x 2 � min max 1 − y m ˆ m x , 0 n . 2 x m =1 n =1 The above problem with q = 1 is called the soft-margin SVM in [Cortes-Vapnik, 1995]. Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 11 / 51
Related Works Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 12 / 51
Existing Works I ρ 2 � Ax − b � 2 + � x � q min q x - Lower bound theory [Chen-Xu-Ye, 2010] - Strong NP-hardness [Chen-Ge-Wang-Ye, 2014] - Iterative reweighted L 1 and L 2 minimization algorithms [Xu-Chang-Xu-Zhang, 2012; Lai-Xu-Yin, 2013;...] Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 13 / 51
Existing Works I ρ 2 � Ax − b � 2 + � x � q min q x - Lower bound theory [Chen-Xu-Ye, 2010] - Strong NP-hardness [Chen-Ge-Wang-Ye, 2014] - Iterative reweighted L 1 and L 2 minimization algorithms [Xu-Chang-Xu-Zhang, 2012; Lai-Xu-Yin, 2013;...] � x � q min q s.t. Ax = b x - Sufficient conditions in recovering the sparsest solution [Chartrand, 2007; Chartrand-Staneva, 2008; Foucart-Lai, 2009] - Strong NP-hardness and a potential reduction algorithm [Ge-Jiang-Ye, 2011] - Iterative reweighted minimization methods [Chartrand-Yin, 2008; Daubechies et al., 2010; ...] - Extend to the matrix case [Ji-Sze-Zhou-So-Ye, 2013] Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 13 / 51
Existing Works II h ( x ) + � x � q min (3) q x Smoothing quadratic regularization (SQR) algorithm and O ( ǫ − 2 ) worst-case iteration complexity analysis [Bian-Chen, 2013] First and second order interior-point methods, O ( ǫ − 2 ) and O ( ǫ − 3 / 2 ) iteration complexity results [Bian-Chen-Ye, 2014] Lower bound theory, iterative reweighted minimization methods, unified global convergence analysis [Lv, 2012] Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 14 / 51
Existing Works III M � | a T m x | q min h ( x ) + (4) x m =1 Second order necessary and sufficient conditions [Chen-Niu-Yuan, 2013] Smoothing trust region Newton (STRN) method [Chen-Niu-Yuan, 2013] An SQR algorithm and O ( ǫ − 2 ) iteration complexity analysis [Bian-Chen, 2014] Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 15 / 51
There Are More in This Workshop! “A Smoothing Majorization Method for ℓ 2 - ℓ p Matrix Minimization” [Zhang] “An Improved Algorithm for the L 2 - L p Minimization Problem [Ge] “ p -Norm Constrained Quadratic Programming: Conic Approximation Methods” [Xing] · · · · · · Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 16 / 51
Some Remarks The definitions of ǫ -KKT points in the aforementioned works are different and thus are not comparable to each other. Ya-Feng Liu (LSEC ICMSEC AMSS CAS) Polyhedral Constrained Composite Lq Minimization Sept. 3, 2014, PKU 17 / 51
Recommend
More recommend
