Convex optimization problems (II) Lijun Zhang zlj@nju.edu.cn - PowerPoint PPT Presentation

Convex optimization problems (II) Lijun Zhang zlj@nju.edu.cn http://cs.nju.edu.cn/zlj

Outline  Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

Linear Optimization Problems  Linear Program (LP) � ��� and ���   It is common to omit the constant  Maximization problem with affine objective and constraint functions is also an LP  The feasible set of LP is a polyhedron

Linear Optimization Problems  Geometric Interpretation of an LP  The objective 𝑑 � 𝑦 is linear, so its level curves are hyperplanes orthogonal to 𝑑  𝑦 ∗ is as far as possible in the direction �𝑑

Two Special Cases of LP  Standard Form LP �  The only inequalities are  Inequality Form LP �  No equality constraint

Converting to Standard Form  Conversion � �  To use an algorithm for standard LP  Introduce Slack Variables � �

Converting to Standard Form  Decompose 𝑦 � 𝑦 � � 𝑦 � , 𝑦 � , 𝑦 � ≽ 0  Standard Form LP � � � � � � � � � � �

Example  Diet Problem Choose nonnegative quantities 𝑦 � , … , 𝑦 � of 𝑜 foods  One unit of food 𝑘 contains amount 𝑏 �� of nutrient  𝑗 , and costs 𝑑 � Healthy diet requires nutrient 𝑗 in quantities at  least 𝑐 �  Determine the cheapest diet that satisfies the nutritional requirements. �

Example  Chebyshev Center of a Polyhedron  Find the largest Euclidean ball that lies in the polyhedron � 𝑦 � 𝑐 � , 𝑗 � 1, … , 𝑛� 𝒬 � �𝑦 ∈ 𝐒 � |𝑏 �  The center of the optimal ball is called the Chebyshev center of the polyhedron  Represent the ball as ℬ � �𝑦 � � 𝑣| 𝑣 � � 𝑠�  𝑦 � ∈ 𝐒 � and 𝑠 are variables, and we wish to maximize 𝑠 subject to ℬ ∈ 𝒬 � 𝑦 � � 𝑣 � 𝑐 � , 𝑣 � � 𝑠 ⟺ � 𝑦 � 𝑐 � ⟺ 𝑏 �  ∀𝑦 ∈ ℬ, 𝑏 � � 𝑦 � � 𝑠 𝑏 � 𝑏 � � � 𝑐 �

Example  Chebyshev Center of a Polyhedron  Find the largest Euclidean ball that lies in the polyhedron � 𝑦 � 𝑐 � , 𝑗 � 1, … , 𝑛� 𝒬 � �𝑦 ∈ 𝐒 � |𝑏 �  The center of the optimal ball is called the Chebyshev center of the polyhedron  Represent the ball as ℬ � �𝑦 � � 𝑣| 𝑣 � � 𝑠�  𝑦 � ∈ 𝐒 � and 𝑠 are variables, and we wish to maximize 𝑠 subject to ℬ ∈ 𝒬 max 𝑠 � 𝑦 � � 𝑠 𝑏 � s. t. 𝑏 � � � 𝑐 � , 𝑗 � 1, … , 𝑛

Example  Chebyshev Inequalities  is a random variable on � � �  � , � �  is a linear function of � � ���  Prior knowledge is given as � 𝑞 � 𝛾 � , 𝛽 � � 𝑏 � 𝑗 � 1, … , 𝑛 �  To find a lower bound of � � � 𝑞 min 𝑏 � 𝑞 ≽ 0, 𝟐 � 𝑞 � 1 s. t. � 𝑞 � 𝛾 � , 𝛽 � � 𝑏 � 𝑗 � 1, … , 𝑛

Example  Piecewise-linear Minimization  Consider the (unconstrained) problem � � � ���,…,�  The epigraph problem � � � ���,…,�  An LP problem � � �

Linear-fractional Programming  Linear-fractional Program min 𝑔 � �𝑦� s. t. 𝐻𝑦 ≼ ℎ 𝐵𝑦 � 𝑐  The objective function is a ratio of affine � 𝑦 � 𝑑 � 𝑦 � 𝑒 functions 𝑔 𝑓 � 𝑦 � 𝑔  The domain is � � �𝑦|𝑓 � 𝑦 � 𝑔 � 0� dom 𝑔  A quasiconvex optimization problem

Linear-fractional Programming  Transforming to a linear program 𝑑 � 𝑧 � 𝑒𝑨 min � 𝑦 � 𝑑 � 𝑦 � 𝑒 min 𝑔 s. t. 𝐻𝑧 � ℎ𝑨 ≼ 0 𝑓 � 𝑦 � 𝑔 𝐵𝑧 � 𝑐𝑨 � 0 s. t. 𝐻𝑦 ≼ ℎ 𝑓 � 𝑧 � 𝑔𝑨 � 1 𝐵𝑦 � 𝑐 𝑨 � 0  Proof � � 𝑦 is feasible in LFP ⇒ 𝑧 � � � ��� , 𝑨 � � � ��� is feasible , 𝑑 � 𝑧 � 𝑒𝑨 � 𝑔 � �𝑦� ⇒ the optimal value of LFP is in LP greater than or equal to the optimal value of LP

Linear-fractional Programming  Transforming to a linear program 𝑑 � 𝑧 � 𝑒𝑨 min � 𝑦 � 𝑑 � 𝑦 � 𝑒 min 𝑔 s. t. 𝐻𝑧 � ℎ𝑨 ≼ 0 𝑓 � 𝑦 � 𝑔 𝐵𝑧 � 𝑐𝑨 � 0 s. t. 𝐻𝑦 ≼ ℎ 𝑓 � 𝑧 � 𝑔𝑨 � 1 𝐵𝑦 � 𝑐 𝑨 � 0  Proof ⁄ �𝑧, 𝑨� is feasible in LP and 𝑨 � 0 ⇒ 𝑦 � 𝑧 𝑨 is feasible � 𝑦 � 𝑑 � 𝑧 � 𝑒𝑨 ⇒ the optimal value of FLP is , 𝑔 in LFP less than or equal to the optimal value of LP �𝑧, 𝑨� is feasible in LP , 𝑨 � 0 and 𝑦 � is feasible in LFP ⇒ 𝑦 � 𝑦 � � 𝑢𝑧 is feasible in LFP for all 𝑢 � 0 , � �𝑦 � � 𝑢𝑧� � 𝑑 � 𝑧 � 𝑒𝑨 �→� 𝑔 lim

Generalized Linear-fractional Programming  Generalized Linear-fractional Program � 𝑦 � 𝑒 � 𝑑 � 𝑔 � 𝑦 � max � 𝑦 � 𝑔 𝑓 � ���,…,� � �  � � �  A quasiconvex optimization problem  Von Neumann Growth Problem � 𝑦 � ⁄ max ���,…,� 𝑦 � min 𝑦 � ≽ 0 s. t. 𝐶𝑦 � ≼ 𝐵𝑦

Generalized Linear-fractional Programming  Von Neumann Growth Problem � 𝑦 � ⁄ max ���,…,� 𝑦 � min 𝑦 � ≽ 0 s. t. 𝐶𝑦 � ≼ 𝐵𝑦  𝑦, 𝑦 � ∈ 𝐒 � : activity levels of 𝑜 sectors, in current and next period 𝐵𝑦 � , 𝐶𝑦 � � : produced and consumed  amounts of good 𝑗  𝐶𝑦 � ≼ 𝐵𝑦 : goods consumed in the next period cannot exceed the goods produced in the current period � 𝑦 � ⁄  𝑦 � growth rate of sector 𝑗

Outline  Linear Optimization Problems  Quadratic Optimization Problems  Geometric Programming  Generalized Inequality Constraints  Vector Optimization

Quadratic Optimization Problems  Quadratic Program (QP) ⁄ �𝑦 � 𝑄𝑦 � 𝑟 � 𝑦 � 𝑠 min �1 2 s. t. 𝐻𝑦 ≼ ℎ 𝐵𝑦 � 𝑐 ��� and � ���  �  The objective function is (convex) quadratic  The constraint functions are affine  When , QP becomes LP

Quadratic Optimization Problems  Geometric Illustration of QP  The feasible set 𝒬 is a polyhedron  The contour lines of the objective function are shown as dashed curves.

Quadratic Optimization Problems  Quadratically Constrained Quadratic Program (QCQP) � 𝑦 � 𝑠 𝑦 � 𝑄 � 𝑦 � 𝑟 � min 1 2 ⁄ � � 𝑦 � 𝑠 𝑦 � 𝑄 � 𝑦 � 𝑟 � ⁄ s. t. 1 2 � � 0, 𝑗 � 1, … , 𝑛 𝐵𝑦 � 𝑐 � , 𝑗 � 0, … , 𝑛  𝑄 � ∈ 𝐓 �  The inequality constraint functions are (convex) quadratic  The feasible set is the intersection of ellipsoids (when 𝑄 � ≻ 0 ) and an affine set  Include QP as a special case

Examples  Least-squares and Regression � � 𝑦 � 𝐵 � 𝐵𝑦 � 2𝑐 � 𝐵𝑦 � 𝑐 � 𝑐 min 𝐵𝑦 � 𝑐 �  Analytical solution: 𝑦 � 𝐵 � 𝑐  Can add linear constraints, e.g., 𝑚 ≼ 𝑦 ≼ 𝑣  Distance Between Polyhedra � � � � � � � � � �  Find the distance between the polyhedra 𝒬 � � �𝑦|𝐵 � 𝑦 ≼ 𝑐 � � and 𝒬 � � �𝑦|𝐵 � 𝑦 ≼ 𝑐 � � dist 𝒬 � , 𝒬 � � inf 𝑦 � � 𝑦 � � 𝑦 � ∈ 𝒬 � , 𝑦 � ∈ 𝒬 �

Example  Bounding Variance  is a random variable on � � �  � , �  The variance of a random variable � � � 𝐅𝑔 � � 𝐅𝑔 � � � � 𝑞 � 𝑔 � � 𝑔 � 𝑞 � � ��� ���  Maximize the variance � � � � 𝑞 � max � 𝑔 � � 𝑔 � 𝑞 � � ��� ��� 𝑞 ≽ 0, 𝟐 � 𝑞 � 1 s. t. � 𝑞 � 𝛾 � , 𝑗 � 1, … , 𝑛 𝛽 � � 𝑏 �

Second-order Cone Programming  Second-order Cone Program (SOCP) 𝑔 � 𝑦 min � 𝑦 � 𝑒 � , s. t. 𝐵 � 𝑦 � 𝑐 � � � 𝑑 � 𝑗 � 1, … , 𝑛 𝐺𝑦 � 𝑕  𝐵 � ∈ 𝐒 � � �� , 𝐺 ∈ 𝐒 ���  Second-order Cone (SOC) constraint: 𝐵𝑦 � 𝑐 � � 𝑑 � 𝑦 � 𝑒 where 𝐵 ∈ 𝐒 ��� , is same as requiring 𝐵𝑦 � 𝑐, 𝑑 � 𝑦 � 𝑒 ∈ SOC in 𝐒 ��� 𝑦, 𝑢 ∈ 𝐒 ��� | 𝑦 � � 𝑢 SOC � � 𝐽 𝑦 𝑢 | 𝑦 𝑦 0 � 𝑢 � 0, 𝑢 � 0 𝑢 0 �1

Second-order Cone Programming  Second-order Cone Program (SOCP) 𝑔 � 𝑦 min � 𝑦 � 𝑒 � , s. t. 𝐵 � 𝑦 � 𝑐 � � � 𝑑 � 𝑗 � 1, … , 𝑛 𝐺𝑦 � 𝑕  𝐵 � ∈ 𝐒 � � �� , 𝐺 ∈ 𝐒 ���  Second-order Cone (SOC) constraint: 𝐵𝑦 � 𝑐 � � 𝑑 � 𝑦 � 𝑒 where 𝐵 ∈ 𝐒 ��� , is same as requiring 𝐵𝑦 � 𝑐, 𝑑 � 𝑦 � 𝑒 ∈ SOC in 𝐒 ���  If 𝑑 � � 0, 𝑗 � 1, … , 𝑛 , it reduces to QCQP by squaring each inequality constraint  More general than QCQP and LP