Convex Optimization 2. Convex Sets Prof. Ying Cui Department of - PowerPoint PPT Presentation

Convex Optimization 2. Convex Sets Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2018 SJTU Ying Cui 1 / 33

Outline Affine and convex sets Some important examples Operations that preserve convexity Generalized inequalities Separating and supporting hyperplanes Dual cones and generalized inequalities SJTU Ying Cui 2 / 33

Lines and line segments ◮ line passing through two distinct points x 1 , x 2 ∈ R n , x 1 � = x 2 : points of the form ◮ sum of x 1 scaled by θ and x 2 scaled by 1 − θ y = θ x 1 + (1 − θ ) x 2 , θ ∈ R ◮ sum of base point x 2 and direction x 1 − x 2 scaled by θ y = x 2 + θ ( x 1 − x 2 ) , θ ∈ R ◮ line segment between two distinct points x 1 , x 2 ∈ R n , x 1 � = x 2 : points of the form y = θ x 1 + (1 − θ ) x 2 , θ ∈ [0 , 1] θ = 1 . 2 x 1 θ = 1 θ = 0 . 6 x 2 θ = 0 θ = − 0 . 2 Figure 2.1 The line passing through x 1 and x 2 is described parametrically by θx 1 + (1 − θ ) x 2 , where θ varies over R . The line segment between x 1 and x 2 , which corresponds to θ between 0 and 1, is shown darker. SJTU Ying Cui 3 / 33

Affine sets ◮ affine set: an affine set C ⊆ R n contains the line through any two distinct points in C ◮ ∀ x 1 , x 2 ∈ C and θ ∈ R , θ x 1 + (1 − θ ) x 2 ∈ C ◮ affine combination of points x 1 , · · · , x k : a point of the form θ 1 x 1 + · · · + θ k x k , where θ 1 , · · · , θ k ∈ R and θ 1 + · · · + θ k = 1 ◮ an affine set contains every affine combination of its points ◮ affine hull of set C ⊆ R n : the set of all affine combinations of points in C , i.e., aff C = { θ 1 x 1 + · · · + θ k x k | x 1 , · · · , x k ∈ C , θ 1 + · · · + θ k = 1 } ◮ aff C is the smallest affine set that contains C : if S is any affine set with C ⊆ S , then aff C ⊆ S ◮ example: empty set ∅ , any single point (i.e., singleton) { x 0 } , line, hyperplane, whole space R n , solution set of linear equations C = { x | Ax = b } ◮ solution set of a system of linear equations is an affine set ◮ every affine set can be expressed as the solution set of a system of linear equations SJTU Ying Cui 4 / 33

Convex sets ◮ convex set: a convex set C ⊆ R n contains the line segment between any two distinct points in C ◮ ∀ x 1 , x 2 ∈ C and θ ∈ [0 , 1], θ x 1 + (1 − θ ) x 2 ∈ C ◮ every affine set is also convex ◮ convex combination of points x 1 , · · · , x k : a point of the form θ 1 x 1 + · · · + θ k x k , where θ 1 , · · · , θ k ≥ 0 and θ 1 + · · · + θ k = 1 ◮ can be generalized to include infinite sums, integrals, and probability distributions ◮ a convex set contains every convex combination of its points ◮ convex hull of set C ⊆ R n : the set of all convex combinations of points in C , i.e., conv C = { θ 1 x 1 + · · · + θ k x k | x i ∈ C , θ i ≥ 0 , i = 1 , · · · , k , θ 1 + · · · + θ k = 1 } ◮ conv C is the smallest convex set that contains C : if B is any convex set with C ⊆ B , then conv C ⊆ B ◮ example: line segment, ray Figure 2.2 Some simple convex and nonconvex sets. Left. The hexagon, which includes its boundary (shown darker), is convex. Middle. The kidney Figure 2.3 The convex hulls of two sets in R 2 . Left. The convex hull of a shaped set is not convex, since the line segment between the two points in the set shown as dots is not contained in the set. Right. The square contains set of fifteen points (shown as dots) is the pentagon (shown shaded). Right. The convex hull of the kidney shaped set in figure 2.2 is the shaded set. some boundary points but not others, and is not convex. SJTU Ying Cui 5 / 33

Cones ◮ cone (or nonnegative homogeneous): a set C ⊆ R n satisfies ◮ ∀ x ∈ C and θ ≥ 0, θ x ∈ C ◮ convex cone: a convex cone C ⊆ R n is convex and a cone ◮ ∀ x 1 , x 2 ∈ C and θ 1 , θ 2 ≥ 0, θ 1 x 1 + θ 2 x 2 ∈ C ◮ conic combination (or nonnegative linear combination) of points x 1 , · · · , x k : a point of the form θ 1 x 1 + · · · + θ k x k , where θ 1 , · · · , θ k ≥ 0 ◮ can be generalized to include infinite sums and integrals ◮ a convex cone contains all conic combinations of its elements ◮ conic hull of set C ⊆ R n : set of all conic combinations of points in C , { θ 1 x 1 + · · · + θ k x k | x i ∈ C , θ i ≥ 0 , i = 1 , · · · , k } ◮ the smallest convex cone that contains C ◮ example: subspace, line passing through origin, ray with base origin x 1 x 2 0 Figure 2.4 The pie slice shows all points of the form θ 1 x 1 + θ 2 x 2 , where θ 1 , θ 2 ≥ 0. The apex of the slice (which corresponds to θ 1 = θ 2 = 0) is at 0 0 0; its edges (which correspond to θ 1 = 0 or θ 2 = 0) pass through the points x 1 and x 2 . Figure 2.5 The conic hulls (shown shaded) of the two sets of figure 2.3. SJTU Ying Cui 6 / 33

Hyperplanes and halfspaces ◮ hyperplane: set of form { x | a T x = b } ( a ∈ R n , a � = 0 , b ∈ R ) ◮ analytical interpretation: solution set of a nontrivial linear equation ◮ hyperplane is an affine set ◮ geometric interpretation: ◮ set of points with constant inner product b to given vector a ◮ hyperplane with normal vector a and offset from the origin determined by b : an offset x 0 plus all vectors orthogonal to normal vector a { x | a T ( x − x 0 ) = 0 } = x 0 + { v | a T v = 0 } � �� � � a ⊥ where x 0 is any point in the hyperplane satisfying a T x 0 = b and a ⊥ denotes the orthogonal complement of a a x 0 x a T x = b Figure 2.6 Hyperplane in R 2 , with normal vector a and a point x 0 in the hyperplane. For any point x in the hyperplane, x − x 0 (shown as the darker arrow) is orthogonal to a . SJTU Ying Cui 7 / 33

Hyperplanes and halfspaces ◮ halfspace: set of form { x | a T x ≤ b } ( a ∈ R n , a � = 0 , b ∈ R ) ◮ analytical interpretation: solution set of a nontrivial linear inequality ◮ halfspace is not an affine set, but a convex set ◮ geometric interpretation: an offset x 0 plus all vectors making an obtuse (or right) angle with outward normal vector a { x | a T ( x − x 0 ) ≤ 0 } = x 0 + { v | a T v ≤ 0 } where x 0 is any point in the hyperplane satisfying a T x 0 = b a a T x ≥ b x 0 a T x ≤ b Figure 2.7 A hyperplane defined by a T x = b in R 2 determines two halfspaces. The halfspace determined by a T x ≥ b (not shaded) is the halfspace extending in the direction a . The halfspace determined by a T x ≤ b (which is shown shaded) extends in the direction − a . The vector a is the outward normal of this halfspace. ◮ a hyperplane divides R n into two halfspaces ◮ the boundary of a halfspace is a hyperplane SJTU Ying Cui 8 / 33

Euclidean balls and ellipsoids ◮ (Euclidean) ball with center x c and radius r : B ( x c , r ) = { x | || x − x c || 2 ≤ r } = { x c + ru | || u || 2 ≤ 1 } ◮ || u || 2 = ( u T u ) 1 / 2 denotes the Euclidean norm ( l 2 norm) ◮ convex set ◮ ellipsoid with center x c : E = { x | ( x − x c ) T P − 1 ( x − x c ) ≤ 1 } , where P ∈ S n ++ E = { x c + Au | || u || 2 ≤ 1 } , where A ∈ S n ++ ◮ P determines how far the ellipsoid extends in every direction from x c , the lengths of the semi-axes of the ellipsoid are given by √ λ i , where λ i are eigenvalues of P ◮ when P = r 2 I , ellipsoid becomes ball ◮ when A = P 1 / 2 , two representations are the same ◮ convex set x c Figure 2.9 An ellipsoid in R 2 , shown shaded. The center x c is shown as a dot, and the two semi-axes are shown as line segments. SJTU Ying Cui 9 / 33

Norm balls and norm cones ◮ norm: a function || · || : R n → R (measure of length of vector) ◮ nonnegative: || x || ≥ 0 for all x ∈ R n ◮ definite: || x || = 0 only if x = 0 ◮ homogeneous: || tx || = | t ||| x || , for all x ∈ R n and t ∈ R ◮ triangle inequality: || x + y || ≤ || x || + || y || , for all x , y ∈ R n ◮ norm ball with center x c and radius r : { x | || x − x c || ≤ r } ◮ convex set ◮ norm cone: { ( x , t ) | || x || ≤ t } ⊆ R n +1 ◮ convex cone ◮ second-order (Euclidean norm) cone , i.e., { ( x , t ) | || x || 2 ≤ t } 1 0 . 5 t 0 1 1 0 0 x 2 − 1 − 1 x 1 2 ) 1 / 2 ≤ Figure 2.10 Boundary of second-order cone in R 3 , { ( x 1 , x 2 , t ) | ( x 2 1 + x 2 t } . SJTU Ying Cui 10 / 33

Polyhedra ◮ polyhedron: solution set of a finite number of linear inequalities and equalities (can be bounded or unbounded) P = { x | a T j x ≤ b j , j = 1 , · · · , m , c T j x = d j , j = 1 , · · · , p } = { x | Ax � b , Cx = d }  a T   c T  1 1 . . . . where A =  , C =  , � denotes vector inequality    .  .   a T c T m p ◮ intersection of a finite number of halfspaces and hyperplanes ◮ affine sets (e.g., subspaces, hyperplanes, lines), rays, line segments, and halfspaces are all polyhedra a 1 a 2 P a 5 a 3 a 4 Figure 2.11 The polyhedron P (shown shaded) is the intersection of five halfspaces, with outward normal vectors a 1 , . . . . , a 5 . SJTU Ying Cui 11 / 33