2. Convex sets affine and convex sets some important examples - PowerPoint PPT Presentation

Convex Optimization — Boyd & Vandenberghe 2. Convex sets • affine and convex sets • some important examples • operations that preserve convexity • generalized inequalities • separating and supporting hyperplanes • dual cones and generalized inequalities 2–1

Affine set line through x 1 , x 2 : all points x = θx 1 + (1 − θ ) x 2 ( θ ∈ R ) x 1 θ = 1 . 2 θ = 1 θ = 0 . 6 x 2 θ = 0 θ = − 0 . 2 affine set : contains the line through any two distinct points in the set example : solution set of linear equations { x | Ax = b } (conversely, every affine set can be expressed as solution set of system of linear equations) Convex sets 2–2

Convex set line segment between x 1 and x 2 : all points x = θx 1 + (1 − θ ) x 2 with 0 ≤ θ ≤ 1 convex set : contains line segment between any two points in the set x 1 , x 2 ∈ C, 0 ≤ θ ≤ 1 = ⇒ θx 1 + (1 − θ ) x 2 ∈ C examples (one convex, two nonconvex sets) Convex sets 2–3

Convex combination and convex hull convex combination of x 1 ,. . . , x k : any point x of the form x = θ 1 x 1 + θ 2 x 2 + · · · + θ k x k with θ 1 + · · · + θ k = 1 , θ i ≥ 0 convex hull conv S : set of all convex combinations of points in S Convex sets 2–4

Convex cone conic (nonnegative) combination of x 1 and x 2 : any point of the form x = θ 1 x 1 + θ 2 x 2 with θ 1 ≥ 0 , θ 2 ≥ 0 x 1 x 2 0 convex cone : set that contains all conic combinations of points in the set Convex sets 2–5

Hyperplanes and halfspaces hyperplane : set of the form { x | a T x = b } ( a � = 0 ) a x 0 x a T x = b halfspace: set of the form { x | a T x ≤ b } ( a � = 0 ) a a T x ≥ b x 0 a T x ≤ b • a is the normal vector • hyperplanes are affine and convex; halfspaces are convex Convex sets 2–6

Euclidean balls and ellipsoids (Euclidean) ball with center x c and radius r : B ( x c , r ) = { x | � x − x c � 2 ≤ r } = { x c + ru | � u � 2 ≤ 1 } ellipsoid: set of the form { x | ( x − x c ) T P − 1 ( x − x c ) ≤ 1 } with P ∈ S n ++ ( i.e. , P symmetric positive definite) x c other representation: { x c + Au | � u � 2 ≤ 1 } with A square and nonsingular Convex sets 2–7

Norm balls and norm cones norm: a function � · � that satisfies • � x � ≥ 0 ; � x � = 0 if and only if x = 0 • � tx � = | t | � x � for t ∈ R • � x + y � ≤ � x � + � y � notation: � · � is general (unspecified) norm; � · � symb is particular norm norm ball with center x c and radius r : { x | � x − x c � ≤ r } 1 norm cone: { ( x, t ) | � x � ≤ t } 0 . 5 t Euclidean norm cone is called second- order cone 0 1 1 0 0 x 2 − 1 − 1 x 1 norm balls and cones are convex Convex sets 2–8

Polyhedra solution set of finitely many linear inequalities and equalities Ax � b, Cx = d ( A ∈ R m × n , C ∈ R p × n , � is componentwise inequality) a 1 a 2 P a 5 a 3 a 4 polyhedron is intersection of finite number of halfspaces and hyperplanes Convex sets 2–9

Positive semidefinite cone notation: • S n is set of symmetric n × n matrices + = { X ∈ S n | X � 0 } : positive semidefinite n × n matrices • S n X ∈ S n z T Xz ≥ 0 for all z ⇐ ⇒ + S n + is a convex cone ++ = { X ∈ S n | X ≻ 0 } : positive definite n × n matrices • S n 1 � � x y 0 . 5 ∈ S 2 z example: + y z 0 1 1 0 0 . 5 y x − 1 0 Convex sets 2–10

Operations that preserve convexity practical methods for establishing convexity of a set C 1. apply definition x 1 , x 2 ∈ C, 0 ≤ θ ≤ 1 = ⇒ θx 1 + (1 − θ ) x 2 ∈ C 2. show that C is obtained from simple convex sets (hyperplanes, halfspaces, norm balls, . . . ) by operations that preserve convexity • intersection • affine functions • perspective function • linear-fractional functions Convex sets 2–11

Intersection the intersection of (any number of) convex sets is convex example: S = { x ∈ R m | | p ( t ) | ≤ 1 for | t | ≤ π/ 3 } where p ( t ) = x 1 cos t + x 2 cos 2 t + · · · + x m cos mt for m = 2 : 2 1 1 p ( t ) 0 S x 2 0 − 1 − 1 − 2 0 π/ 3 2 π/ 3 π − 2 − 1 0 1 2 x 1 t Convex sets 2–12

Affine function suppose f : R n → R m is affine ( f ( x ) = Ax + b with A ∈ R m × n , b ∈ R m ) • the image of a convex set under f is convex S ⊆ R n convex = ⇒ f ( S ) = { f ( x ) | x ∈ S } convex • the inverse image f − 1 ( C ) of a convex set under f is convex C ⊆ R m convex f − 1 ( C ) = { x ∈ R n | f ( x ) ∈ C } convex = ⇒ examples • scaling, translation, projection • solution set of linear matrix inequality { x | x 1 A 1 + · · · + x m A m � B } (with A i , B ∈ S p ) • hyperbolic cone { x | x T Px ≤ ( c T x ) 2 , c T x ≥ 0 } (with P ∈ S n + ) Convex sets 2–13

Perspective and linear-fractional function perspective function P : R n +1 → R n : P ( x, t ) = x/t, dom P = { ( x, t ) | t > 0 } images and inverse images of convex sets under perspective are convex linear-fractional function f : R n → R m : f ( x ) = Ax + b dom f = { x | c T x + d > 0 } c T x + d, images and inverse images of convex sets under linear-fractional functions are convex Convex sets 2–14

example of a linear-fractional function 1 f ( x ) = x 1 + x 2 + 1 x 1 1 x 2 x 2 C 0 0 f ( C ) − 1 − 1 − 1 0 1 − 1 0 1 x 1 x 1 Convex sets 2–15

Generalized inequalities a convex cone K ⊆ R n is a proper cone if • K is closed (contains its boundary) • K is solid (has nonempty interior) • K is pointed (contains no line) examples + = { x ∈ R n | x i ≥ 0 , i = 1 , . . . , n } • nonnegative orthant K = R n • positive semidefinite cone K = S n + • nonnegative polynomials on [0 , 1] : K = { x ∈ R n | x 1 + x 2 t + x 3 t 2 + · · · + x n t n − 1 ≥ 0 for t ∈ [0 , 1] } Convex sets 2–16

generalized inequality defined by a proper cone K : x � K y ⇐ ⇒ y − x ∈ K, x ≺ K y ⇐ ⇒ y − x ∈ int K examples • componentwise inequality ( K = R n + ) x � R n + y ⇐ ⇒ x i ≤ y i , i = 1 , . . . , n • matrix inequality ( K = S n + ) X � S n + Y ⇐ ⇒ Y − X positive semidefinite these two types are so common that we drop the subscript in � K properties: many properties of � K are similar to ≤ on R , e.g. , x � K y, u � K v = ⇒ x + u � K y + v Convex sets 2–17

Minimum and minimal elements � K is not in general a linear ordering : we can have x �� K y and y �� K x x ∈ S is the minimum element of S with respect to � K if y ∈ S = ⇒ x � K y x ∈ S is a minimal element of S with respect to � K if y ∈ S, y � K x = ⇒ y = x example ( K = R 2 + ) S 2 S 1 x 2 x 1 is the minimum element of S 1 x 2 is a minimal element of S 2 x 1 Convex sets 2–18

Separating hyperplane theorem if C and D are nonempty disjoint convex sets, there exist a � = 0 , b s.t. a T x ≤ b for x ∈ C, a T x ≥ b for x ∈ D a T x ≥ b a T x ≤ b D C a the hyperplane { x | a T x = b } separates C and D strict separation requires additional assumptions ( e.g. , C is closed, D is a singleton) Convex sets 2–19

Supporting hyperplane theorem supporting hyperplane to set C at boundary point x 0 : { x | a T x = a T x 0 } where a � = 0 and a T x ≤ a T x 0 for all x ∈ C a x 0 C supporting hyperplane theorem: if C is convex, then there exists a supporting hyperplane at every boundary point of C Convex sets 2–20

Dual cones and generalized inequalities dual cone of a cone K : K ∗ = { y | y T x ≥ 0 for all x ∈ K } examples + : K ∗ = R n • K = R n + + : K ∗ = S n • K = S n + • K = { ( x, t ) | � x � 2 ≤ t } : K ∗ = { ( x, t ) | � x � 2 ≤ t } • K = { ( x, t ) | � x � 1 ≤ t } : K ∗ = { ( x, t ) | � x � ∞ ≤ t } first three examples are self-dual cones dual cones of proper cones are proper, hence define generalized inequalities: y T x ≥ 0 for all x � K 0 y � K ∗ 0 ⇐ ⇒ Convex sets 2–21

Minimum and minimal elements via dual inequalities minimum element w.r.t. � K x is minimum element of S iff for all λ ≻ K ∗ 0 , x is the unique minimizer S of λ T z over S x minimal element w.r.t. � K • if x minimizes λ T z over S for some λ ≻ K ∗ 0 , then x is minimal λ 1 x 1 S λ 2 x 2 • if x is a minimal element of a convex set S , then there exists a nonzero λ � K ∗ 0 such that x minimizes λ T z over S Convex sets 2–22

optimal production frontier • different production methods use different amounts of resources x ∈ R n • production set P : resource vectors x for all possible production methods • efficient (Pareto optimal) methods correspond to resource vectors x that are minimal w.r.t. R n + fuel example ( n = 2 ) x 1 , x 2 , x 3 are efficient; x 4 , x 5 are not P x 1 x 5 x 4 x 2 λ x 3 labor Convex sets 2–23