Convexity and Polyhedra Carlo Mannino (from Geir Dahl notes on - PowerPoint PPT Presentation

Convexity and Polyhedra Carlo Mannino (from Geir Dahl notes on convexity) University of Oslo, INF-MAT5360 - Autumn 2011 (Mathematical optimization)

Convex Sets Set C  R n is convex if (1-  ) x 1 +  x 2  C whenever x 1 , x 2  C 0 ≤  ≤ 1 (the segment joining x 1 , x 2 is contained in C) x 2 x 1 non-convex x 2 x 1 convex • Show that the unit ball B = { x  R n : || x || ≤ 1} is convex. (Hint use the triangle inequality || x+y || ≤ || x||+ ||y ||)

Half-spaces H = { x  R n : a T x ≤ a 0 } Example of convex sets: half-spaces H x 1  H  a T x 1 ≤ a 0  (1-  ) a T x 1 ≤ (1-  ) a 0 0 ≤  ≤ 1 x 2  H  a T x 2 ≤ a 0   a T x 2 ≤  a 0 x 2  H  a T ((1-  ) x 1 +  x 2 ) ≤ a 0 summing up (1-  ) x 1 +  x 2  H

Convex Cones The set of solutions to a linear system of equation is a polyhedron. H = { x  ℝ n : Ax ≤ b , -Ax ≤ -b } H = { x  ℝ n : Ax = b } Convex Cone : C ⊆ℝ n if 𝜇 1 x 1 + 𝜇 2 x 2 ∈ C whenever x 1 , x 2 ∈ C and 𝜇 1 , 𝜇 2 ≥ 0. Each convex cone is a convex set. (show) . Then C = { x  ℝ n : Ax ≤ 0} is a convex cone (show). Let A ∈ ℝ m,n 𝑢 Let x 1 ,…, x t ∈ ℝ n , and 𝜇 1 ,…, 𝜇 t ≥ 0 . The vector x = 𝜇 𝑘 x j is 𝑘=1 a nonnegative (or conical ) combination of x 1 ,…, x t The set C(x 1 ,…, x t ) of all nonnegative combinations of x 1 ,…, x t ∈ ℝ n is a convex cone (show), called finitely generated cone.

Linear Programming Property: C 1 , C 2 convex sets → C 1 ∩ C 2 convex (show!) x = ( x 1 , …, x n ) Linear programming: maximize c 1 x 1 + … + c n x n Subject to 𝑏 11 x 1 + … +𝑏 1𝑜 x n ≤ 𝑐 1 P ⋮ 𝑏 𝑛1 x 1 + +𝑏 𝑛𝑜 x n ≤ 𝑐 𝑛 x 1 , …, x n ≥ 0 max { c T x : x ϵ P }, with P = { x  R n : Ax ≤ b , x ≥ 0 } Find the optimum solution in P P intersection of a finite number of half-spaces: convex set ( polyhedron) The set of optimal solutions to a linear program is a polyhedron (show!)

Convex Combinations 𝑢 Let x 1 ,…, x t ∈ ℝ n , and 𝜇 1 ,…, 𝜇 t ≥ 0, such that 𝜇 𝑘 = 1. The vector 𝑘=1 𝑢 x = 𝜇 𝑘 𝑦 𝑘 is called convex combination of x 1 ,…, x t 𝑘=1 x 3 2 2 2 6 𝑦 3 + 6 𝑦 3 + 6 𝑦 3 x 5 x 4 2 1 3 𝑦 1 + 3 𝑦 2 x 2 x 1 1 2 3 𝑦 1 + 3 𝑦 2

Convex Combinations Theorem: a set C is convex if and only if it contains all convex combinations of its points. If C contains all convex combinations → it contains all convex combinations of any 2 points → C is convex Suppose C contains all convex combinations of t- 1 points. True if t ≤ 3 (since C convex). 𝑢 𝑢 Let x 1 ,…, x t ∈ ℝ n , and let x = 𝜇 𝑘 x j where 𝜇 1 ,…, 𝜇 t > 0, 𝜇 𝑘 = 1 𝑘=1 𝑘=1 𝑢 𝑢 x = 𝜇 1 x 1 + x j = 𝜇 1 x 1 + (1- 𝜇 1 ) 𝜇 𝑘 ( 𝜇 𝑘 /(1 − 𝜇 1 )) x j 𝑘=2 𝑘=2 𝑢 ( 𝜇 𝑘 /(1 − 𝜇 1 )) x j = y ∈ C 𝑢 𝑢 = 1 → 𝜇 𝑘 ( 𝜇 𝑘 /(1 − 𝜇 1 ) = 1 𝑘=2 𝑘=1 𝑘=2 x = 𝜇 1 x 1 + (1- 𝜇 1 ) y ∈ C

Convex and Conical Hull There many convex sets containing a given set of points S . The smallest is the set conv( S ) of all convex combinations of the points in S. conv( S ) is called convex hull of S The set cone ( S ) of all nonnegative (conical) combinations of points in S is called conical hull

Convex Hull Proposition 2.2.1 (Convex hull). ⊆ ℝ n . Then conv( S ) is Let S • equal to the intersection of all convex sets containing S . If S is finite, conv (S) is called polytope . Consider the following optimization problem: max { c T x : x ϵ P }, with P = conv(S), S = { x 1 ,…, x t } Let x * : c T x * = max { c T x : x ϵ S } = v ( x * optimum in S ) 𝑢 𝑢 For any y ϵ P there exist 𝜇 1 ,…, 𝜇 t ≥ 0, = 1, such that y = 𝜇 𝑘 𝜇 𝑘 x j 𝑘=1 𝑘=1 𝑢 𝑢 𝑢 𝑢 c T y = c T 𝜇 𝑘 x j = 𝜇 𝑘 c T x j ≤ 𝜇 𝑘 c T x* = 𝜇 𝑘 v = v 𝑘=1 𝑘=1 𝑘=1 𝑘=1 x * optimum in P

Affine independence 𝑢 A set of vectors x 1 ,…, x t ∈ ℝ n , are affinely independent if 𝜇 𝑘 𝑦 𝑘 = 0 𝑘=1 𝑢 and 𝜇 𝑘 = 0 imply 𝜇 1 =…= 𝜇 t = 0. 𝑘=1 Proposition 2.3.1 (Affine independence). The vectors x 1 ,…, x t ∈ ℝ n are affinely independent if and only if the t -1 vectors x 2 -x 1 ,…,x t -x 1 are linearly independent. Only if. x 1 ,…, x t ∈ ℝ n affinely independent and assume 𝜇 2 ,…, 𝜇 t ∈ ℝ n with 𝑢 𝑢 𝑢 -( + 𝜇 𝑘 )𝑦 1 𝜇 𝑘 𝑦 𝑘 = 0 𝜇 𝑘 (𝑦 𝑘 −𝑦 1 ) = 0 𝑘=2 𝑘=2 𝑘=2 𝑢 𝑢 -( + x 1 ,…, x t affinely independent 𝜇 𝑘 ) 𝜇 𝑘 = 0 and 𝑘=2 𝑘=2 𝜇 2 =…= 𝜇 t = 0 x 2 -x 1 ,…,x t -x 1 linearly independent.

Affine independence Proposition 2.3.1 (Affine independence). The vectors x 1 ,…, x t ∈ ℝ n are affinely independent if and only if the t-1 vectors x 2 -x 1 ,…,x t -x 1 are linearly independent. if. x 2 -x 1 ,…,x t -x 1 linearly independent. 𝑢 𝑢 𝑢 Assume = 0 and = 0. Then 𝜇 1 = - 𝜇 𝑘 𝑦 𝑘 𝜇 𝑘 𝜇 𝑘 𝑘=1 𝑘=1 𝑘=2 𝑢 𝑢 𝑢 𝑢 0 = = −( + = 𝜇 𝑘 𝑦 𝑘 𝜇 𝑘 )𝑦 1 𝜇 𝑘 𝑦 𝑘 𝜇 𝑘 (𝑦 𝑘 −𝑦 1 ) 𝑘=1 𝑘=2 𝑘=2 𝑘=2 As x 2 -x 1 ,…,x t -x 1 linearly independent 𝜇 2 =…= 𝜇 t = 0 𝑢 Also 𝜇 1 = - 𝜇 𝑘 = 0 𝑘=2 Corollary. There are at mots n+ 1 affinely independent vectors in ℝ n .

Dimension ⊆ ℝ n is the maximal The dimension dim( S ) of a set S • number of affinely independent points of S minus 1. Ex. S = { x 1 = (0,0), x 2 = (0,1), x 3 = (1,0)}.   0     dim( S ) = 2 ( x 2 – x 1 , x 3 – x 1 are linearly independent) 1   0   1       0  0  ⊆ ℝ n is the convex hull of a set S of affinely A simplex P • independent vectors in ℝ n

Caratheodory’s theorem ⊆ ℝ n . Then each Theorem. 2.5.1 ( Caratheodory’s theorem) Let S • x ∈ conv( S ) is the convex combination of m affinely independent points in S, with m ≤ n +1. x can be obtained as a convex combination of points in S Choose one with smallest number of points: 𝑢 𝑢 x = with 𝜇 1 ,…, 𝜇 t > 0, 𝜇 𝑘 𝑦 𝑘 𝜇 𝑘 = 1 and t smallest possible 𝑘=1 𝑘=1 Then x 1 ,…, x t are affinely independent (with t ≤ n +1 ). Suppose not. 𝑢 𝑢 There are 𝜈 1 ,…, 𝜈 t not all 0 such that and 𝜈 𝑘 𝑦 𝑘 = 0 𝜈 𝑘 = 0 𝑘=1 𝑘=1 Then there is at least one positive coefficient, say 𝜈 1

Caratheodory’s theorem 𝑢 𝑢 , 𝜈 𝑘 𝑦 𝑘 = 0 𝜈 𝑘 = 0 , 𝜈 1 > 0 𝑘=1 𝑘=1 𝑢 𝑢 Combining x = and α 𝜇 𝑘 𝑦 𝑘 𝜈 𝑘 𝑦 𝑘 = 0 for α ≥ 0 𝑘=1 𝑘=1 𝑢 x = ( 𝜇 𝑘 − α𝜈 𝑘 )𝑦 𝑘 𝑘=1 Increase α from 0 to α 0 until the first coefficient becomes 0, say the r-th . 𝜇 𝑘 − α𝜈 𝑘 ≥ 0 𝑘 = 1, … , 𝑢 and 𝜇 𝑠 − α𝜈 𝑠 = 0 𝑢 𝑢 𝑢 𝑢 ( 𝜇 𝑘 − α𝜈 𝑘 ) = 𝜇 𝑘 − α𝜈 𝑘 = 𝜇 𝑘 =1 𝑘=1 𝑘=1 𝑘=1 𝑘=1 T hen 𝑦 is obtained as a convex combination of t- 1 point in S, contrad. A similar result for conical hulls. Theorem. 2.5.2. ( Caratheodory’s theorem for conical hulls). Let ⊆ ℝ n . Then each x ∈ cone( S ) is the conical combination of m S • linearly independent points in S, with m ≤ n .

Caratheodory’s theorem for cones Any point in conv(S)  R n can be generated by (at most) n+ 1 points of S. The generators of a point x are not necessarily unique. The generators of different points may be different. conv(S) S x x y

Caratheodory’s theorem and LP Consider LP: max {c T x : x  P }, with P = { x  R n : Ax = b , x ≥ 0} A  R m , n , m ≤ n. Let a 1 , …, a n  R m be the columns of A 𝑜 , x 1 , …, x n  R + Ax can be written as 𝑦 𝑘 a 𝑘 𝑘=1 P ≠  if and only if b  cone({ a 1 , …, a n }) Caratheodory: b can be obtained conical combination of t ≤ m linearly independent a j ’s . Equivalently : there exists a non-negative x  R n with at least n-t components being 0 and Ax = b … … and the non -zeros of x correspond to linearly independent columns of A (basic fesible solution ) Fundamental result : if an LP is non-empty then it contains a basic feasible solution

Supporting Hyperplanes A hyperplane is a set H ⊂ ℝ n of the form H = { x ∈ ℝ n : a T x = α } for some nonzero vector a and a real number α . Let H - = { x ∈ ℝ n : a T x ≤ α } and H + = { x ∈ ℝ n : a T x ≥ α } be the two halfspaces identified by H . H is a convex set ( H = H - ∩ H + ). If S ⊂ ℝ n is contained in one of the two halfspaces H - and H + , and S ∩ H is non-empty, then H is a supporting hyperplane of S. H supports S at x for x ∈ S ∩ H . If S is convex, then S ∩ H is called exposed face of S, which is convex ( S and H are convex). S S