Chapter 1 Linear Programming Paragraph 3 Mathematical Foundations - PowerPoint PPT Presentation

Chapter 1 Linear Programming Paragraph 3 Mathematical Foundations - Geometry of the Solution Set

What we did so far • By combining ideas of a specialized algorithm with a geometrical view on the problem, we developed an algorithm idea: Find a feasible corner (somehow). Check neighboring corners and see if one is better. Move over to the next corner until no better neighboring solution exists. • We have an intuitive understanding how our geometrical view generalizes to more dimensions: • Corners correspond to solutions of equation systems. • Inequalities partition restrict the solution space to halfspaces. CS 195 - Intro to CO 2

Analysis • Algorithm Idea Find a feasible corner (somehow). Check neighboring corners and see if one is better. Move over to the next corner until no better neighboring solution exists. • Open Questions – Can we find a mathematical formalization of linear optimization problems for which we can define what “corner” and “neighboring corner” means? – Can we prove optimality? – How do we find a feasible starting solution? CS 195 - Intro to CO 3

What is a corner? • A corner in our geometrical view is defined by the intersection of two lines. And a line is defined by an equation � a corner is a solution to an equation-system! • What if there are more than two variables? How does an inequality look like then? – Given x,y,z, how does x ¥ 1 look like? – Given x,y,z, how does x+y+z = 1 look like? y y z x x z CS 195 - Intro to CO 4

Equation Systems • So an equation in an n-dimensional space defines an n-1-dimensional hyperplane! – n = 2: equations define lines – n = 3: equations define planes • Every inequality divides the space in two halfspaces! • A corner in an n-dimensional space is defined by the intersection of n hyperplanes. Therefore, a corner defines a solution to an equation system and vice versa. CS 195 - Intro to CO 5

Mathematical Foundations • We shall now define and study formally – what corners are, – how they correspond to basic solutions of equation systems, – what neighboring corners are, and – how optimal solutions to LPs are characterized. CS 195 - Intro to CO 6

Convex Sets • Definition – Assume that {a 1 ,…,a m } ⊆ Ñ n and α 1 ,…, α m œ Ñ ¥ 0 . – Ê i α i a i is called a non-negative linear combination. – In case that Ê i α i = 1, we call Ê i α i a i a convex combination . It is called true convex combination iff α 1 ,…, α m œ Ñ >0 . – We define κ (a 1 ,…,a m ) as the set of all convex combinations of a 1 ,…,a m . – For a,b œ Ñ n , κ (a,b) is called the line between a and b. – A set K ⊆ Ñ n is called convex, iff for all a,b œ K: κ (a,b) ⊆ K. CS 195 - Intro to CO 7

Convex Sets b κ (a,b) a Not Convex Convex CS 195 - Intro to CO 8

Convex Sets • Remark – If K ⊆ Ñ n is convex and a 1 ,…,a m œ K, then κ (a 1 ,…,a m ) ⊆ K. – The intersection of convex sets is convex. • Proof: CS 195 - Intro to CO 9

Convex Sets • Examples – Let a œ Ñ n , a ≠ 0, and α œ Ñ . H = {x œ Ñ n | a T x = α } is called a hyperplane and is convex. – Let A œ Ñ m x n and b œ Ñ m . V = {x œ Ñ n | Ax = b} is called affine vector space and is convex. – Let a œ Ñ n , a ≠ 0, and α œ Ñ . H ¥ = {x œ Ñ n | a T x ¥ α } is called halfspace and is convex. – P = {x œ Ñ n | Ax = b and x ¥ 0} is convex. – D = {x œ Ñ n | ||x|| 2 ≤ 1} is convex. CS 195 - Intro to CO 10

Convex Sets • Definition – For M ⊆ Ñ n , we define κ (M) := U k œ ô , a1,…,ak œ M κ (a 1 ,…,a k ). – κ (M) is called the convex hull of M. • Remark – It holds that κ (M) equals the intersection of all convex sets that contain M. Thus, κ (M) is the smallest convex set that contains M. CS 195 - Intro to CO 11

Extreme Points • Definition – A point x œ K, K ⊆ Ñ n convex, is called extreme point, iff it cannot be represented as a true convex combination of two points in K. We denote the set of all extreme points of K with ε (K). • Remark: Given K ⊆ Ñ n convex, the following statements are equivalent: – x 0 is an extreme point of K. – For all a,b œ K with x 0 œ κ (a,b) it is x 0 =a or x 0 =b. – For all y œ Ñ n , y ≠ 0 it is x 0 + y ∉ K or x 0 - y ∉ K. – K\{x 0 } is convex. CS 195 - Intro to CO 12

Extreme Points • Examples – ε ({x œ Ñ n | x ¥ 0}) = {0} – ε (H ¥ ) = ∅ {x œ Ñ n | ||x|| 2 = 1} – ε (D) = – ε ({ x | ||x|| 2 < 1}) = ∅ {1} – ε ({1}) = CS 195 - Intro to CO 13

Convex Polyeders • Definition – A convex polyeder (convex polyhedron) P is defined as the finite intersection of halfspaces, i.e. P = ∩ H ¥ = {x | Ax ¥ b}. – A convex polyeder P ≠ ∅ that is bounded is called (convex) polytope. – A hyperplane H is called supporting plane of P iff H ∩ P ≠ ∅ and P ⊆ H ¥ . – If H ∩ P = {x 0 }, then x 0 is called a corner of P. – If H ∩ P = κ (a,b) for some a,b œ P, then κ (a,b) is called an edge of P. CS 195 - Intro to CO 14

Theorem • Let P denote a convex polyeder. – Every corner of P is an extreme point. – P is the convex hull of its corners. • Proof: CS 195 - Intro to CO 15

Basic Solutions and Corners X 2 27 26 24 0 X 1 26 28 30 32 0 CS 195 - Intro to CO 16

Basic Solutions • Definition – Given A œ Ñ m x n with rank(A) = m ≤ n, b œ Ñ m , let B : ô m � ô n , N : ô n-m � ô n injective such that B( ô m ) U N( ô n-m ) = ô n and A B = (a B(1) ,…,a B(m) ) with rank(A B ) = rank(A) = m. – When setting x B := A B -1 b and x N := 0, we call x T := (x B T ,x N T ) basic (feasible) solution of Ax = b. x is called feasible, iff x ¥ 0. CS 195 - Intro to CO 17

Basic Solutions • Remark ⎛ ⎞ x ⎜ ⎟ B − 1 = = + = + = ( , ) 0 Ax A A A x A x A A b A b ⎜ ⎟ B N B B N N B B N ⎝ ⎠ x N • Theorem – Let A œ Ñ m x n with rank(A) = m ≤ n and b œ Ñ m . For x 0 œ P := {x œ Ñ n | Ax=b and x ¥ 0} it is equivalent to say: • x 0 is extreme point of P. • {a j | x 0 j > 0 } is linear independent. • x 0 is basic feasible solution • x 0 is a corner of P. CS 195 - Intro to CO 18

Basic Solutions • Corollary – 0 œ P ⇒ 0 œ ε (P) – Every corner has at most m entries that differ from 0! ⎛ ⎞ n ⎜ ⎟ – S has at most corners! ⎜ ⎟ ⎝ ⎠ m • Definition – A corner is called degenerated iff | { j | x j > 0} | < m. • Remark – If x 0 is not degenerated, the corresponding basis is uniquely defined! CS 195 - Intro to CO 19

Basic Solutions - Degeneracy X 2 X 1 + 2 X 2 ≤ 80 27 X 1 + X 2 ≤ 55 X 1 ≤ 35 26 X 2 ≤ 30 X 2 ≤ 27 24 2 X 1 + X 2 ≤ 85 22 X 1 ≤ 32 X 1 32 24 26 28 30 ¥ 0 X 1 , X 2 CS 195 - Intro to CO 20

Optimal Solutions • Definition – For P ≠ ∅ we define C(P) := { y œ Ñ n | ∀ x œ P, λ > 0 : x + λ y œ P}, and we say that C(P) is the set of directions of P. • Remark – C(P) = { y œ Ñ n | Ay = 0 and y ¥ 0} • Theorem – P = κ ( ε (P)) + C(P) CS 195 - Intro to CO 21

Optimal Solutions P κ ( ε (P)) C(P) ε (P) CS 195 - Intro to CO 22

Optimal Solutions • Corollary – P ≠ ∅ ⇒ P has corners! – If P contains an optimal solution, then there exists a corner with optimal objective value! – If P ≠ ∅ and P has no optimal solution, then there exists y œ C(P) such that c T y < 0. – If P ≠ ∅ and P is bounded, then P = κ ( ε (P)). CS 195 - Intro to CO 23

Optimal Solutions • Remark – The previous corollary yields an algorithm: Determine all basic solutions, eliminate all that are infeasible, and pick the one with the best objective function value. – What is the runtime of that algorithm? CS 195 - Intro to CO 24

Basis Changes • Given x 0T = (x 0 T ,x 0 T ) a basic feasible solution (i.e. B N N = 0, and x 0 ¥ 0) and x œ Ñ n such that x 0 B = A B -1 b, x 0 Ax = b, assume that for y := x-x 0 œ Ñ n it holds that Ay = 0. • It holds y N = x N . Because of = = + 0 Ay A y A y B B N N we have that ⎛ − ⎞ − 1 A A x ⎜ ⎟ ⇒ − 1 B N N = − = y A A x y ⎜ ⎟ B B N N ⎝ ⎠ x N CS 195 - Intro to CO 25

Basis Changes • Assume that we set specifically x N = e k œ Ñ n-m , t := N(k), and A := A B-1 A. • Then, we have that y B = -a t := - A B-1 a t . And therefore, y T = (-a t T (B) , e k T (N) ). • For λ œ Ñ and x λ := x 0 + λ y, we thus have Ax λ = b. ⎧ ⎫ 0 ⎪ ⎪ ⇒ x λ is feasible. x • ( ) B j ≤ λ ≤ ⎨ < ⎬ 0 min | 0 y ( ) ⎪ B j ⎪ − y ⎩ ⎭ B ( j ) CS 195 - Intro to CO 26

Basis Changes • Theorem – If there exists y B(j) < 0, we choose λ as large as possible ( λ < ∞ ). Then, x 1 :=x λ is a basic feasible solution. The corresponding basis is given by B*(i) := B(i) if i ≠ r, and B*(r) := t, whereby r such that λ = x 0 B(r) / -y B(r) = min {x 0 B(i) / -y B(j) | y B(j) < 0 }. – If y B ¥ 0, then y (defined as before) is greater or equal 0, and thus: x λ = x + λ y is feasible for all λ ¥ 0. • Definition – A solution x 1 obtained by an exchange as discussed above with 0 < λ < ∞ is called a neighboring corner of x 0 . CS 195 - Intro to CO 27