CS672: Approximation Algorithms Spring 2020 Linear Programming - PowerPoint PPT Presentation

CS672: Approximation Algorithms Spring 2020 Linear Programming Review Instructor: Shaddin Dughmi

Outline Linear Programming Basics 1 Duality and Its Interpretations 2 3 Properties of Duals Weak and Strong Duality 4 5 Consequences of Duality Uses and Examples of Duality 6 Solvability of LP 7

Outline Linear Programming Basics 1 Duality and Its Interpretations 2 3 Properties of Duals Weak and Strong Duality 4 5 Consequences of Duality Uses and Examples of Duality 6 Solvability of LP 7

LP General Form minimize (or maximize) c ⊺ x for i ∈ C 1 . a ⊺ i x ≤ b i , subject to for i ∈ C 2 . a ⊺ i x ≥ b i , a ⊺ for i ∈ C 3 . i x = b i , Decision variables: x ∈ R n Parameters: c ∈ R n defines the linear objective function a i ∈ R n and b i ∈ R define the i ’th constraint. Linear Programming Basics 1/37

Standard Form maximize c ⊺ x subject to a ⊺ i x ≤ b i , for i = 1 , . . . , m. x j ≥ 0 , for j = 1 , . . . , n. Every LP can be transformed to this form minimizing c ⊺ x is equivalent to maximizing − c ⊺ x ≥ constraints can be flipped by multiplying by − 1 Each equality constraint can be replaced by two inequalities Uconstrained variable x j can be replaced by x + j − x − j , where both x + j and x − j are constrained to be nonnegative. Linear Programming Basics 2/37

Geometric View Linear Programming Basics 3/37

Geometric View Linear Programming Basics 3/37

A 2-D example maximize x 1 + x 2 subject to x 1 + 2 x 2 ≤ 2 2 x 1 + x 2 ≤ 2 x 1 , x 2 ≥ 0 Linear Programming Basics 4/37

Application: Optimal Production n products, m raw materials Every unit of product j uses a ij units of raw material i There are b i units of material i available Product j yields profit c j per unit Facility wants to maximize profit subject to available raw materials maximize c ⊺ x subject to a ⊺ i x ≤ b i , for i = 1 , . . . , m. x j ≥ 0 , for j = 1 , . . . , n. Linear Programming Basics 5/37

Terminology Hyperplane: The region defined by a linear equality Halfspace: The region defined by a linear inequality a ⊺ i x ≤ b i . Polyhedron: The intersection of a set of linear inequalities Feasible region of an LP is a polyhedron Polytope: Bounded polyhedron Equivalently: convex hull of a finite set of points Vertex: A point x is a vertex of polyhedron P if � ∃ y � = 0 with x + y ∈ P and x − y ∈ P Face of P : The intersection with P of a hyperplane H disjoint from the interior of P Linear Programming Basics 6/37

Basic Facts about LPs and Polyhedrons Fact Feasible regions of LPs (i.e. polyhedrons) are convex Linear Programming Basics 7/37

Basic Facts about LPs and Polyhedrons Fact Feasible regions of LPs (i.e. polyhedrons) are convex Fact Set of optimal solutions of an LP is convex In fact, a face of the polyhedron intersection of P with hyperplane c ⊺ x = OPT Linear Programming Basics 7/37

Basic Facts about LPs and Polyhedrons Fact Feasible regions of LPs (i.e. polyhedrons) are convex Fact Set of optimal solutions of an LP is convex In fact, a face of the polyhedron intersection of P with hyperplane c ⊺ x = OPT Fact A feasible point x is a vertex if and only if n linearly independent constraints are tight (i.e., satisfied with equality) at x . Linear Programming Basics 7/37

Basic Facts about LPs and Polyhedrons Fact An LP either has an optimal solution, or is unbounded or infeasible Linear Programming Basics 8/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Linear Programming Basics 9/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Proof Assume not, and take a non-vertex optimal solution x with the maximum number of tight constraints Linear Programming Basics 9/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Proof Assume not, and take a non-vertex optimal solution x with the maximum number of tight constraints There is y � = 0 s.t. x ± y are feasible Linear Programming Basics 9/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Proof Assume not, and take a non-vertex optimal solution x with the maximum number of tight constraints There is y � = 0 s.t. x ± y are feasible y is perpendicular to the objective function and the tight constraints at x . i.e. c ⊺ y = 0 , and a ⊺ i y = 0 whenever the i ’th constraint is tight for x . Linear Programming Basics 9/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Proof Assume not, and take a non-vertex optimal solution x with the maximum number of tight constraints There is y � = 0 s.t. x ± y are feasible y is perpendicular to the objective function and the tight constraints at x . i.e. c ⊺ y = 0 , and a ⊺ i y = 0 whenever the i ’th constraint is tight for x . Can choose y s.t. y j < 0 for some j Linear Programming Basics 9/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Proof Assume not, and take a non-vertex optimal solution x with the maximum number of tight constraints There is y � = 0 s.t. x ± y are feasible y is perpendicular to the objective function and the tight constraints at x . i.e. c ⊺ y = 0 , and a ⊺ i y = 0 whenever the i ’th constraint is tight for x . Can choose y s.t. y j < 0 for some j Let α be the largest constant such that x + αy is feasible Such an α exists Linear Programming Basics 9/37

Fundamental Theorem of LP If an LP in standard form has an optimal solution, then it has a vertex optimal solution. Proof Assume not, and take a non-vertex optimal solution x with the maximum number of tight constraints There is y � = 0 s.t. x ± y are feasible y is perpendicular to the objective function and the tight constraints at x . i.e. c ⊺ y = 0 , and a ⊺ i y = 0 whenever the i ’th constraint is tight for x . Can choose y s.t. y j < 0 for some j Let α be the largest constant such that x + αy is feasible Such an α exists An additional constraint becomes tight at x + αy , a contradiction. Linear Programming Basics 9/37

Counting non-zero Variables Corollary If an LP in standard form has an optimal solution, then there is an optimal solution with at most m non-zero variables. maximize c ⊺ x a ⊺ i x ≤ b i , subject to for i = 1 , . . . , m. x j ≥ 0 , for j = 1 , . . . , n. e.g. for optimal production with n products and m raw materials, there is an optimal plan with at most m products. Linear Programming Basics 10/37

Outline Linear Programming Basics 1 Duality and Its Interpretations 2 3 Properties of Duals Weak and Strong Duality 4 5 Consequences of Duality Uses and Examples of Duality 6 Solvability of LP 7

Linear Programming Duality Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y = c y � 0 A ∈ R m × n , c ∈ R n , b ∈ R m y i is the dual variable corresponding to primal constraint A i x ≤ b i A T j y = c j is the dual constraint corresponding to primal variable x j Duality and Its Interpretations 11/37

Linear Programming Duality: Standard Form, and Visualization Primal LP Dual LP maximize c ⊺ x minimize y ⊺ b subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Duality and Its Interpretations 12/37

Linear Programming Duality: Standard Form, and Visualization Primal LP Dual LP maximize c ⊺ x minimize y ⊺ b subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 x 1 x 2 x 3 x 4 y 1 a 11 a 12 a 13 a 14 b 1 y 2 a 21 a 22 a 23 a 24 b 2 y 3 a 31 a 32 a 33 a 34 b 3 c 1 c 2 c 3 c 4 Duality and Its Interpretations 12/37

Linear Programming Duality: Standard Form, and Visualization Primal LP Dual LP maximize c ⊺ x minimize y ⊺ b subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 x 1 x 2 x 3 x 4 y 1 a 11 a 12 a 13 a 14 b 1 y 2 a 21 a 22 a 23 a 24 b 2 y 3 a 31 a 32 a 33 a 34 b 3 c 1 c 2 c 3 c 4 y i is the dual variable corresponding to primal constraint A i x ≤ b i A T j y ≥ c j is the dual constraint corresponding to primal variable x j Duality and Its Interpretations 12/37

Interpretation 1: Economic Interpretation Recall the Optimal Production problem from last lecture n products, m raw materials Every unit of product j uses a ij units of raw material i There are b i units of material i available Product j yields profit c j per unit Facility wants to maximize profit subject to available raw materials Duality and Its Interpretations 13/37

Interpretation 1: Economic Interpretation Primal LP � n max j =1 c j x j � n j =1 a ij x j ≤ b i , for i ∈ [ m ] . s.t. x j ≥ 0 , for j ∈ [ n ] . Duality and Its Interpretations 13/37

Interpretation 1: Economic Interpretation Primal LP Dual LP � n � m max j =1 c j x j min i =1 b i y i � n � m j =1 a ij x j ≤ b i , for i ∈ [ m ] . i =1 a ij y i ≥ c j , for j ∈ [ n ] . s.t. s.t. y i ≥ 0 , for i ∈ [ m ] . x j ≥ 0 , for j ∈ [ n ] . Duality and Its Interpretations 13/37