CSE 521: Design & subject to a finite number of linear equality - PDF document

Linear Programming  The process of minimizing a linear objective function CSE 521: Design & subject to a finite number of linear equality and Analysis of Algorithms I inequality constraints.  The word “programming” is historical and predates computer programming.  Example applications:  airline crew scheduling Linear Programming  manufacturing and production planning  telecommunications network design From slides by Paul Beame  “Few problems studied in computer science have greater application in the real world.” 1 2 Linear Programming An Example: The Diet Problem  Suggested Readings:  A student is trying to decide on lowest cost diet that provides sufficient amount of protein, with two choices:  Chapter 7 of text by Dasgupta, Papadimitriou, Vazirani (link on web page).  steak: 2 units of protein/pound, $3 /pound  peanut butter: 1 unit of protein/pound, $2 /pound  “Linear Programming”, by Howard Karloff  In proper diet, need 4 units protein/day.  First 34 pages on Simplex Algorithm available through Let x = # pounds peanut butter/day in the diet. Google books preview Let y = # pounds steak/day in the diet.  “Linear Programming”, by Vasek Chvatal Goal: minimize 2x + 3y (total cost) subject to constraints:  “Understanding and Using Linear Programming”, x + 2y ≥ 4 by Jiri Matousek and Bernd Gartner This is an LP- formulation x ≥ 0, y ≥ 0 of our problem 3 4 1 �

An Example: The Diet Problem Linear Program - Definition A linear program is a problem with n variables x 1 , Goal: minimize 2x + 3y (total cost) …, x n , that has: subject to constraints: A linear objective function, which must be x + 2y ≥ 4 1. x ≥ 0, y ≥ 0 minimized/maximized. Looks like: min ( max ) c 1 x 1 + c 2 x 2 + … + c n x n  This is an optimization problem. A set of m linear constraints. A constraint 2.  Any solution meeting the nutritional demands is called looks like: a feasible solution a i1 x 1 + a i2 x 2 + … + a in x n ≤ b i (or ≥ or = )  A feasible solution of minimum cost is called the optimal solution . Note: the values of the coefficients c i ,b i , a i, j are given in the problem input. 5 6 Visually… Optimal vector occurs at some x= peanut butter, y = steak corner of the feasible set! x=0 y Opt: x=0, y=2 feasible set feasible set y=0 x 2x+3y=15 Minimal price of x+2y=4 x+2y=4 one protein unit = 6/4=1.5 2x+3y=6 7 2x+3y=0 8 2 �

Optimal vector occurs at some Standard Form of a Linear Program. corner of the feasible set Maximize c 1 x 1 + c 2 x 2 +…+ c n x n y subject to Σ 1 ≤ j ≤ n a ij x j ≤ b j i = 1 .. m An Example with 6 x j ≥ 0 j = 1 .. n constraints. or Minimize b 1 y 1 + b 2 y 2 +…+ b m y m feasible set subject to Σ 1 ≤ i ≤ m a ij y i ≥ c j j = 1 ... n y i ≥ 0 i = 1 .. m x 9 10 Feasible Set The Feasible Set  Intersection of a set of half-spaces, called a  Each linear inequality divides n-dimensional space into two halfspaces, one where the polyhedron. inequality is satisfied, and one where it’s not.  If it’s bounded and nonempty, it’s a polytope.  Feasible Set : solutions to a family of linear There are 3 cases: inequalities.  Convex: for any 2 points in feasible set, the line  feasible set is empty. segment joining them is in feasible set.  cost function is unbounded on feasible set.  The linear cost functions, defines a family of  cost has a minimum (or maximum) on feasible parallel hyperplanes (lines in 2D, planes in set. 3D, etc.). Want to find one of minimum cost  must occur at corner of feasible set. First two cases very uncommon for real problems  Corner= can’t be expressed as convex in economics, science and engineering. combination of 2 or more points in feasible set. 11 12 3 �

LP formulation: another example Solving LPs  There are several algorithms that solve any linear Bob’s bakery sells bagel and muffins. program optimally.  The Simplex method (class of methods, usually To bake a dozen bagels Bob needs 5 cups of very good but worst-case exponential for known flour, 2 eggs, and 1 cup of sugar. methods)  The Ellipsoid method (polynomial-time) To bake a dozen muffins Bob needs 4 cups of  More flour, 4 eggs and 2 cups of sugar.  These algorithms can be implemented in various Bob can sell bagels at $10 /dozen and muffins at ways. $12 /dozen.  There are many existing software packages for LP. Bob has 50 cups of flour, 30 eggs and 20 cups of  It is convenient to use LP as a ``black box'' for sugar. solving various optimization problems. How many bagels and muffins should Bob bake in order to maximize his revenue? 13 14 LP formulation: Bob’s bakery Idea of the Simplex Method Avail. Bagels Muffins 5 4 50 The Toy Factory Problem (TFP): Flour 5 4 A = 2 4 30 Eggs 2 4 A toy factory produces dolls and cars. 20 Sugar 1 2 1 2 Danny, a new employee, is hired. He can produce 2 cars and 3 dolls a day. However, the packaging machine can 50 Revenue 10 12 only pack 4 items a day. The company’s profit from each c T = 10 12 b = 30 doll is $10 and from each car is $15. What should 20 Maximize 10x 1 +12x 2 Danny be asked to do? s.t. 5x 1 +4x 2 ≤ 50 Step 1: Describe the problem as an LP problem. Maximize c ⋅ x 2x 1 +4x 2 ≤ 30 Let x 1 ,x 2 denote the number of cars and dolls produced by s.t. Ax ≤ b Danny. x 1 +2x 2 ≤ 20 x ≥ 0 . x 1 ≥ 0, x 2 ≥ 0 15 16 4 �

The Toy Factory Problem The Toy Factory Problem Let x 1 ,x 2 denote the number of cars and dolls produced by x 2 Constant profit Danny. lines – They are x 2 Objective: x 1 =2 always parallel. x 1 =2 Max z = 15x 1 + 10x 2 We are looking x 2 =3 for the best one s.t x 1 ≤ 2 x 2 =3 z=15 that still x 2 ≤ 3 ‘touches’ the x 1 +x 2 ≤ 4 feasible region. Feasible x 1 +x 2 =4 x 1 ≥ 0 Feasible region x 1 +x 2 =4 region x 2 ≥ 0 x 1 x 1 z=30 z=40 17 18 Important Observations: Important Observations: 1. An optimum solution to the LP is always at a 2. If a corner point feasible solution has an objective corner point function value that is better than or equal to all its x 2 adjacent corner point feasible solutions then it is It might be that the optimal. objective line is parallel x 1 =2 3. There is a finite number of to a constraint. In this corner point feasible solutions. case there are many x 2 x 2 =3 optimum points, in x 1 =2 particular at the relevant The Simplex method: Travel corner points (consider x 2 =3 Feasible along the corner points till a x 1 +x 2 =4 z = 15x 1 + 15x 2 ). region local maximum. Feasible x 1 +x 2 =4 region x 1 x 1 z=50 19 z=50 20 5 �

The Simplex Method Example: The Toy Factory Problem Phase 1 (start-up): Find any corner point feasible Phase 1: start at (0,0) solution. In many standard LPs the origin can serve x 2 Objective value = Z(0,0)=0 as the start-up corner point. Iteration 1: Move to (2,0). x 1 =2 Phase 2 (iterate): Repeatedly move to a better Z(2,0)=30. An Improvement adjacent corner point feasible solution until no Iteration 2: Move to (2,2) x 2 =3 (1,3) further better adjacent corner point feasible solution Z(2,2)=50. An Improvement can be found. The final corner point defines the Iteration 3: Consider moving to (2,2) (1,3), Z(1,3)=45 < 50. optimum point. Feasible x 1 +x 2 =4 Conclude that (2,2) is region optimum! (0,0) (2,0) x 1 21 22 A Central Result of LP Theory: Primal: Maximize c T x subject to A x ≤ b , x ≥ 0 Duality Theorem Dual: Minimize b T y subject to A T y ≥ c , y ≥ 0  Every linear program has a dual  In the primal, c is cost function and b was in  If the original is a minimization, the dual is a the constraint. In the dual, reversed. maximization and vice versa  Inequality sign is changed and minimization  Solution of one leads to solution of other turns to maximization. Primal: Maximize c T x subject to Ax ≤ b , x ≥ 0 Dual: Minimize b T y subject to A T y ≥ c , y ≥ 0 Dual: Primal: maximize 2x + 3y If one has optimal solution so does the other, and minimize 4p +q + 2r s.t their values are the same. s.t x+2y ≤ 4, p+2q + r ≥ 2, 2x + 5y ≤ 1, 2p+5q -3r ≥ 3, x - 3y ≤ 2, x ≥ 0, y ≥ 0 p,q,r ≥ 0 23 24 6 �