Solving Linear and Integer Programs Robert E. Bixby ILOG, Inc. - PDF document

15 September 2003 Solving Linear and Integer Programs Robert E. Bixby ILOG, Inc. and Rice University Ed Rothberg ILOG, Inc. Outline � Linear Programming: Bob Bixby � Example and introduction to basic LP, including duality � Primal and dual simplex algorithms � Computational progress in linear programming � Implementing the dual simplex algorithm � Mixed-Integer Programming: Ed Rothberg 2 INFORMS Practice 2002 1

15 September 2003 An Example 3 Diet Problem * Bob wants to plan a nutritious diet, but he is on a limited budget, so he wants to spend as little money as possible. His nutritional requirements are as follows: 1. 2000 kcal 2. 55 g protein 3. 800 mg calcium * From Linear Programming, by Va ŝ ek Chvátal 4 INFORMS Practice 2002 2

15 September 2003 Diet Problem Nutritional values Bob is considering the following foods: Food Serving Size Energy (kcal) Protein (g) Calcium (mg) Price per serving Oatmeal 28 g 110 4 2 $0.30 Chicken 100 g 205 32 12 $2.40 Eggs 2 large 160 13 54 $1.30 Whole milk 237 cc 160 8 285 $0.90 Cherry pie 170 g 420 4 22 $0.20 Pork and beans 260 g 260 14 80 $1.90 5 Diet Problem Variables We can represent the number of servings of each type of food in the diet by the variables: x 1 servings of oatmeal x 2 servings of chicken x 3 servings of eggs x 4 servings of milk x 5 servings of cherry pie x 6 servings of pork and beans 6 INFORMS Practice 2002 3

15 September 2003 Diet Problem Nutritional values Bob is considering the following foods: Food Serving Size Energy (kcal) Protein (g) Calcium (mg) Price per serving x 1 Oatmeal 28 g 110 4 2 $0.30 x 2 Chicken 100 g 205 32 12 $2.40 x 3 Eggs 2 large 160 13 54 $1.30 x 4 Whole milk 237 cc 160 8 285 $0.90 x 5 Cherry pie 170 g 420 4 22 $2.00 x 6 Pork and beans 260 g 260 14 80 $1.90 KCAL constraint: 110 x 1 + 205 x 2 + 160 x 3 + 160 x 4 + 420 x 5 + 260 x 6 ≥ 2000 (110 x 1 = kcals in oatmeal) 7 Diet Problem LP formulation Minimize Cost 0.3 x 1 + 2.40 x 2 + 1.30 x 3 + 0.90 x 4 + 2.0 x 5 + 1.9 x 6 subject to: Nutritional requirements 110 x 1 + 205 x 2 + 160 x 3 + 160 x 4 + 420 x 5 + 260 x 6 ≥ 2000 4 x 1 + 32 x 2 + 13 x 3 + 8 x 4 + 4 x 5 + 14 x 6 ≥ 55 2 x 1 + 12 x 2 + 54 x 3 + 285 x 4 + 22 x 5 + 80 x 6 ≥ 800 Bounds ≥ x , x , x , x , x , x 0 1 2 3 4 5 6 8 INFORMS Practice 2002 4

15 September 2003 Diet Problem Solution When we solve the preceding LP (using CPLEX, of course) we get a solution value of $6.71, which is achieved with the following menu: 14.24 servings of oatmeal 0 servings of chicken 0 servings of eggs 2.71 servings of milk 0 servings of cherry pie 0 servings of pork and beans 9 Some Basic Theory 10 INFORMS Practice 2002 5

15 September 2003 Linear Program – Definition A linear program ( LP ) in standard form is an optimization problem of the form Minimize c T x (P) Subject to Ax = b x ≥ 0 Where c ∈ R n , b ∈ R m , A ∈ R m×n , and x is a vector of n variables . c T x is known as the objective function , Ax=b as the constraints , and x ≥ 0 as the nonnegativity conditions. b is called the right-hand side . 11 Dual Linear Program – Definition The dual ( or adjoint) linear program corresponding to (P) is the optimization problem Maximize b T π Subject to A T π ≤ c (D) π free In this context, (P) is referred to as the primal linear program . Primal Minimize c T x Subject to Ax = b x ≥ 0 12 INFORMS Practice 2002 6

15 September 2003 Weak Duality Theorem (von Neumann 1947) Let x be feasible for (P) and π feasible for (D). Then b T π ≤ c T x Maximize Minimize If b T π = c T x , then x is optimal for (P) and π is optimal for (D); moreover, if either (P) or (D) is unbounded , then the other problem is infeasible . Proof: π T b π T Ax = ≤ c T x � π T A ≤ c T & x ≥ 0 Ax = b 13 Solving Linear Programs � Three types of algorithms are available � Primal simplex algorithms (Dantzig 1947) � Dual simplex algorithms (Lemke 1954) � Developed in context of game theory � Primal-dual log barrier algorithms � Interior-point algorithms (Karmarkar 1989) � Reference: Primal-Dual Interior Point Methods, S. Wright, 1997, SIAM Primary focus: Dual simplex algorithms 14 INFORMS Practice 2002 7

15 September 2003 Basic Solutions – Definition Let B be an ordered set of m distinct indices (B 1 ,…,B m ) taken from {1,…,n}. B is called a basis for (P) if A B is nonsingular. The variables x B are known as the basic variables and the variables x N as the non-basic variables, where N = {1,…,n}\B . The corresponding basic solution X ∈ R n is given by X N =0 and X B =A B-1 b . B is called ( primal ) feasible if X B ≥ 0. Note: AX = b ⇒ A B X B + A N X N = b ⇒ A B X B = b ⇒ X B = A B -1 b 15 Primal Simplex Algorithm (Dantzig, 1947) Input: A feasible basis B and vectors X B = A B -1 b and D N = c N – A N T B -T c B . � Step 1: (Pricing) If D N ≥ 0 , stop, B is optimal; else let j = argmin{D k : k ∈ N}. � Step 2: (FTRAN) Solve A B y=A j . � Step 3: (Ratio test) If y ≤ 0 , stop, (P) is unbounded; else, let i = argmin{X Bk /y k : y k > 0}. � Step 4: (BTRAN) Solve A B T z = e i . � Step 5: (Update) Compute α N =-A N T z . Let B i =j . Update X B (using y ) and D N (using α N ) Note: x j is called the entering variable and x Bi the leaving variable. The D N values are known as reduced costs – like partial derivatives of the objective function relative to the nonbasic variables. 16 INFORMS Practice 2002 8

15 September 2003 Dual Simple Algorithm – Setup Simplex algorithms apply to problems with constraints in equality form. We convert (D) to this form by adding the dual slacks d : Maximize b T π Subject to A T π + d = c ⇔ A T π ≤ c π free, d ≥ 0 17 Dual Simple Algorithm – Setup Simplex algorithms apply to problems with constraints in equality form. We convert (D) to this form by adding the dual slacks d : Maximize b T π π c B A B T I B 0 Subject to A T π + d = c d B = A N T 0 I N c N π free, d ≥ 0 d N Given a basis B, the corresponding dual basic solution Π ,D is determined as follows: D B =0 ⇒ Π = A B-T c B ⇒ D N =c N – A NT Π . B is dual feasible if D N ≥ 0. 18 INFORMS Practice 2002 9

15 September 2003 An Important Fact If X and Π ,D are the respective primal and dual basic solutions determined by a basis B , then Π T b = c T X . Hence, by weak duality, if B is both primal and dual feasible, then X is optimal for (P) and Π is optimal for (D). Proof: c T X = c BT X B (since X N =0 ) = Π T A B X B (since Π = A B-T c B ) = Π T b (since A B X B =b ) � 19 Dual Simplex Algorithm (Lemke, 1954) Input: A dual feasible basis B and vectors X B = A B -1 b and D N = c N – A N T B -T c B . � Step 1: (Pricing) If X B ≥ 0 , stop, B is optimal; else let i = argmin{X Bk : k ∈ {1,…,m}}. � Step 2: (BTRAN) Solve B T z = e i . Compute α N =-A N T z . � Step 3: (Ratio test) If α N ≤ 0 , stop, (D) is unbounded; else, let j = argmin{D k / α k : α k > 0}. � Step 4: (FTRAN) Solve A B y = A j . � Step 5: (Update) Set B i =j . Update X B (using y ) and D N (using α N ) Note: d Bi is the entering variable and d j is the leaving variable. (Expressed in terms of the primal: x Bi is the leaving variable and x j is the entering variable) 20 INFORMS Practice 2002 10

15 September 2003 Simplex Algorithms Input: A primal feasible basis B Input: A dual feasible basis B and and vectors vectors X B =A B -1 b & D N =c N – A N T A B -T c B . X B =A B -1 b & D N =c N – A N T A B -T c B . � Step 1: (Pricing) If D N ≥ 0 , stop, � Step 1: (Pricing) If X B ≥ 0 , stop, B is optimal; else, let B is optimal; else, let j = argmin{D k : k ∈ N}. i = argmin{X Bk : k ∈ {1,…,m}}. T z = � Step 2: (FTRAN) Solve A B y=A j . � Step 2: (BTRAN) Solve A B e i . Compute α N =-A N T z . � Step 3: (Ratio test) If y ≤ 0 , stop, � Step 3: (Ratio test) If α N ≤ 0 , (P) is unbounded; else, let stop, (D) is unbounded; else, let i = argmin{X Bk /y k : y k > 0}. j = argmin{D k / α k : α k > 0}. T z = � Step 4: (BTRAN) Solve A B e i . � Step 4: (FTRAN) Solve A B y = A j . � Step 5: (Update) Compute α N = T z . Let B i =j . Update X B -A N � Step 5: (Update) Set B i =j . (using y ) and D N (using α N ) Update X B (using y ) and D N (using α N ) 21 Correctness: Dual Simplex Algorithm � Termination criteria � Optimality (DONE – by “An Important Fact” !!!) � Unboundedness � Other issues � Finding starting dual feasible basis, or showing that no feasible solution exists � Input conditions are preserved (i.e., that B is still a feasible basis) � Finiteness 22 INFORMS Practice 2002 11