The Simplex Method Marco Chiarandini Department of Mathematics - PowerPoint PPT Presentation

DM559/DM545 Linear and Integer Programming Lecture 3 The Simplex Method Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Simplex Method Outline 1. Simplex Method Standard Form Basic Feasible Solutions Algorithm Tableaux and Dictionaries 2

Simplex Method Outline 1. Simplex Method Standard Form Basic Feasible Solutions Algorithm Tableaux and Dictionaries 3

Simplex Method A Numerical Example n max � c j x j max 6 x 1 + 8 x 2 j = 1 5 x 1 + 10 x 2 ≤ 60 n � a ij x j ≤ b i , i = 1 , . . . , m 4 x 1 + 4 x 2 ≤ 40 j = 1 x 1 , x 2 ≥ 0 x j ≥ 0 , j = 1 , . . . , n � 6 8 � � x 1 � max max c T x x 2 A x ≤ b x ≥ 0 � 5 10 � � x 1 � � 60 � ≤ 4 4 x 2 40 x ∈ R n , c ∈ R n , A ∈ R m × n , b ∈ R m x 1 , x 2 ≥ 0 4

Simplex Method Outline 1. Simplex Method Standard Form Basic Feasible Solutions Algorithm Tableaux and Dictionaries 5

Simplex Method Standard Form Every LP problem can be converted in the form: max c T x • if equations, then put two constraints, A x ≤ b a x ≤ b and a x ≥ b x ∈ R n • if a x ≥ b then − a x ≤ − b c ∈ R n , A ∈ R m × n , b ∈ R m • if min c T x then max ( − c T x ) and then be put in equational standard form max c T x 1. “ = ” constraints A x = b x ≥ 0 2. x ≥ 0 nonnegativity constraints 3. ( b ≥ 0) x ∈ R n , c ∈ R n , A ∈ R m × n , b ∈ R m 4. max 6

Simplex Method Transformation to Std Form Every LP problem can be transformed in eq. std. form 1. introduce slack variables (or surplus) 5 x 1 + 10 x 2 + x 3 = 60 4 x 1 + 4 x 2 + x 4 = 40 x 1 = x ′ 1 − x ′′ 1 2. if x 1 � 0 then x ′ 1 ≥ 0 x ′′ 1 ≥ 0 3. ( b ≥ 0) 4. min c T x ≡ max ( − c T x ) LP in m × n converted into LP with at most ( m + 2 n ) variables and m equations ( n # original variables, m # constraints) 7

Simplex Method Geometry of LP in Eq. Std. Form From linear algebra: max { c T x | A x = b , x ≥ 0 } • the set of solutions of A x = b is an affine space (hyperplane not passing through the origin). • x ≥ 0 nonegative orthant (octant in R 3 ) In R 3 : 8

Simplex Method • A x = b is a system of equations that we can solve by Gaussian elimination • Elementary row operations of � � A | b do not affect set of feasible solutions • multiplying all entries in some row of � � A | b by a nonzero real number λ • replacing the i th row of � � A | b by the sum of the i th row and j th row for some i � = j • Let n ′ be the number of vars in eq. std. form. we assume n ′ ≥ m and rank ( � A | b � ) = rank ( A ) = m ie, rows of A are linearly independent otherwise, remove linear dependent rows 9

Simplex Method Outline 1. Simplex Method Standard Form Basic Feasible Solutions Algorithm Tableaux and Dictionaries 10

Simplex Method Basic Feasible Solutions Basic feasible solutions are the vertices of the feasible region: More formally: Let B = { 1 . . . m } , N = { m + 1 . . . n + m = n ′ } be subsets partitioning the columns of A : A B be made of columns of A indexed by B : Definition x ∈ R n is a basic feasible solution of the linear program max { c T x | A x = b , x ≥ 0 } for an index set B if: • x j = 0 ∀ j �∈ B • the square matrix A B is nonsingular, ie, all columns indexed by B are lin. indep. • x B = A − 1 B b is nonnegative, ie, x B ≥ 0 (feasibility) 11

Simplex Method We call x j for j ∈ B basic variables and remaining variables nonbasic variables. Theorem A basic feasible solution is uniquely determined by the set B . Proof: A x = A B x B + A N x N = b x B + A − 1 B A N x N = A − 1 B b x B = A − 1 A B is nonsingular hence one solution B b Note: we call B a (feasible) basis 12

Simplex Method Extreme points and basic feasible solutions are geometric and algebraic manifestations of the same concept: Theorem Let P be a (convex) polyhedron from LP in eq. std. form. For a point v ∈ P the following are equivalent: (i) v is an extreme point (vertex) of P (ii) v is a basic feasible solution of LP Proof: by recognizing that vertices of P are linear independent and such are the columns in A B Theorem Let LP = max { c T x | A x = b , x ≥ 0 } be feasible and bounded, then the optimal solution is a basic feasible solution. Proof. consequence of previous theorem and fundamental theorem of linear programming 13

Simplex Method Note, a similar theorem is valid for arbitrary linear programs (not in eq. form) Definition A basic feasible solution of a linear program with n variables is a feasible solution for which some n linearly independent constraints hold with equality. However, an optimal solution does not need to be basic: max x 1 + x 2 subject to x 1 + x 2 ≤ 1 14

Simplex Method • Idea for solution method: • examine all basic solutions. � m + n • There are finitely many: � . m � 2 m � • However, if n = m then ≈ 4 m . m 15

Simplex Method Outline 1. Simplex Method Standard Form Basic Feasible Solutions Algorithm Tableaux and Dictionaries 16

Simplex Method Simplex Method � 6 8 � � x 1 � max z = x 2 Canonical eq. std. form: one decision variable is isolated in each constraint   x 1 with coefficient 1 and does not appear � � � � 5 10 1 0 x 2 60    = in the other constraints nor in the   4 4 0 1 x 3 40  obj. func. and b terms are positive x 4 x 1 , x 2 , x 3 , x 4 ≥ 0 It gives immediately a basic feasible solution: x 1 = 0 , x 2 = 0 , x 3 = 60 , x 4 = 40 Is it optimal? Look at signs in z � if positive then an increase would improve. 17

Let’s try to increase a promising variable, ie, x 1 , one with positive coefficient in z x 3 5 x 1 + x 3 = 60 x 1 = 60 5 − x 3 5 x 3 = 60 − 5 x 1 ≥ 0 If x 1 > 12 then x 3 < 0 x 1 5 x 1 + x 3 = 60 4 x 1 + x 4 = 40 x 4 x 1 = 40 4 − x 4 4 x 4 = 40 − 4 x 1 ≥ 0 If x 1 > 10 then x 4 < 0 x 1 4 x 1 + x 4 = 40 we can take the minimum of the two � x 1 increased to 10 x 4 exits the basis and x 1 enters

Simplex Method Simplex Tableau First simplex tableau: we want to reach this new tableau x 1 x 2 x 3 x 4 − z b x 1 x 2 x 3 x 4 − z b x 3 5 10 1 0 0 60 x 3 0 ? 1 ? 0 ? x 4 4 4 0 1 0 40 x 1 1 ? 0 ? 0 ? 6 8 0 0 1 0 0 ? 0 ? 1 ? Pivot operation: 1. Choose pivot: column: one s with positive coefficient in obj. func. row: ratio between coefficient b and pivot column: choose the one with smallest ratio: � b i � θ increase value θ = min : a is > 0 , of entering var. a is i 2. elementary row operations to update the tableau 19

• x 4 leaves the basis, x 1 enters the basis • Divide pivot row by pivot • Send to zero the coefficient in the pivot column of the first row • Send to zero the coefficient of the pivot column in the third (cost) row | | x1 | x2 | x3 | x4 | -z | b | |---------------+----+----+----+------+----+-----| | I’=I-5II’ | 0 | 5 | 1 | -5/4 | 0 | 10 | | II’=II/4 | 1 | 1 | 0 | 1/4 | 0 | 10 | |---------------+----+----+----+------+----+-----| | III’=III-6II’ | 0 | 2 | 0 | -6/4 | 1 | -60 | From the last row we read: 2 x 2 − 3 / 2 x 4 − z = − 60, that is: z = 60 + 2 x 2 − 3 / 2 x 4 . Since x 2 and x 4 are nonbasic we have z = 60 and x 1 = 10 , x 2 = 0 , x 3 = 10 , x 4 = 0. • Done? No! Let x 2 enter the basis | | x1 | x2 | x3 | x4 | -z | b | |--------------+----+----+------+------+----+-----| | I’=I/5 | 0 | 1 | 1/5 | -1/4 | 0 | 2 | | II’=II-I’ | 1 | 0 | -1/5 | 1/2 | 0 | 8 | |--------------+----+----+------+------+----+-----| | III’=III-2I’ | 0 | 0 | -2/5 | -1 | 1 | -64 |

Simplex Method Definition (Reduced costs) We call reduced costs the coefficients in the objective function of the nonbasic variables, ¯ c N Proposition (Optimality Condition) The basic feasible solution is optimal when the reduced costs in the corresponding simplex tableau are nonpositive, ie, such that: c N ≤ 0 ¯ Proof: Let z 0 be the obj value when ¯ c N ≤ 0 . For any other feasible solution ˜ x we have: c T ˜ c T ˜ x N ≥ 0 and x = z 0 + ¯ N ˜ x N ≤ z 0 21

Simplex Method Graphical Representation x 2 x 2 x 1 x 1 ? ? ? ? 22

Simplex Method Outline 1. Simplex Method Standard Form Basic Feasible Solutions Algorithm Tableaux and Dictionaries 23

Simplex Method Tableaux and Dictionaries n n max � c j x j x n + i = b i − � a ij x j , i = 1 , . . . , m j = 1 j = 1 n n � a ij x j ≤ b i , i = 1 , . . . , m � z = c j x j j = 1 j = 1 x j ≥ 0 , j = 1 , . . . , n Tableau Dictionary   x r = ¯ b r − � a rs x s , ¯ r ∈ B ¯ ¯ 0 I A N b s �∈ B   z = ¯ d + � c s x s ¯     s �∈ B − ¯ 0 ¯ c N 1 d pivot operations in dictionary form: choose col s with r.c. > 0 choose row with min {− ¯ b i / ¯ a is | a is < 0 , i = 1 , . . . , m } update: express entering variable and substitute in other rows 24

Simplex Method Example max 6 x 1 + 8 x 2 5 x 1 + 10 x 2 ≤ 60 4 x 1 + 4 x 2 ≤ 40 x 1 , x 2 ≥ 0 x 1 x 2 x 3 x 4 − z b x 3 = 60 − 5 x 1 − 10 x 2 x 3 5 10 1 0 0 60 x 4 = 40 − 4 x 1 − 4 x 2 x 4 4 4 0 1 0 40 z = + 6 x 1 + 8 x 2 6 8 0 0 1 0 After 2 iterations: x 1 x 2 x 3 x 4 − z b x 2 = 2 − 1 / 5 x 3 + 1 / 4 x 4 x 2 0 1 1 / 5 − 1 / 4 0 2 x 1 = 8 + 1 / 5 x 3 − 1 / 2 x 4 x 1 1 0 − 1 / 5 1 / 2 0 8 z = 64 − 2 / 5 x 3 − 1 x 4 0 0 − 2 / 5 − 1 1 − 64 25