Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, - PowerPoint PPT Presentation

Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33

Example of a Linear Program minimize 2 x 1 − + 4 x 3 x 2 subject to + + ≤ 2 x 1 x 2 x 4 3 x 2 − x 3 = 5 x 3 + x 4 ≥ 3 ≥ 0 x 1 x 3 ≤ 0 Remarks. ◮ objective function is linear in vector of variables x = ( x 1 , x 2 , x 3 , x 4 ) T ◮ constraints are linear inequalities and linear equations ◮ last two constraints are special (non-negativity and non-positivity constraint, respectively) 34

General Linear Program c T · x minimize a i T · x ≥ b i subject to for i ∈ M 1 , (2.1) a i T · x = b i for i ∈ M 2 , (2.2) a i T · x ≤ b i for i ∈ M 3 , (2.3) x j ≥ 0 for j ∈ N 1 , (2.4) x j ≤ 0 for j ∈ N 2 , (2.5) with c ∈ R n , a i ∈ R n and b i ∈ R for i ∈ M 1 ˙ ∪ M 2 ˙ ∪ M 3 (finite index sets), and N 1 , N 2 ⊆ { 1 , . . . , n } given. ◮ x ∈ R n satisfying constraints (2.1) – (2.5) is a feasible solution. ◮ feasible solution x ∗ is optimal solution if c T · x ∗ ≤ c T · x for all feasible solutions x . ◮ linear program is unbounded if, for all k ∈ R , there is a feasible solution x ∈ R n with c T · x ≤ k . 35

Special Forms of Linear Programs ◮ maximizing c T · x is equivalent to minimizing ( − c ) T · x . ◮ any linear program can be written in the form c T · x minimize subject to A · x ≥ b for some A ∈ R m × n and b ∈ R m : ◮ rewrite a i T · x = b i as: a i T · x ≥ b i a i T · x ≤ b i , ∧ ◮ rewrite a i T · x ≤ b i as: ( − a i ) T · x ≥ − b i . ◮ Linear program in standard form: c T · x min s.t. A · x = b x ≥ 0 with A ∈ R m × n , b ∈ R m , and c ∈ R n . 36

Example: Diet Problem Given: ◮ n different foods, m different nutrients ◮ a ij := amount of nutrient i in one unit of food j ◮ b i := requirement of nutrient i in some ideal diet ◮ c j := cost of one unit of food j Task: find a cheapest ideal diet consisting of foods 1 , . . . , n . LP formulation: Let x j := number of units of food j in the diet: c T · x c T · x min min s.t. A · x = b or s.t. A · x ≥ b x ≥ 0 x ≥ 0 with A = ( a ij ) ∈ R m × n , b = ( b i ) ∈ R m , c = ( c j ) ∈ R n . 37

Reduction to Standard Form Any linear program can be brought into standard form: ◮ elimination of free (unbounded) variables x j : replace x j with x + x j = x + j , x − ≥ 0: j − x − j j ◮ elimination of non-positive variables x j : replace x j ≤ 0 with ( − x j ) ≥ 0. ◮ elimination of inequality constraint a i T · x ≤ b i : introduce slack variable s ≥ 0 and rewrite: a i T · x + s = b i ◮ elimination of inequality constraint a i T · x ≥ b i : introduce slack variable s ≥ 0 and rewrite: a i T · x − s = b i 38

Example The linear program min 2 x 1 + 4 x 2 s.t. + ≥ 3 x 1 x 2 3 x 1 + 2 x 2 = 14 x 1 ≥ 0 is equivalent to the standard form problem 4 x + 4 x − min 2 x 1 + − 2 2 x + s.t. + − x − − = 3 x 1 x 3 2 2 2 x + 2 x − 3 x 1 + − = 14 2 2 x 1 , x + 2 , x − 2 , x 3 ≥ 0 39

Affine Linear and Convex Functions Lemma 2.1. a An affine linear function f : R n → R given by f ( x ) = c T · x + d with c ∈ R n , d ∈ R , is both convex and concave. b If f 1 , . . . , f k : R n → R are convex functions, then f : R n → R defined by f ( x ) := max i = 1 ,..., k f i ( x ) is also convex. 40

Piecewise Linear Convex Objective Functions Let c 1 , . . . , c k ∈ R n and d 1 , . . . , d k ∈ R . Consider piecewise linear convex function: x �→ max i = 1 ,..., k c i T · x + d i : i = 1 ,..., k c i T · x + d i min max min z z ≥ c i T · x + d i s.t. A · x ≥ b ← → s.t. for all i A · x ≥ b Example: let c 1 , . . . , c n ≥ 0 n n � � min c i · | x i | min c i · z i i = 1 i = 1 s.t. A · x ≥ b ↔ s.t. z i ≥ x i z i ≥ − x i A · x ≥ b 41

Graphical Representation and Solution 2D example: min − x 1 − x 2 x 2 s.t. + 2 x 2 ≤ 3 x 1 3 2 x 1 + x 2 ≤ 3 2.5 x 1 , x 2 ≥ 0 2 1.5 1 0.5 x 1 0 0 0.5 1 1.5 2 2.5 3 42

Graphical Representation and Solution (cont.) 3D example: x 2 min − x 1 − x 2 − x 3 s.t. ≤ 1 x 1 x 3 x 2 ≤ 1 x 3 ≤ 1 1 ≥ 0 x 1 , x 2 , x 3 1 x 1 0 0 1 43

Graphical Representation and Solution (cont.) x 2 another 2D example: 3 min + c 1 x 1 c 2 x 2 2 s.t. − x 1 + x 2 ≤ 1 x 1 , x 2 ≥ 0 1 x 1 0 0 1 2 3 ◮ for c = ( 1 , 1 ) T , the unique optimal solution is x = ( 0 , 0 ) T ◮ for c = ( 1 , 0 ) T , the optimal solutions are exactly the points x = ( 0 , x 2 ) T with 0 ≤ x 2 ≤ 1 ◮ for c = ( 0 , 1 ) T , the optimal solutions are exactly the points x = ( x 1 , 0 ) T with x 1 ≥ 0 ◮ for c = ( − 1 , − 1 ) T , the problem is unbounded, optimal cost is −∞ ◮ if we add the constraint x 1 + x 2 ≤ − 1, the problem is infeasible 44

Properties of the Set of Optimal Solutions In the last example, the following 5 cases occurred: i there is a unique optimal solution ii there exist infinitely many optimal solutions, but the set of optimal solutions is bounded iii there exist infinitely many optimal solutions and the set of optimal solutions is unbounded iv the problem is unbounded, i. e., the optimal cost is −∞ and no feasible solution is optimal v the problem is infeasible, i. e., the set of feasible solutions is empty These are indeed all cases that can occur in general (see later). 45

Visualizing LPs in Standard Form Example: Let A = ( 1 , 1 , 1 ) ∈ R 1 × 3 , b = ( 1 ) ∈ R 1 and consider the set of feasible solutions P = { x ∈ R 3 | A · x = b , x ≥ 0 } . x 2 x 3 1 1 x 1 0 0 1 46

Visualizing LPs in Standard Form More general: ◮ if A ∈ R m × n with m ≤ n and the rows of A are linearly independent, then { x ∈ R n | A · x = b } is an ( n − m ) -dimensional affine subspace in R n . ◮ set of feasible solutions lies in this affine subspace and is only constrained by non-negativity constraints x ≥ 0. 47