Operations Research Linear Programming Duality Ling-Chieh Kung - PowerPoint PPT Presentation

Primal-dual pairs Duality theorems Shadow prices Operations Research Linear Programming Duality Ling-Chieh Kung Department of Information Management National Taiwan University Linear Programming Duality 1 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Introduction ◮ For business, we study how to formulate LPs. ◮ For engineering, we study how to solve LPs. ◮ For science, we study mathematical properties of LPs. ◮ We will study Linear Programming duality . ◮ It still has important applications. Linear Programming Duality 2 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Road map ◮ Primal-dual pairs . ◮ Duality theorems. ◮ Shadow prices. Linear Programming Duality 3 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Upper bounds of a maximization LP ◮ Consider the following LP z ∗ = max 4 x 1 + 5 x 2 + 8 x 3 s.t. x 1 + 2 x 2 + 3 x 3 ≤ 6 2 x 1 + x 2 + 2 x 3 ≤ 4 x 1 ≥ 0 , x 2 ≥ 0 , x 3 ≥ 0 . ◮ Suppose the LP is very hard to solve. ◮ Your friend proposes a solution ˆ x = ( 1 2 , 1 , 1) with ˆ z = 15. ◮ If we know z ∗ , we may compare ˆ z with z ∗ . ◮ How to evaluate the performance of ˆ x without solving the LP? ◮ If we can find an upper bound of z ∗ , that works! ◮ z ∗ cannot be greater than the upper bound. ◮ So if ˆ x is quite good. 1 z is close to the upper bound, ˆ 1 You know 97 is quite high without knowing the highest in this class. Linear Programming Duality 4 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Upper bounds of a maximization LP ◮ How to find an upper bound of z ∗ for z ∗ = max 4 x 1 + 5 x 2 + 8 x 3 s.t. x 1 + 2 x 2 + 3 x 3 ≤ 6 2 x 1 + x 2 + 2 x 3 ≤ 4 x 1 ≥ 0 , x 2 ≥ 0 , x 3 ≥ 0? ◮ How about this: Multiply the first constraint by 2, multiply the second constraint by 1, and then add them together: 2( x 1 + 2 x 2 + 3 x 3 ) + (2 x 1 + x 2 + 2 x 3 ) ≤ 2 × 6 + 4 ⇔ 4 x 1 + 5 x 2 + 8 x 3 ≤ 16 . ◮ Compare this with the objective function, we know z ∗ ≤ 16. ◮ Maybe z ∗ is exactly 16 (and the upper bound is tight ). However, we do not know it here. z = 15 is close to z ∗ = 16, so ˆ ◮ ˆ x is quite good. Linear Programming Duality 5 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Upper bounds of a maximization LP ◮ How to find an upper bound of z ∗ for this one? z ∗ = max 3 x 1 + 4 x 2 + 8 x 3 s.t. x 1 + 2 x 2 + 3 x 3 ≤ 6 2 x 1 + x 2 + 2 x 3 ≤ 4 x 1 ≥ 0 , x 2 ≥ 0 , x 3 ≥ 0 . ◮ 16 is also an upper bound: 3 x 1 + 4 x 2 + 8 x 3 ≤ 4 x 1 + 5 x 2 + 8 x 3 (because x 1 ≥ 0 , x 2 ≥ 0) = 2( x 1 + 2 x 2 + 3 x 3 ) + (2 x 1 + x 2 + 2 x 3 ) ≤ 2 × 6 + 4 = 16 . ◮ It is quite likely that 16 is not a tight upper bound and there is a better one. How to improve our upper bound? Linear Programming Duality 6 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Better upper bounds? z ∗ = max 3 x 1 + 4 x 2 + 8 x 3 s.t. x 1 + 2 x 2 + 3 x 3 6 ≤ 2 x 1 + x 2 + 2 x 3 4 ≤ x 1 ≥ 0 , x 2 ≥ 0 , x 3 ≥ 0 . ◮ Changing coefficients multiplied on the two constraints modifies the proposed upper bound. ◮ Different coefficients result in different linear combinations . ◮ Let’s call the two coefficients y 1 and y 2 , respectively: x 1 + 2 x 2 + 3 x 3 ≤ 6 ( × y 1 ) 2 x 1 + x 2 + 2 x 3 ≤ 4 ( × y 2 ) ( y 1 + 2 y 2 ) x 1 + (2 y 1 + y 2 ) x 2 + (3 y 1 + 2 y 2 ) x 3 ≤ 6 y 1 + 4 y 2 ◮ We need y 1 ≥ 0 and y 2 ≥ 0 to preserve the “ ≤ ”. ◮ When do we have z ∗ ≤ 6 y 1 + 4 y 2 ? Linear Programming Duality 7 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Looking for the lowest upper bound ◮ So we look for two variables y 1 and y 2 such that: ◮ y 1 ≥ 0 and y 2 ≥ 0. ◮ 3 ≤ y 1 + 2 y 2 , 4 ≤ 2 y 1 + y 2 , and 8 ≤ 3 y 1 + 2 y 2 . ◮ Then z ∗ ≤ 6 y 1 + 4 y 2 . ◮ To try our best to look for an upper bound, we minimize 6 y 1 + 4 y 2 . We are solving another LP ! min 6 y 1 + 4 y 2 max 3 x 1 + 4 x 2 + 8 x 3 s.t. y 1 + 2 y 2 3 ≥ s.t. x 1 + 2 x 2 + 3 x 3 6 ≤ ⇒ 2 y 1 + y 2 4 ≥ 2 x 1 + x 2 + 2 x 3 ≤ 4 3 y 1 + 2 y 2 8 ≥ x 1 ≥ 0 , x 2 ≥ 0 , x 3 ≥ 0 . y 1 ≥ 0 , y 2 ≥ 0 . ◮ We call the original LP the primal LP and the new one its dual LP. ◮ This idea applies to any LP. Let’s see more examples. Linear Programming Duality 8 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Nonpositive or free variables ◮ Suppose variables are not all nonnegative: z ∗ = max 3 x 1 + 4 x 2 + 8 x 3 s.t. + 2 x 2 + 3 x 3 ≤ 6 x 1 2 x 1 + + 2 x 3 ≤ 4 x 2 x 1 ≥ 0 , x 2 ≤ 0 , x 3 urs. ◮ If we want 3 x 1 + 4 x 2 + 8 x 3 ≤ ( y 1 + 2 y 2 ) x 1 + (2 y 1 + y 2 ) x 2 + (3 y 1 + 2 y 2 ) x 3 , now we need + 2 y 2 ≥ 3 because x 1 ≥ 0 , y 1 2 y 1 + ≤ 4 because x 2 ≤ 0 , and y 2 3 y 1 + 2 y 2 = 8 because x 3 is free. Linear Programming Duality 9 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Nonpositive or free variables ◮ So the primal and dual LPs are min 6 y 1 + 4 y 2 max 3 x 1 + 4 x 2 + 8 x 3 s.t. y 1 + 2 y 2 ≥ 3 s.t. x 1 + 2 x 2 + 3 x 3 6 ≤ and 2 y 1 + y 2 ≤ 4 2 x 1 + x 2 + 2 x 3 4 ≤ 3 y 1 + 2 y 2 = 8 x 1 ≥ 0 , x 2 ≤ 0 , x 3 urs. y 1 ≥ 0 , y 2 ≥ 0 . ◮ Some observations: ◮ Primal max ⇒ Dual min. ◮ Primal objective ⇒ Dual RHS. ◮ Primal RHS ⇒ Dual objective. ◮ Moreover: ◮ Primal “ ≥ 0” variable ⇒ Dual “ ≥ ” constraint. ◮ Primal “ ≤ 0” variable ⇒ Dual “ ≤ ” constraint. ◮ Primal free variable ⇒ Dual “=” constraint. ◮ What if we have “ ≥ ” or “=” primal constraints? Linear Programming Duality 10 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices No-less-than and equality constraints ◮ Suppose constraints are not all “ ≤ ”: z ∗ = max 3 x 1 + 4 x 2 + 8 x 3 s.t. x 1 + 2 x 2 + 3 x 3 ≥ 6 2 x 1 + x 2 + 2 x 3 = 4 x 1 ≥ 0 , x 2 ≤ 0 , x 3 urs. ◮ To obtain y 1 ( x 1 + 2 x 2 + 3 x 3 ) + y 2 (2 x 1 + x 2 + 2 x 3 ) ≤ 6 y 1 + 4 y 2 , we now need y 1 ≤ 0. y 2 can be of any sign (i.e., free). Linear Programming Duality 11 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices No-less-than and equality constraints ◮ So the primal and dual LPs are min 6 y 1 + 4 y 2 max 3 x 1 + 4 x 2 + 8 x 3 s.t. y 1 + 2 y 2 ≥ 3 s.t. x 1 + 2 x 2 + 3 x 3 6 ≥ and 2 y 1 + 4 y 2 ≤ 2 x 1 + x 2 + 2 x 3 = 4 3 y 1 + 2 y 2 = 8 x 1 ≥ 0 , x 2 ≤ 0 , x 3 urs. y 1 ≤ 0 , y 2 urs. ◮ Some more observations: ◮ Primal “ ≤ ” constraint ⇒ Dual “ ≥ 0” variable. ◮ Primal “ ≥ ” constraint ⇒ Dual “ ≤ 0” variable. ◮ Primal “=” constraint ⇒ Dual free variable. Linear Programming Duality 12 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices The general rule ◮ In general, if the primal LP is max c 1 x 1 + c 2 x 2 + c 3 x 3 s.t. A 11 x 1 + A 12 x 2 + A 13 x 3 ≥ b 1 A 21 x 1 + A 22 x 2 + A 23 x 3 ≤ b 2 A 31 x 1 + A 32 x 2 + A 33 x 3 = b 3 x 1 ≥ 0 , x 2 ≤ 0 , x 3 urs. , its dual LP is min b 1 y 1 + b 2 y 2 + b 3 y 3 s.t. A 11 y 1 + A 21 y 2 + A 31 y 3 ≥ c 1 A 12 y 1 + A 22 y 2 + A 32 y 3 ≤ c 2 A 13 y 1 + A 23 y 2 + A 33 y 3 = c 3 y 1 ≤ 0 , y 2 ≥ 0 , y 3 urs. ◮ Note that the constraint coefficient matrix is “ transposed ”. Linear Programming Duality 13 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices Matrix representation ◮ In general, if the primal LP ◮ In matrix representation: max c 1 x 1 + c 2 x 2 + c 3 x 3 c T x s.t. A 11 x 1 + A 12 x 2 + A 13 x 3 = b 1 max A 21 x 1 + A 22 x 2 + A 23 x 3 = b 2 s.t. Ax = b A 31 x 1 + A 32 x 2 + A 33 x 3 = b 3 x ≥ 0 x 1 ≥ 0 , x 2 ≥ 0 , x 3 ≥ 0 , and is in the standard form , its dual LP is y T b min min + + b 1 y 1 b 2 y 2 b 3 y 3 y T A ≥ c T . s.t. s.t. + + ≥ A 11 y 1 A 21 y 2 A 31 y 3 c 1 + + ≥ A 12 y 1 A 22 y 2 A 32 y 3 c 2 + + ≥ A 13 y 1 A 23 y 2 A 33 y 3 c 3 . Linear Programming Duality 14 / 42 Ling-Chieh Kung (NTU IM)

Primal-dual pairs Duality theorems Shadow prices The dual LP for a minimization primal LP ◮ For a minimization LP, its dual LP is to maximize the lower bound . ◮ Rules for the directions of variables and constraints are reversed : max 6 y 1 + 4 y 2 min 3 x 1 + 4 x 2 + 8 x 3 s.t. y 1 + 2 y 2 3 ≤ s.t. x 1 + 2 x 2 + 3 x 3 6 ≥ ⇒ 2 y 1 + y 2 4 ≥ 2 x 1 + x 2 + 2 x 3 ≤ 4 3 y 1 + 2 y 2 = 8 x 1 ≥ 0 , x 2 ≤ 0 , x 3 urs. y 1 ≥ 0 , y 2 ≤ 0 . ◮ Note that 3 x 1 + 4 x 2 + 8 x 3 ≥ ( y 1 + 2 y 2 ) x 1 + (2 y 1 + y 2 ) x 2 + (3 y 1 + 2 y 2 ) x 3 ≥ ( x 1 + 2 x 2 + 3 x 3 ) y 1 + (2 x 1 + x 2 + 2 x 3 ) y 2 ≥ 6 y 1 + 4 y 2 . Linear Programming Duality 15 / 42 Ling-Chieh Kung (NTU IM)