CS/ECE/ISyE 524 Introduction to Optimization Spring 2017–18 6. Duality ❼ Estimating LP bounds ❼ LP duality ❼ Simple example ❼ Sensitivity and shadow prices ❼ Complementary slackness ❼ Another simple example Laurent Lessard (www.laurentlessard.com)
The Top Brass example revisited maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 Suppose the maximum profit is p ⋆ . How can we bound p ⋆ ? ❼ Finding a lower bound is easy... pick any feasible point! ◮ { f = 0 , s = 0 } is feasible. So p ⋆ ≥ 0 (we can do better...) ◮ { f = 500 , s = 1000 } is feasible. So p ⋆ ≥ 15000. ◮ { f = 1000 , s = 400 } is feasible. So p ⋆ ≥ 15600. ❼ Each feasible point of the LP yields a lower bound for p ⋆ . ❼ Finding the largest lower bound = solving the LP! 6-2
Estimating upper bounds maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 Suppose the maximum profit is p ⋆ . How can we bound p ⋆ ? ❼ Finding an upper bound is harder... (use the constraints!) ◮ 12 f + 9 s ≤ 12 · 1000 + 9 · 1500 = 25500. So p ⋆ ≤ 25500. ◮ 12 f + 9 s ≤ f + (4 f + 2 s ) + 7( f + s ) ≤ 1000 + 4800 + 7 · 1750 = 18050. So p ⋆ ≤ 18050. ❼ Combining the constraints in different ways yields different upper bounds on the optimal profit p ⋆ . 6-3
Estimating upper bounds maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 Suppose the maximum profit is p ⋆ . How can we bound p ⋆ ? What is the best upper bound we can find by combining constraints in this manner? 6-4
Estimating upper bounds maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 ❼ Let λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 be the multipliers. If we can choose them such that for any feasible f and s , we have: 12 f + 9 s ≤ λ 1 (4 f + 2 s ) + λ 2 ( f + s ) + λ 3 f + λ 4 s (1) Then, using the constraints, we will have the following upper bound on the optimal profit: 12 f + 9 s ≤ 4800 λ 1 + 1750 λ 2 + 1000 λ 3 + 1500 λ 4 6-5
Estimating upper bounds maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 ❼ Let λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 be the multipliers. If we can choose them such that for any feasible f and s , we have: 12 f + 9 s ≤ λ 1 (4 f + 2 s ) + λ 2 ( f + s ) + λ 3 f + λ 4 s (1) Rearranging (1), we get: 0 ≤ (4 λ 1 + λ 2 + λ 3 − 12) f + (2 λ 1 + λ 2 + λ 4 − 9) s We can ensure this always holds by choosing λ 1 , λ 2 , λ 3 , λ 4 to make the bracketed terms nonnegative. 6-6
Estimating upper bounds maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 ❼ Recap : If we choose λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 such that: 4 λ 1 + λ 2 + λ 3 ≥ 12 and 2 λ 1 + λ 2 + λ 4 ≥ 9 Then we have a upper bound on the optimal profit: p ⋆ ≤ 4800 λ 1 + 1750 λ 2 + 1000 λ 3 + 1500 λ 4 Finding the best (smallest) upper bound is... an LP! 6-7
The dual of Top Brass maximize 12 f + 9 s f , s subject to: 4 f + 2 s ≤ 4800 , f + s ≤ 1750 0 ≤ f ≤ 1000 , 0 ≤ s ≤ 1500 To find the best upper bound, solve the dual problem: minimize 4800 λ 1 + 1750 λ 2 + 1000 λ 3 + 1500 λ 4 λ 1 ,λ 2 ,λ 3 ,λ 4 subject to: 4 λ 1 + λ 2 + λ 3 ≥ 12 2 λ 1 + λ 2 + λ 4 ≥ 9 λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 6-8
The dual of Top Brass Primal problem: Dual problem: maximize 12 f + 9 s minimize 4800 λ 1 + 1750 λ 2 f , s λ 1 ,...,λ 4 + 1000 λ 3 + 1500 λ 4 subject to: 4 f + 2 s ≤ 4800 f + s ≤ 1750 subject to: 4 λ 1 + λ 2 + λ 3 ≥ 12 f ≤ 1000 2 λ 1 + λ 2 + λ 4 ≥ 9 s ≤ 1500 λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 f , s ≥ 0 Solution is p ⋆ . Solution is d ⋆ . ❼ Primal is a maximization, dual is a minimization. ❼ There is a dual variable for each primal constraint. ❼ There is a dual constraint for each primal variable. ❼ (any feasible primal point) ≤ p ⋆ ≤ d ⋆ ≤ (any feasible dual point) 6-9
The dual of Top Brass Primal problem: Dual problem: � T � T � � 12 4800 f λ 1 max 9 s 1750 λ 2 f , s min 1000 λ 3 λ 1 ,...,λ 4 4 2 4800 1500 λ 4 � f � 1 1 1750 s.t. ≤ 1 0 1000 λ 1 s � 4 � � 12 � 0 1 1500 1 1 0 λ 2 s.t. ≥ 2 1 0 1 λ 3 9 f , s ≥ 0 λ 4 λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 Using matrix notation... Code: Top Brass dual.ipynb 6-10
General LP duality Primal problem (P) Dual problem (D) c T x b T λ maximize minimize x λ A T λ ≥ c subject to: Ax ≤ b subject to: x ≥ 0 λ ≥ 0 If x and λ are feasible points of (P) and (D) respectively: c T x ≤ p ⋆ ≤ d ⋆ ≤ b T λ Powerful fact: if p ⋆ and d ⋆ exist and are finite, then p ⋆ = d ⋆ . This property is known as strong duality . 6-11
General LP duality Primal problem (P) Dual problem (D) c T x b T λ maximize minimize x λ A T λ ≥ c subject to: Ax ≤ b subject to: x ≥ 0 λ ≥ 0 1. optimal p ⋆ is attained 1. optimal d ⋆ is attained 2. unbounded: p ⋆ = + ∞ 2. unbounded: d ⋆ = −∞ 3. infeasible: p ⋆ = −∞ 3. infeasible: d ⋆ = + ∞ Which combinations are possible? Remember: p ⋆ ≤ d ⋆ . 6-12
General LP duality Primal problem (P) Dual problem (D) c T x b T λ maximize minimize x λ A T λ ≥ c subject to: Ax ≤ b subject to: x ≥ 0 λ ≥ 0 There are exactly four possibilities: 1. (P) and (D) are both feasible and bounded, and p ⋆ = d ⋆ . 2. p ⋆ = + ∞ (unbounded primal) and d ⋆ = + ∞ (infeasible dual). 3. p ⋆ = −∞ (infeasible primal) and d ⋆ = −∞ (unbounded dual). 4. p ⋆ = −∞ (infeasible primal) and d ⋆ = + ∞ (infeasible dual). 6-13
More properties of the dual To find the dual of an LP that is not in standard form: 1. convert the LP to standard form True for LP duality, 2. write the dual not true in general. 3. make simplifications Example: What is the dual of the dual? the primal! b T λ ( − c ) T z min − min dual λ z equiv A T λ ≥ c s.t. s.t. ( − A ) z ≥ ( − b ) λ ≥ 0 z ≥ 0 ( − b ) T λ c T z − max max λ z ( − A T ) λ ≤ ( − c ) s.t. s.t. Az ≤ b equiv λ ≥ 0 z ≥ 0 6-14
More duals b T λ c T x max min dual x λ Standard form: s.t. Ax ≤ b s.t. λ ≥ 0 A T λ ≥ c x ≥ 0 b T λ c T x min max dual x λ Free form: s.t. Ax ≤ b s.t. λ ≥ 0 A T λ = c x free b T λ + g T µ c T x min max λ,µ x dual s.t. Ax ≤ b s.t. λ ≥ 0 Mixed constraints: Fx = g µ free A T λ + F T µ = c x free 6-15
More duals Equivalences between primal and dual problems Minimization Maximization Nonnegative variable ≥ Inequality constraint ≤ Nonpositive variable ≤ Inequality constraint ≥ Free variable Equality constraint = Inequality constraint ≥ Nonnegative variable ≥ Inequality constraint ≤ Nonpositive variable ≤ Equality constraint = Free Variable 6-16
Simple example Why should we care about the dual? 1. It can sometimes make a problem easier to solve min 2 λ λ max 3 x + y + 2 z s.t. λ ≥ 3 x , y , z dual s.t. x + 2 y + z ≤ 2 2 λ ≥ 1 x , y , z ≥ 0 λ ≥ 2 λ ≥ 0 ◮ Dual is much easier in this case! ◮ Many solvers take advantage of duality. 2. Duality is related to the idea of sensitivity: how much do each of your constraints affect the optimal cost? 6-17
Sensitivity Primal problem: Dual problem: maximize 12 f + 9 s minimize 4800 λ 1 + 1750 λ 2 f , s λ 1 ,...,λ 4 + 1000 λ 3 + 1500 λ 4 subject to: 4 f + 2 s ≤ 4800 f + s ≤ 1750 subject to: 4 λ 1 + λ 2 + λ 3 ≥ 12 f ≤ 1000 2 λ 1 + λ 2 + λ 4 ≥ 9 s ≤ 1500 λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 f , s ≥ 0 Solution is p ⋆ . Solution is d ⋆ . If Millco offers to sell me more wood at a price of ✩ 1 per board foot, should I accept the offer? 6-18
Sensitivity Primal problem: Dual problem: maximize 12 f + 9 s minimize 4800 λ 1 + 1750 λ 2 f , s λ 1 ,...,λ 4 + 1000 λ 3 + 1500 λ 4 subject to: 4 f + 2 s ≤ 4800 f + s ≤ 1750 subject to: 4 λ 1 + λ 2 + λ 3 ≥ 12 f ≤ 1000 2 λ 1 + λ 2 + λ 4 ≥ 9 s ≤ 1500 λ 1 , λ 2 , λ 3 , λ 4 ≥ 0 f , s ≥ 0 Solution is p ⋆ . Solution is d ⋆ . ❼ changes in primal constraints are changes in the dual cost . ❼ a small change to the feasible set of the primal problem can change the optimal f and s , but λ 1 , . . . , λ 4 will not change! ❼ if we increase 4800 by 1, then p ⋆ = d ⋆ increases by λ 1 . 6-19
Sensitivity of Top Brass 1 , 500 0 max 12 f + 9 s f , s 6 1 , 250 s.t. 4 f + 2 s ≤ 5200 (dual variables) f + s ≤ 1750 soccer balls ( s ) 1 , 000 0 ≤ f ≤ 1000 0 ≤ s ≤ 1500 750 1.5 What happens if we p 500 = add 400 wood? p $ 9 = , 0 0 Profit goes up by ✩ 600! $ 0 6 250 , 0 0 0 0 shadow price is ✩ 1.50, so ✩ 1 is a good price. 0 0 250 500 750 1 , 000 footballs ( f ) 6-20
Recommend
More recommend
