CS672: Approximation Algorithms Spring 14 Introduction to Linear - PowerPoint PPT Presentation

CS672: Approximation Algorithms Spring 14 Introduction to Linear Programming II Instructor: Shaddin Dughmi

Outline Recall: Duality and Its Interpretations 1 Weak and Strong Duality 2 Consequences of Duality 3 Uses and Examples of Duality 4 Solvability of LP 5

Outline Recall: Duality and Its Interpretations 1 Weak and Strong Duality 2 Consequences of Duality 3 Uses and Examples of Duality 4 Solvability of LP 5

Linear Programming Duality “Flexible” Form: Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y = c y � 0 Recall: Duality and Its Interpretations 1/21

Linear Programming Duality “Flexible” Form: Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y = c y � 0 Packing/Covering form: Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Recall: Duality and Its Interpretations 1/21

Interpretation 1: Economic Interpretation Primal LP � n max j =1 c j x j s.t. � n j =1 a ij x j ≤ b i , ∀ i ∈ [ m ] . x j ≥ 0 , ∀ j ∈ [ n ] . n products, m raw materials Every unit of product j uses a ij units of raw material i There are b i units of material i available Product j yields profit c j per unit Facility wants to maximize profit Recall: Duality and Its Interpretations 2/21

Interpretation 1: Economic Interpretation Primal LP Dual LP � n max j =1 c j x j � m min i =1 b i y i s.t. s.t. � n j =1 a ij x j ≤ b i , ∀ i ∈ [ m ] . � m i =1 a ij y i ≥ c j , ∀ j ∈ [ n ] . x j ≥ 0 , ∀ j ∈ [ n ] . y i ≥ 0 , ∀ i ∈ [ m ] . n products, m raw materials y i is a proposed price per unit of raw material i Every unit of product j uses a ij units of raw material i Feasibility means facility has incentive to sell as opposed to There are b i units of material i produce available Buyer wants to spend as little as Product j yields profit c j per unit possible to buy materials Facility wants to maximize profit Recall: Duality and Its Interpretations 2/21

Interpretation 2: Finding the Best Upperbound Primal LP Dual LP maximize minimize c ⊺ x b ⊺ y subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Multiplying each row i by y i and summing gives the inequality y T Ax ≤ y T b When y T A � c T , we have c T x ≤ y T Ax ≤ y T b The dual LP can be thought of as trying to find the best upperbound on the primal that can be achieved by combining inequalities this way. Recall: Duality and Its Interpretations 3/21

Interpretation 3: Physical Forces Apply force field c to a ball inside polytope Ax � b . Recall: Duality and Its Interpretations 4/21

Interpretation 3: Physical Forces Apply force field c to a ball inside polytope Ax � b . Eventually, ball will come to rest against the walls of the polytope. Recall: Duality and Its Interpretations 4/21

Interpretation 3: Physical Forces Apply force field c to a ball inside polytope Ax � b . Eventually, ball will come to rest against the walls of the polytope. Wall a i x ≤ b i applies some force − y i a i to the ball Recall: Duality and Its Interpretations 4/21

Interpretation 3: Physical Forces Apply force field c to a ball inside polytope Ax � b . Eventually, ball will come to rest against the walls of the polytope. Wall a i x ≤ b i applies some force − y i a i to the ball Since the ball is still, c T = � i y i a i = y T A . Recall: Duality and Its Interpretations 4/21

Interpretation 3: Physical Forces Apply force field c to a ball inside polytope Ax � b . Eventually, ball will come to rest against the walls of the polytope. Wall a i x ≤ b i applies some force − y i a i to the ball Since the ball is still, c T = � i y i a i = y T A . Dual can be thought of as trying to minimize “work” � i y i b i to bring ball back to origin by moving polytope Recall: Duality and Its Interpretations 4/21

Outline Recall: Duality and Its Interpretations 1 Weak and Strong Duality 2 Consequences of Duality 3 Uses and Examples of Duality 4 Solvability of LP 5

Weak Duality Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Theorem (Weak Duality) For every primal feasible x and dual feasible y , we have c ⊺ x ≤ b ⊺ y . Corollary If primal and dual both feasible and bounded, OPT ( Primal ) ≤ OPT ( Dual ) If primal is unbounded, dual is infeasible If dual is unbounded, primal is infeasible Weak and Strong Duality 5/21

Weak Duality Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Theorem (Weak Duality) For every primal feasible x and dual feasible y , we have c ⊺ x ≤ b ⊺ y . Corollary If x is primal feasible, and y is dual feasible, and c ⊺ x = b ⊺ y , then both are optimal. Weak and Strong Duality 5/21

Interpretation of Weak Duality Economic Interpretation If selling the raw materials is more profitable than making any individual product, then total money collected from sale of raw materials would exceed profit from production. Weak and Strong Duality 6/21

Interpretation of Weak Duality Economic Interpretation If selling the raw materials is more profitable than making any individual product, then total money collected from sale of raw materials would exceed profit from production. Upperbound Interpretation Self explanatory Weak and Strong Duality 6/21

Interpretation of Weak Duality Economic Interpretation If selling the raw materials is more profitable than making any individual product, then total money collected from sale of raw materials would exceed profit from production. Upperbound Interpretation Self explanatory Physical Interpretation Work required to bring ball back to origin by pulling polytope is at least potential energy difference between origin and primal optimum. Weak and Strong Duality 6/21

Proof of Weak Duality Primal LP Dual LP maximize c ⊺ x minimize b ⊺ y subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 c ⊺ x ≤ y ⊺ Ax ≤ y ⊺ b Weak and Strong Duality 7/21

Strong Duality Primal LP Dual LP maximize minimize c ⊺ x b ⊺ y subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Theorem (Strong Duality) If either the primal or dual is feasible and bounded, then so is the other and OPT ( Primal ) = OPT ( Dual ) . Weak and Strong Duality 8/21

Interpretation of Strong Duality Economic Interpretation Buyer can offer prices for raw materials that would make facility indifferent between production and sale. Weak and Strong Duality 9/21

Interpretation of Strong Duality Economic Interpretation Buyer can offer prices for raw materials that would make facility indifferent between production and sale. Upperbound Interpretation The method of scaling and summing inequalities yields a tight upperbound on the primal optimal value. Weak and Strong Duality 9/21

Interpretation of Strong Duality Economic Interpretation Buyer can offer prices for raw materials that would make facility indifferent between production and sale. Upperbound Interpretation The method of scaling and summing inequalities yields a tight upperbound on the primal optimal value. Physical Interpretation There is an assignment of forces to the walls of the polytope that brings ball back to the origin without wasting energy. Weak and Strong Duality 9/21

Informal Proof of Strong Duality Recall the physical interpretation of duality Weak and Strong Duality 10/21

Informal Proof of Strong Duality Recall the physical interpretation of duality When ball is stationary at x , we expect force c to be neutralized only by constraints that are tight i.e. force multipliers y such that y i ( b i − a i x ) = 0 Weak and Strong Duality 10/21

Informal Proof of Strong Duality Recall the physical interpretation of duality When ball is stationary at x , we expect force c to be neutralized only by constraints that are tight i.e. force multipliers y such that y i ( b i − a i x ) = 0 y ⊺ b − c ⊺ x = y ⊺ b − y T Ax = � y i ( b i − a i x ) = 0 i We found a primal and dual solution that are equal in value! Weak and Strong Duality 10/21

Outline Recall: Duality and Its Interpretations 1 Weak and Strong Duality 2 Consequences of Duality 3 Uses and Examples of Duality 4 Solvability of LP 5

Complementary Slackness Primal LP Dual LP maximize minimize c ⊺ x y ⊺ b subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Consequences of Duality 11/21

Complementary Slackness Primal LP Dual LP maximize minimize c ⊺ x y ⊺ b subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Let s i = ( b − Ax ) i be the i ’th primal slack variable Let t j = ( A ⊺ y − c ) j be the j ’th dual slack variable Consequences of Duality 11/21

Complementary Slackness Primal LP Dual LP maximize minimize c ⊺ x y ⊺ b subject to Ax � b subject to A ⊺ y � c x � 0 y � 0 Let s i = ( b − Ax ) i be the i ’th primal slack variable Let t j = ( A ⊺ y − c ) j be the j ’th dual slack variable Complementary Slackness x 1 x 2 x 3 x 4 y 1 a 11 a 12 a 13 a 14 b 1 x and y are optimal if and only if y 2 a 21 a 22 a 23 a 24 b 2 x j t j = 0 for all j = 1 , . . . , n y 3 a 31 a 32 a 33 a 34 b 3 y i s i = 0 for all i = 1 , . . . , m c 1 c 2 c 3 c 4 Consequences of Duality 11/21