Linear Program Duality Frdric Giroire FG Duality 1/24 Motivation - PowerPoint PPT Presentation

Linear Program Duality Frédéric Giroire FG Duality 1/24

Motivation • Finding bounds on the optimal solution. Provides a measure of the "goodness" of a solution. • Provide certificate of optimality. • Economic interpretation of the dual problem. FG Duality 2/24

** Introduction to Duality ** FG Duality 3/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : − − + ≤ x 1 x 2 x 3 3 x 4 1 + + + ≤ 5 x 1 x 2 3 x 3 8 x 4 55 − x 1 + + − ≤ 2 x 2 3 x 3 5 x 4 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Lower bound: a feasible solution, e.g. ( 0 , 0 , 1 , 0 ) ⇒ z ∗ ≥ 5. What if we want an upper bound? FG Duality 4/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : x 1 − x 2 − x 3 + 3 x 4 ≤ 1 5 x 1 + x 2 + 3 x 3 + 8 x 4 ≤ 55 − x 1 + 2 x 2 + 3 x 3 − 5 x 4 ≤ 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Second Inequation × 5 / 3: 25 3 x 1 + 5 3 x 2 + 5 x 3 + 40 3 x 4 ≤ 275 3 . Note that (all variables are positive), 4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤ 25 3 x 1 + 5 3 x 2 + 5 x 3 + 40 3 x 4 Hence, a first bound: z ∗ ≤ 275 3 . FG Duality 5/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : x 1 − x 2 − x 3 + 3 x 4 ≤ 1 5 x 1 + x 2 + 3 x 3 + 8 x 4 ≤ 55 − x 1 + 2 x 2 + 3 x 3 − 5 x 4 ≤ 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Second Inequation × 5 / 3: 25 3 x 1 + 5 3 x 2 + 5 x 3 + 40 3 x 4 ≤ 275 3 . Note that (all variables are positive), 4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤ 25 3 x 1 + 5 3 x 2 + 5 x 3 + 40 3 x 4 Hence, a first bound: z ∗ ≤ 275 3 . FG Duality 5/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : x 1 − x 2 − x 3 + 3 x 4 ≤ 1 5 x 1 + x 2 + 3 x 3 + 8 x 4 ≤ 55 − x 1 + 2 x 2 + 3 x 3 − 5 x 4 ≤ 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Second Inequation × 5 / 3: 25 3 x 1 + 5 3 x 2 + 5 x 3 + 40 3 x 4 ≤ 275 3 . Note that (all variables are positive), 4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤ 25 3 x 1 + 5 3 x 2 + 5 x 3 + 40 3 x 4 Hence, a first bound: z ∗ ≤ 275 3 . FG Duality 5/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : − − + ≤ x 1 x 2 x 3 3 x 4 1 + + + ≤ 5 x 1 x 2 3 x 3 8 x 4 55 − x 1 + + − ≤ 2 x 2 3 x 3 5 x 4 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Similarly, 2 d + 3 d constraints: 4 x 1 + 3 x 2 + 6 x 3 + 3 x 4 ≤ 58 . Hence, a second bound: z ∗ ≤ 58 . → need for a systematic strategy. FG Duality 6/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : − − + ≤ x 1 x 2 x 3 3 x 4 1 + + + ≤ 5 x 1 x 2 3 x 3 8 x 4 55 − x 1 + + − ≤ 2 x 2 3 x 3 5 x 4 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Similarly, 2 d + 3 d constraints: 4 x 1 + 3 x 2 + 6 x 3 + 3 x 4 ≤ 58 . Hence, a second bound: z ∗ ≤ 58 . → need for a systematic strategy. FG Duality 6/24

Duality Theorem: introduction + + + Maximize 4 x 1 x 2 5 x 3 3 x 4 Subject to : x 1 − x 2 − x 3 + 3 x 4 ≤ 1 × y 1 + + + ≤ × y 2 5 x 1 x 2 3 x 3 8 x 4 55 − x 1 + 2 x 2 + 3 x 3 − 5 x 4 ≤ 3 × y 3 x 1 , x 2 , x 3 , x 4 ≥ 0 . Build linear combinations of the constraints. Summing: ( y 1 + 5 y 2 − y 3 ) x 1 +( − y 1 + y 2 + 2 y 3 ) x 2 +( − y 1 + 3 y 2 + 3 y 3 ) x 3 +( 3 y 1 + 8 y 2 − 5 y 3 ) x 4 ≤ y 1 + 55 y 2 + 3 y 3 . We want left part upper bound of z . We need coefficient of x j ≥ coefficient in z : + − ≥ y 1 5 y 2 y 3 4 − y 1 + + ≥ y 2 2 y 3 1 − y 1 + + ≥ 3 y 2 3 y 3 5 + − ≥ 3 . 3 y 1 8 y 2 5 y 3 FG Duality 7/24

Duality Theorem: introduction Build linear combinations of the constraints. Summing: ( y 1 + 5 y 2 − y 3 ) x 1 +( − y 1 + y 2 + 2 y 3 ) x 2 +( − y 1 + 3 y 2 + 3 y 3 ) x 3 +( 3 y 1 + 8 y 2 − 5 y 3 ) x 4 ≤ y 1 + 55 y 2 + 3 y 3 . We want left part upper bound of z . We need coefficient of x j ≥ coefficient in z : + − ≥ y 1 5 y 2 y 3 4 − y 1 + y 2 + 2 y 3 ≥ 1 − y 1 + + ≥ 3 y 2 3 y 3 5 + − ≥ 3 . 3 y 1 8 y 2 5 y 3 If the y i ≥ 0 and satisfy theses inequations, then 4 x 1 + x 2 + 5 x 3 + 3 x 4 ≤ y 1 + 55 y 2 + 3 y 3 . In particular, z ∗ ≤ y 1 + 55 y 2 + 3 y 3 . FG Duality 7/24

Duality Theorem: introduction Objective: smallest possible upper bound. Hence, we solve the following PL: + + Minimize y 1 55 y 2 3 y 3 Subject to: y 1 + 5 y 2 − y 3 ≥ 4 − y 1 + + ≥ y 2 2 y 3 1 (1) − y 1 + + ≥ 3 y 2 3 y 3 5 + − ≥ 3 y 1 8 y 2 5 y 3 3 y 1 , y 2 , y 3 ≥ 0 . It is the dual problem of the problem. FG Duality 8/24

** Duality ** FG Duality 9/24

The Dual Problem Primal problem: ∑ n Maximize j = 1 c j x j ∑ n ≤ ( i = 1 , 2 , ··· , m ) Subject to: j = 1 a ij x j b i (2) x j ≥ 0 ( j = 1 , 2 , ··· , n ) Its dual problem is defined by the LP problem: ∑ m Minimize i = 1 b i y i ∑ m ≥ ( j = 1 , 2 , ··· , n ) Subject to: i = 1 a ij y i c j (3) ≥ ( i = 1 , 2 , ··· , m ) y i 0 FG Duality 10/24

Weak Duality Theorem Theorem: If ( x 1 , x 2 ,..., x n ) is feasible for the primal and ( y 1 , y 2 ,... y n ) is feasible for the dual, then c j x j ≤ ∑ ∑ b i y i . j i Proof: ≤ ∑ j ( ∑ i y i a ij ) x j dual definition: ∑ i y i a ij ≥ c j ∑ j c j x j = ∑ i ( ∑ j a ij x j ) y i ≤ ∑ i b i y i primal definition: ∑ i x i a ij ≤ b j FG Duality 11/24

Gap or No Gap? An important question: Is there a gap between the largest primal value and the smallest dual value? FG Duality 12/24

Strong Duality Theorem Theorem: If the primal problem has an optimal solution, x ∗ = ( x ∗ 1 ,..., X ∗ n ) , then the dual also has an optimal solution, y ∗ = ( y ∗ 1 ,..., y ∗ n ) , and j = ∑ c j x ∗ b i y ∗ ∑ i . j i FG Duality 13/24

Relationship between the Primal and Dual Problems Lemma: The dual of the dual is always the primal problem. Corollary: + (Strong Duality Theorem) ⇒ Primal has an optimal solution iff dual has an optimal solution. Weak duality: Primal unbounded ⇒ dual unfeasible. Dual Optimal Unfeasible Unbounded Optimal X Primal Unfeasible X X Unbounded X FG Duality 14/24

Relationship between the Primal and Dual Problems Lemma: The dual of the dual is always the primal problem. Corollary: + (Strong Duality Theorem) ⇒ Primal has an optimal solution iff dual has an optimal solution. Weak duality: Primal unbounded ⇒ dual unfeasible. Dual Optimal Unfeasible Unbounded Optimal X Primal Unfeasible X X Unbounded X FG Duality 14/24

** Certificate of Optimality ** FG Duality 15/24

Complementary Slackness Theorem: Let x ∗ 1 ,... x ∗ n be a feasible solution of the primal and y ∗ 1 ,... y ∗ n be a feasible solution of the dual. Then, m a ij y ∗ x ∗ ∑ i = c j j = 0 or both ( j = 1 , 2 ,... n ) or i = 1 n a ij x ∗ y ∗ ∑ j = b i i = 0 or both ( i = 1 , 2 ,... m ) or j = 1 are necessary and sufficient conditions to have the optimality of x ∗ and y ∗ . FG Duality 16/24

Complementary Slackness - Proof x ∗ feasible ⇒ b i − ∑ j a ij x j ≥ 0. y ∗ dual feasible, hence non negative. Thus ( b i − ∑ a ij x j ) y i ≥ 0 . j Similarly, y ∗ dual feasible ⇒ ∑ i a ij y i − c j ≥ 0. x ∗ feasible, hence non negative. ( ∑ a ij y i − c j ) x j ≥ 0 . i FG Duality 17/24

Complementary Slackness - Proof ( b i − ∑ ( ∑ a ij x j ) y i ≥ 0 and a ij y i − c j ) x j ≥ 0 j i By summing, we get: ∑ ( b i − ∑ ∑ ( ∑ a ij x j ) y i ≥ 0 a ij y i − c j ) x j ≥ 0 and i j j i Summing + strong duality theorem: c j x j = ∑ ∑ b i y i − ∑ a ij x j y i + ∑ a ij y i x j − ∑ b i y i − ∑ c j x j = 0 . i i , j j , i j i Implies: inequalities must be equalities: ∀ i , ( b i − ∑ ∀ j ( ∑ a ij x j ) y i = 0 a ij y i − c j ) x j = 0 . and j i XY = 0 if X = 0 or Y = 0. Done. FG Duality 18/24

Complementary Slackness - Proof ( b i − ∑ ( ∑ a ij x j ) y i ≥ 0 and a ij y i − c j ) x j ≥ 0 j i By summing, we get: ∑ ( b i − ∑ ∑ ( ∑ a ij x j ) y i ≥ 0 a ij y i − c j ) x j ≥ 0 and i j j i Summing + strong duality theorem: c j x j = ∑ ∑ b i y i − ∑ a ij x j y i + ∑ a ij y i x j − ∑ b i y i − ∑ c j x j = 0 . i i , j j , i j i Implies: inequalities must be equalities: ∀ i , ( b i − ∑ ∀ j ( ∑ a ij x j ) y i = 0 a ij y i − c j ) x j = 0 . and j i XY = 0 if X = 0 or Y = 0. Done. FG Duality 18/24

Complementary Slackness - Proof ( b i − ∑ ( ∑ a ij x j ) y i ≥ 0 and a ij y i − c j ) x j ≥ 0 j i By summing, we get: ∑ ( b i − ∑ ∑ ( ∑ a ij x j ) y i ≥ 0 a ij y i − c j ) x j ≥ 0 and i j j i Summing + strong duality theorem: c j x j = ∑ ∑ b i y i − ∑ a ij x j y i + ∑ a ij y i x j − ∑ b i y i − ∑ c j x j = 0 . i i , j j , i j i Implies: inequalities must be equalities: ∀ i , ( b i − ∑ ∀ j ( ∑ a ij x j ) y i = 0 a ij y i − c j ) x j = 0 . and j i XY = 0 if X = 0 or Y = 0. Done. FG Duality 18/24