CS 473: Algorithms Chandra Chekuri Ruta Mehta University of Illinois, Urbana-Champaign Fall 2016 Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 39
CS 473: Algorithms, Fall 2016 LP Duality Lecture 20 November 2, 2016 Chandra & Ruta (UIUC) CS473 2 Fall 2016 2 / 39
An easy LP? max cx subject to Ax = b which is compact form for max c 1 x 1 + c 2 x 2 + . . . + c n x n a i 1 x 1 + a i 2 x 2 + . . . + a in x n = 1 ≤ i ≤ m b i Question: Is this a geneal LP problem or is it some how easy? Chandra & Ruta (UIUC) CS473 3 Fall 2016 3 / 39
An easy LP? max cx subject to Ax = b Basically reduces to linear system solving. Three cases for Ax = b . The system Ax = b is infeasible, that is, no solution The system Ax = b has a unique solution x ∗ when rank ([ A b ]) = n (full rank). Optimum solution value is cx ∗ The system Ax = b has infinite solutions when rank ([ A b ]) < n . There all vectors of the form x ∗ + y are feasible where y is null-space ( A ) = { y | Ay = 0 } . Let d be dimension of null-space ( A ) and let e 1 , e 2 , . . . , e d be an orthonormal basis. Then cx = cx ∗ + cy = cx ∗ + c ( λ 1 e 1 + λ 2 e 2 + . . . + λ d e d . If ce i � = 0 for any i then optimum solution value is unbounded. Otherwise cx ∗ . Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 39
LP Canonical Forms Two basic canonical forms: max cx , Ax = b , x ≥ 0 max cx , Ax ≤ b , x ≥ 0 What makes LP non-trivial and different from linear system solving is the additional non-negativity constraint on variables. Chandra & Ruta (UIUC) CS473 5 Fall 2016 5 / 39
Part I Derivation and Definition of Dual LP Chandra & Ruta (UIUC) CS473 6 Fall 2016 6 / 39
Feasible Solutions and Lower Bounds Consider the program maximize 4 x 1 + 2 x 2 subject to x 1 + 3 x 2 ≤ 5 2 x 1 − 4 x 2 ≤ 10 x 1 + x 2 ≤ 7 ≤ 5 x 1 Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Feasible Solutions and Lower Bounds Consider the program maximize 4 x 1 + 2 x 2 subject to x 1 + 3 x 2 ≤ 5 2 x 1 − 4 x 2 ≤ 10 x 1 + x 2 ≤ 7 ≤ 5 x 1 (0 , 1) satisfies all the constraints and gives value 2 for the 1 objective function. Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Feasible Solutions and Lower Bounds Consider the program maximize 4 x 1 + 2 x 2 subject to x 1 + 3 x 2 ≤ 5 2 x 1 − 4 x 2 ≤ 10 x 1 + x 2 ≤ 7 ≤ 5 x 1 (0 , 1) satisfies all the constraints and gives value 2 for the 1 objective function. Thus, optimal value σ ∗ is at least 4 . 2 Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Feasible Solutions and Lower Bounds Consider the program maximize 4 x 1 + 2 x 2 subject to x 1 + 3 x 2 ≤ 5 2 x 1 − 4 x 2 ≤ 10 x 1 + x 2 ≤ 7 ≤ 5 x 1 (0 , 1) satisfies all the constraints and gives value 2 for the 1 objective function. Thus, optimal value σ ∗ is at least 4 . 2 (2 , 0) also feasible, and gives a better bound of 8 . 3 Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Feasible Solutions and Lower Bounds Consider the program maximize 4 x 1 + 2 x 2 subject to x 1 + 3 x 2 ≤ 5 2 x 1 − 4 x 2 ≤ 10 x 1 + x 2 ≤ 7 ≤ 5 x 1 (0 , 1) satisfies all the constraints and gives value 2 for the 1 objective function. Thus, optimal value σ ∗ is at least 4 . 2 (2 , 0) also feasible, and gives a better bound of 8 . 3 How good is 8 when compared with σ ∗ ? 4 Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Obtaining Upper Bounds Let us multiply the first constraint by 2 and the and add it to 1 second constraint 2( x 1 + 3 x 2 ) ≤ 2(5) +1( 2 x 1 − 4 x 2 ) ≤ 1(10) 4 x 1 + 2 x 2 ≤ 20 Thus, 20 is an upper bound on the optimum value! 2 Chandra & Ruta (UIUC) CS473 8 Fall 2016 8 / 39
Generalizing . . . Multiply first equation by y 1 , second by y 2 , third by y 3 and 1 fourth by y 4 (all of y 1 , y 2 , y 3 , y 4 being positive) and add y 1 ( x 1 + 3 x 2 ) ≤ y 1 (5) + y 2 ( 2 x 1 − 4 x 2 ) ≤ y 2 (10) + y 3 ( x 1 + ) ≤ y 3 (7) x 2 + y 4 ( x 1 ) ≤ y 4 (5) ( y 1 + 2 y 2 + y 3 + y 4 ) x 1 + (3 y 1 − 4 y 2 + y 3 ) x 2 ≤ . . . 5 y 1 + 10 y 2 + 7 y 3 + 5 y 4 is an upper bound, provided 2 coefficients of x i are same as in the objective function, i.e., y 1 + 2 y 2 + y 3 + y 4 = 4 3 y 1 − 4 y 2 + y 3 = 2 The best upper bound is when 5 y 1 + 10 y 2 + 7 y 3 + 5 y 4 is 3 minimized! Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 39
Dual LP: Example Thus, the optimum value of program maximize 4 x 1 + 2 x 2 subject to x 1 + 3 x 2 ≤ 5 2 x 1 − 4 x 2 ≤ 10 x 1 + x 2 ≤ 7 x 1 ≤ 5 is upper bounded by the optimal value of the program minimize 5 y 1 + 10 y 2 + 7 y 3 + 5 y 4 subject to y 1 + 2 y 2 + y 3 + y 4 = 4 3 y 1 − 4 y 2 + y 3 = 2 y 1 , y 2 ≥ 0 Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 39
Dual Linear Program Given a linear program Π in canonical form � d maximize j =1 c j x j � d subject to j =1 a ij x j ≤ b i i = 1 , 2 , . . . n the dual Dual (Π) is given by � n minimize i =1 b i y i � n subject to i =1 y i a ij = c j j = 1 , 2 , . . . d y i ≥ 0 i = 1 , 2 , . . . n Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 39
Dual Linear Program Given a linear program Π in canonical form � d maximize j =1 c j x j � d subject to j =1 a ij x j ≤ b i i = 1 , 2 , . . . n the dual Dual (Π) is given by � n minimize i =1 b i y i � n subject to i =1 y i a ij = c j j = 1 , 2 , . . . d y i ≥ 0 i = 1 , 2 , . . . n Proposition Dual ( Dual (Π)) is equivalent to Π Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 39
Duality Theorems Theorem (Weak Duality) If x ′ is a feasible solution to Π and y ′ is a feasible solution to Dual (Π) then c · x ′ ≤ y ′ · b . Chandra & Ruta (UIUC) CS473 12 Fall 2016 12 / 39
Duality Theorems Theorem (Weak Duality) If x ′ is a feasible solution to Π and y ′ is a feasible solution to Dual (Π) then c · x ′ ≤ y ′ · b . Theorem (Strong Duality) If x ∗ is an optimal solution to Π and y ∗ is an optimal solution to Dual (Π) then c · x ∗ = y ∗ · b . Many applications! Maxflow-Mincut theorem can be deduced from duality. Chandra & Ruta (UIUC) CS473 12 Fall 2016 12 / 39
Proof of Weak Duality We already saw the proof by the way we derived it but we will do it again formally. Since y ′ is feasible to Dual (Π) : y ′ A = c Therefore c · x ′ = y ′ Ax ′ Since x ′ is feasible Ax ′ ≤ b and hence, c · x ′ = y ′ Ax ′ ≤ y ′ · b Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 39
Duality for another canonical form maximize 4 x 1 + x 2 + 3 x 3 subject to x 1 + 4 x 2 ≤ 2 2 x 1 − x 2 + x 3 ≤ 4 x 1 , x 2 , x 3 ≥ 0 Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 39
Duality for another canonical form maximize 4 x 1 + x 2 + 3 x 3 subject to x 1 + 4 x 2 ≤ 2 2 x 1 − x 2 + x 3 ≤ 4 x 1 , x 2 , x 3 ≥ 0 Choose non-negative y 1 , y 2 and multiply inequalities maximize 4 x 1 + x 2 + 3 x 3 subject to y 1 ( x 1 + 4 x 2 ) ≤ 2 y 1 y 2 (2 x 1 − x 2 + x 3 ) ≤ 4 y 2 x 1 , x 2 , x 3 ≥ 0 Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 39
Duality for another canonical form Choose non-negative y 1 , y 2 and multiply inequalities maximize 4 x 1 + x 2 + 3 x 3 subject to y 1 ( x 1 + 4 x 2 ) ≤ 2 y 1 y 2 (2 x 1 − x 2 + x 3 ) ≤ 4 y 2 x 1 , x 2 , x 3 ≥ 0 Adding the inequalities we get an inequality below that is valid for any feasible x and any non-negative y : ( y 1 + 2 y 2 ) x 1 + (4 y 1 − y 2 ) x 2 + y 2 ≤ 2 y 1 + 4 y 2 Suppose we choose y 1 , y 2 such that y 1 + 2 y 2 ≥ 4 and 4 y 2 − y 2 ≥ 1 and 2 y 1 ≥ 3 Then, since x 1 , x 2 , x 3 ≥ 0 , we have 4 x 1 + x 2 + 3 x 3 ≤ 2 y 1 + 4 y 2 Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 39
Duality for another canonical form maximize 4 x 1 + x 2 + 3 x 3 subject to x 1 + 4 x 2 ≤ 2 2 x 1 − x 2 + x 3 ≤ 4 x 1 , x 2 , x 3 ≥ 0 is upper bounded by minimize 2 y 1 + 4 y 2 subject to y 1 + 2 y 2 ≥ 4 4 y 1 − y 2 ≥ 1 2 y 1 ≥ 3 y 1 , y 2 ≥ 0 Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 39
Duality for another canonical form Compactly, For the primal LP max cx subject to Ax ≤ b , x ≥ 0 the dual LP is min yb subject to yA ≥ c , y ≥ 0 Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 39
Some Useful Duality Properties Assume primal LP is a maximization LP. For a given LP, Dual is another LP. The variables in the dual correspond to “non-trival” primal constraints and vice-versa. Dual of the dual LP give us back the primal LP. Weak and strong duality theorems. If primal is unbounded (objective achieves infinity) then dual LP is infeasible. Why? If dual LP had a feasible solution it would upper bound the primal LP which is not possible. If primal is infeasible then dual LP is unbounded. Primal and dual optimum solutions satisfy complementary slackness conditions (discussed soon). Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 39
Part II Examples of Duality Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 39
Max matching in bipartite graph as LP Input:G = ( V = L ∪ R , E ) � max x uv uv ∈ E � s . t . x uv ≤ 1 ∀ v ∈ V . uv ∈ E x uv ≥ 0 ∀ uv ∈ E Chandra & Ruta (UIUC) CS473 20 Fall 2016 20 / 39
Recommend
More recommend
