Linear Programming Lecture 16 October 21, 2014 Sariel (UIUC) - PowerPoint PPT Presentation

CS 573: Algorithms, Fall 2014 Linear Programming Lecture 16 October 21, 2014 Sariel (UIUC) CS573 1 Fall 2014 1 / 35

Part I Linear Programming Sariel (UIUC) CS573 2 Fall 2014 2 / 35

Economic planning Guns/nuclear-bombs/napkins/star-wars/professors/butter/mice problem Penguina: a country. 1 Ruler need to decide how to allocate resources. 2 Maximize benefit. 3 Budget allocation 4 (i) Nuclear bomb has a tremendous positive effect on security while being expensive. (ii) Guns, on the other hand, have a weaker effect. Penguina need to prove a certain level of security: 5 x gun + 1000 ∗ x nuclear − bomb ≥ 1000 , where x guns : # guns x nuclear − bomb : # nuclear-bombs constructed. 100 ∗ x gun + 1000000 ∗ x nuclear − bomb ≤ x security 6 x security : total amount spent on security. 100 / 1 , 000 , 000 : price of producing a single gun/nuclear bomb. Sariel (UIUC) CS573 3 Fall 2014 3 / 35

Linear programming An instance of linear programming ( LP ): x 1 , . . . , x n : variables. 1 For j = 1 , . . . , m : a j 1 x 1 + . . . + a jn x n ≤ b j : linear 2 inequality. i.e., constraint . 3 Q: ∃ s an assignment of values to x 1 , . . . , x n such that all 4 inequalities are satisfied. Many possible solutions... Want solution that maximizes some 5 linear quantity. objective function : linear inequality being maximized. 6 Sariel (UIUC) CS573 4 Fall 2014 4 / 35

Linear programming – example a 11 x 1 + . . . + a 1 n x n ≤ b 1 a 21 x 1 + . . . + a 2 n x n ≤ b 2 . . . a m 1 x 1 + . . . + a mn x n ≤ b m max c 1 x 1 + . . . + c n x n . Sariel (UIUC) CS573 5 Fall 2014 5 / 35

History 1939: L. V. Kantorovich noticed the importance of certain type 1 of Linear Programming problems for resource allocation. 1947: Dantzig invented the simplex method for solving LP 2 problems for the US Air force planning problems. 1947: T. C. Koopmans showed LP provide the right model for 3 the analysis of classical economic theories. 1975: Koopmans and Kantorovich got the Nobel prize of 4 economics. Kantorovich the only the Russian economist that got the Nobel 5 prize Sariel (UIUC) CS573 6 Fall 2014 6 / 35

Network flow via linear programming Input: G = ( V , E ) with source s and sink t , and capacities c ( · ) on the edges. Compute max flow in G . ∀ ( u → v ) ∈ E 0 ≤ x u → v x u → v ≤ c ( u → v ) � � ∀ v ∈ V \ { s , t } x u → v − x v → w ≤ 0 ( u → v ) ∈ E ( v → w ) ∈ E � � x u → v − x v → w ≥ 0 ( u → v ) ∈ E ( v → w ) ∈ E maximizing � ( s → u ) ∈ E x s → u Sariel (UIUC) CS573 7 Fall 2014 7 / 35

Part II The Simplex Algorithm Sariel (UIUC) CS573 8 Fall 2014 8 / 35

Rewriting an LP n � max c j x j j =1 n � subject to a ij x j ≤ b i for i = 1 , 2 , . . . , m j =1 Rewrite: so every variable is non-negative. 1 Replace variable x i by x ′ i and x ′′ i , where new constraints are: 2 x i = x ′ i − x ′′ x ′ x ′′ i , i ≥ 0 and i ≥ 0 . Example: The (silly) LP 2 x + y ≥ 5 rewritten: 3 2 x ′ − 2 x ′′ + y ′ − y ′′ ≥ 5 , x ′ ≥ 0 , y ′ ≥ 0 , x ′′ ≥ 0 , and y ′′ ≥ 0 . Sariel (UIUC) CS573 9 Fall 2014 9 / 35

Rewriting an LP n � max c j x j j =1 n � subject to a ij x j ≤ b i for i = 1 , 2 , . . . , m j =1 Rewrite: so every variable is non-negative. 1 Replace variable x i by x ′ i and x ′′ i , where new constraints are: 2 x i = x ′ i − x ′′ x ′ x ′′ i , i ≥ 0 and i ≥ 0 . Example: The (silly) LP 2 x + y ≥ 5 rewritten: 3 2 x ′ − 2 x ′′ + y ′ − y ′′ ≥ 5 , x ′ ≥ 0 , y ′ ≥ 0 , x ′′ ≥ 0 , and y ′′ ≥ 0 . Sariel (UIUC) CS573 9 Fall 2014 9 / 35

Rewriting an LP n � max c j x j j =1 n � subject to a ij x j ≤ b i for i = 1 , 2 , . . . , m j =1 Rewrite: so every variable is non-negative. 1 Replace variable x i by x ′ i and x ′′ i , where new constraints are: 2 x i = x ′ i − x ′′ x ′ x ′′ i , i ≥ 0 and i ≥ 0 . Example: The (silly) LP 2 x + y ≥ 5 rewritten: 3 2 x ′ − 2 x ′′ + y ′ − y ′′ ≥ 5 , x ′ ≥ 0 , y ′ ≥ 0 , x ′′ ≥ 0 , and y ′′ ≥ 0 . Sariel (UIUC) CS573 9 Fall 2014 9 / 35

Rewriting an LP into standard form Lemma Given an instance I of LP , one can rewrite it into an equivalent LP , such that all the variables must be non-negative. This takes linear time in the size of I . An LP where all variables must be non-negative is in standard form Sariel (UIUC) CS573 10 Fall 2014 10 / 35

Rewriting an LP into standard form Lemma Given an instance I of LP , one can rewrite it into an equivalent LP , such that all the variables must be non-negative. This takes linear time in the size of I . An LP where all variables must be non-negative is in standard form Sariel (UIUC) CS573 10 Fall 2014 10 / 35

Standard form of LP A linear program in standard form. n � max c j x j j =1 n � subject to a ij x j ≤ b i for i = 1 , 2 , . . . , m j =1 x j ≥ 0 for j = 1 , . . . , n . Sariel (UIUC) CS573 11 Fall 2014 11 / 35

Standard form of LP Because everything is clearer when you use matrices. Not.   c , b and A : a 11 a 12 . . . a 1( n − 1) a 1 n prespecified. a 21 a 22 . . . a 2( n − 1) a 2 n     . .   x is vector of . . A = ,  . . . . . . . . . . .    unknowns.   a ( m − 1)1 a ( m − 1)2 . . . a ( m − 1)( n − 1) a ( m − 1) n   Solve LP for   . . . a m 1 a m 2 a m ( n − 1) a mn x . LP in standard form. (Matrix notation.)   x 1     c 1 b 1 x 2   c T x   max . . .   . . . c =   , b =    , x =  . .  .        s.t. Ax ≤ b .     c n b m x n − 1     x ≥ 0 . x n Sariel (UIUC) CS573 12 Fall 2014 12 / 35

Slack Form Next rewrite LP into slack 1 form . c T x max Every inequality becomes 2 subject to Ax = b . equality. x ≥ 0 . All variables must be positive. 3 See resulting form on the 4 right. New slack variables . Rewrite inequality: � n i =1 a i x i ≤ b . As: 1 n � x n +1 = b − a i x i i =1 x n +1 ≥ 0 . Value of slack variable x n +1 encodes how far is the original 2 inequality for holding with equality. Sariel (UIUC) CS573 13 Fall 2014 13 / 35

Slack Form Next rewrite LP into slack 1 form . c T x max Every inequality becomes 2 subject to Ax = b . equality. x ≥ 0 . All variables must be positive. 3 See resulting form on the 4 right. New slack variables . Rewrite inequality: � n i =1 a i x i ≤ b . As: 1 n � x n +1 = b − a i x i i =1 x n +1 ≥ 0 . Value of slack variable x n +1 encodes how far is the original 2 inequality for holding with equality. Sariel (UIUC) CS573 13 Fall 2014 13 / 35

Slack Form Next rewrite LP into slack 1 form . c T x max Every inequality becomes 2 subject to Ax = b . equality. x ≥ 0 . All variables must be positive. 3 See resulting form on the 4 right. New slack variables . Rewrite inequality: � n i =1 a i x i ≤ b . As: 1 n � x n +1 = b − a i x i i =1 x n +1 ≥ 0 . Value of slack variable x n +1 encodes how far is the original 2 inequality for holding with equality. Sariel (UIUC) CS573 13 Fall 2014 13 / 35

Slack Form Next rewrite LP into slack 1 form . c T x max Every inequality becomes 2 subject to Ax = b . equality. x ≥ 0 . All variables must be positive. 3 See resulting form on the 4 right. New slack variables . Rewrite inequality: � n i =1 a i x i ≤ b . As: 1 n � x n +1 = b − a i x i i =1 x n +1 ≥ 0 . Value of slack variable x n +1 encodes how far is the original 2 inequality for holding with equality. Sariel (UIUC) CS573 13 Fall 2014 13 / 35

Slack form... LP now made of equalities of the form: 1 � n x n +1 = b − i =1 a i x i Variables on left: basic variables . 2 Variables on right: nonbasic variables . 3 LP in this form is in slack form . 4 Linear program in slack form. � max z = v + c j x j , j ∈ N � s.t. x i = b i − a ij x j for i ∈ B , j ∈ N x i ≥ 0 , ∀ i = 1 , . . . , n + m . Sariel (UIUC) CS573 14 Fall 2014 14 / 35

Slack form... LP now made of equalities of the form: 1 � n x n +1 = b − i =1 a i x i Variables on left: basic variables . 2 Variables on right: nonbasic variables . 3 LP in this form is in slack form . 4 Linear program in slack form. � max z = v + c j x j , j ∈ N � s.t. x i = b i − a ij x j for i ∈ B , j ∈ N x i ≥ 0 , ∀ i = 1 , . . . , n + m . Sariel (UIUC) CS573 14 Fall 2014 14 / 35

Slack form... LP now made of equalities of the form: 1 � n x n +1 = b − i =1 a i x i Variables on left: basic variables . 2 Variables on right: nonbasic variables . 3 LP in this form is in slack form . 4 Linear program in slack form. � max z = v + c j x j , j ∈ N � s.t. x i = b i − a ij x j for i ∈ B , j ∈ N x i ≥ 0 , ∀ i = 1 , . . . , n + m . Sariel (UIUC) CS573 14 Fall 2014 14 / 35

Slack form... LP now made of equalities of the form: 1 � n x n +1 = b − i =1 a i x i Variables on left: basic variables . 2 Variables on right: nonbasic variables . 3 LP in this form is in slack form . 4 Linear program in slack form. � max z = v + c j x j , j ∈ N � s.t. x i = b i − a ij x j for i ∈ B , j ∈ N x i ≥ 0 , ∀ i = 1 , . . . , n + m . Sariel (UIUC) CS573 14 Fall 2014 14 / 35