Primal-Dual Algorithm Math 482, Lecture 29 Misha Lavrov April 17, - PowerPoint PPT Presentation

Introduction The direction-finding problem The augmenting step The restricted primal Primal-Dual Algorithm Math 482, Lecture 29 Misha Lavrov April 17, 2020

Introduction The direction-finding problem The augmenting step The restricted primal The problem Our goal: to solve the primal-dual pair of linear programs below.   u T b c T x minimize maximize   u ∈ R m x ∈ R n     ( P ) ( D ) u T A ≤ c T subject to A x = b subject to     x ≥ 0 u unrestricted  

Introduction The direction-finding problem The augmenting step The restricted primal The problem Our goal: to solve the primal-dual pair of linear programs below.   u T b c T x minimize maximize   u ∈ R m x ∈ R n     ( P ) ( D ) u T A ≤ c T subject to A x = b subject to     x ≥ 0 u unrestricted   We will try to improve on the simplex algorithm by taking long jumps across the feasible region for ( D ).

Introduction The direction-finding problem The augmenting step The restricted primal The problem Our goal: to solve the primal-dual pair of linear programs below.   u T b c T x minimize maximize   u ∈ R m x ∈ R n     ( P ) ( D ) u T A ≤ c T subject to A x = b subject to     x ≥ 0 u unrestricted   We will try to improve on the simplex algorithm by taking long jumps across the feasible region for ( D ). Motivation: the Ford–Fulkerson method, where a single augmenting steps changes many variables at once.

Introduction The direction-finding problem The augmenting step The restricted primal The direction-finding problem Consider the following example:  maximize 3 u 1 + 3 u 2 u ∈ R 2      subject to 2 u 1 + 4 u 2 ≤ 2 ( D ) u 1 − u 2 ≤ 2      − 4 u 1 + u 2 ≤ 1 We are at the point u = (1 , 0) and want to pick a direction v to go. What is the best direction, and how do we find it?

Introduction The direction-finding problem The augmenting step The restricted primal The direction-finding problem Consider the following example:  maximize 3 u 1 + 3 u 2 u ∈ R 2      subject to 2 u 1 + 4 u 2 ≤ 2 ( D ) u 1 − u 2 ≤ 2      − 4 u 1 + u 2 ≤ 1 We are at the point u = (1 , 0) and want to pick a direction v to go. What is the best direction, and how do we find it? Answer 1. (From calculus.) Gradient descent: take v proportional to b = (3 , 3), the cost vector.

Introduction The direction-finding problem The augmenting step The restricted primal The direction-finding problem Consider the following example:  maximize 3 u 1 + 3 u 2 u ∈ R 2      subject to 2 u 1 + 4 u 2 ≤ 2 ( D ) u 1 − u 2 ≤ 2      − 4 u 1 + u 2 ≤ 1 We are at the point u = (1 , 0) and want to pick a direction v to go. What is the best direction, and how do we find it? Answer 1. (From calculus.) Gradient descent: take v proportional to b = (3 , 3), the cost vector. Answer 2 . We should also make sure we don’t accidentally leave the feasible region.

Introduction The direction-finding problem The augmenting step The restricted primal The linear program for v We can find v using a linear program:

Introduction The direction-finding problem The augmenting step The restricted primal The linear program for v We can find v using a linear program: Objective function: we want to improve 3 u 1 + 3 u 2 as quickly as possible, so we maximize 3 v 1 + 3 v 2 .

Introduction The direction-finding problem The augmenting step The restricted primal The linear program for v We can find v using a linear program: Objective function: we want to improve 3 u 1 + 3 u 2 as quickly as possible, so we maximize 3 v 1 + 3 v 2 . Boundary conditions: At u = (1 , 0), the constraint 2 u 1 + 4 u 2 ≤ 2 is tight, so we make sure we don’t violate it and ask that 2 v 1 + 4 v 2 ≤ 0 .

Introduction The direction-finding problem The augmenting step The restricted primal The linear program for v We can find v using a linear program: Objective function: we want to improve 3 u 1 + 3 u 2 as quickly as possible, so we maximize 3 v 1 + 3 v 2 . Boundary conditions: At u = (1 , 0), the constraint 2 u 1 + 4 u 2 ≤ 2 is tight, so we make sure we don’t violate it and ask that 2 v 1 + 4 v 2 ≤ 0 . The constraints u 1 − u 2 ≤ 2 and − 4 u 1 + u 2 ≤ 1 are slack, so we ignore them.

Introduction The direction-finding problem The augmenting step The restricted primal The linear program for v We can find v using a linear program: Objective function: we want to improve 3 u 1 + 3 u 2 as quickly as possible, so we maximize 3 v 1 + 3 v 2 . Boundary conditions: At u = (1 , 0), the constraint 2 u 1 + 4 u 2 ≤ 2 is tight, so we make sure we don’t violate it and ask that 2 v 1 + 4 v 2 ≤ 0 . The constraints u 1 − u 2 ≤ 2 and − 4 u 1 + u 2 ≤ 1 are slack, so we ignore them. Scaling constraints: we just want a direction, not a magnitude. So we limit v by asking that v 1 , v 2 ≤ 1 .

Introduction The direction-finding problem The augmenting step The restricted primal The linear program for v We can find v using a linear program: Objective function: we want to improve 3 u 1 + 3 u 2 as quickly as possible, so we maximize 3 v 1 + 3 v 2 . Boundary conditions: At u = (1 , 0), the constraint 2 u 1 + 4 u 2 ≤ 2 is tight, so we make sure we don’t violate it and ask that 2 v 1 + 4 v 2 ≤ 0 . The constraints u 1 − u 2 ≤ 2 and − 4 u 1 + u 2 ≤ 1 are slack, so we ignore them. Scaling constraints: we just want a direction, not a magnitude. So we limit v by asking that v 1 , v 2 ≤ 1 . (Fine print: this only works if the coefficients in the objective function u T b are nonnegative, but we can make sure that this holds.)

Introduction The direction-finding problem The augmenting step The restricted primal The direction-finding linear program The original LP, ( D ), leads to the auxiliary LP, ( DRP ): u T b v T b   maximize maximize   u ∈ R m v ∈ R m     ( D ) ( DRP ) u T A ≤ c T v T A J ≤ 0 T subject to subject to     u unrestricted v 1 , . . . , v m ≤ 1   (Here, J indexes constraints which are tight at the initial point u .)

Introduction The direction-finding problem The augmenting step The restricted primal The direction-finding linear program The original LP, ( D ), leads to the auxiliary LP, ( DRP ): u T b v T b   maximize maximize   u ∈ R m v ∈ R m     ( D ) ( DRP ) u T A ≤ c T v T A J ≤ 0 T subject to subject to     u unrestricted v 1 , . . . , v m ≤ 1   (Here, J indexes constraints which are tight at the initial point u .) In our example, we get:  maximize 3 v 1 + 3 v 2 v ∈ R 2      subject to 2 v 1 + 4 v 2 ≤ 0 ( DRP ) ≤ 1 v 1      v 2 ≤ 1

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly.

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly. 1 Start at some solution to ( D ). ( u 1 , u 2 ) = (1 , 0)

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly. 1 Start at some solution to ( D ). ( u 1 , u 2 ) = (1 , 0) 2 Write the direction-finding problem ( DRP ). (previous slide)

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly. 1 Start at some solution to ( D ). ( u 1 , u 2 ) = (1 , 0) 2 Write the direction-finding problem ( DRP ). (previous slide) 3 Solve ( DRP ) to find a direction v . We get ( v 1 , v 2 ) = (1 , − 1 2 )

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly. 1 Start at some solution to ( D ). ( u 1 , u 2 ) = (1 , 0) 2 Write the direction-finding problem ( DRP ). (previous slide) 3 Solve ( DRP ) to find a direction v . We get ( v 1 , v 2 ) = (1 , − 1 2 ) 4 Find the largest t such that u + t v is still feasible for ( D ).

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly. 1 Start at some solution to ( D ). ( u 1 , u 2 ) = (1 , 0) 2 Write the direction-finding problem ( DRP ). (previous slide) 3 Solve ( DRP ) to find a direction v . We get ( v 1 , v 2 ) = (1 , − 1 2 ) 4 Find the largest t such that u + t v is still feasible for ( D ).  2( u 1 + tv 1 ) + 4( u 2 + tv 2 ) ≤ 2 holds automatically    

Introduction The direction-finding problem The augmenting step The restricted primal The augmenting step We can now do the primal-dual algorithm, just very badly. 1 Start at some solution to ( D ). ( u 1 , u 2 ) = (1 , 0) 2 Write the direction-finding problem ( DRP ). (previous slide) 3 Solve ( DRP ) to find a direction v . We get ( v 1 , v 2 ) = (1 , − 1 2 ) 4 Find the largest t such that u + t v is still feasible for ( D ).  2( u 1 + tv 1 ) + 4( u 2 + tv 2 ) ≤ 2 holds automatically   ⇒ t ≤ 2 ( u 1 + tv 1 ) − ( u 2 + tv 2 ) ≤ 2 = 3  