Linear Programming and Network Optimization Zongpeng Li Department - PowerPoint PPT Presentation

Linear Programming and Network Optimization Zongpeng Li Department of Computer Science University of Calgary Zongpeng Li – p.1/28

Outline • Hello World linear program • The power of LP • LP models in network optimization • LP duality • Solving LPs • Beyond LP Zongpeng Li – p.2/28

Hello World maximize 2 x + y s . t . : ≤ 2 x ≤ 2 y x + y ≤ 3 x, y ≥ 0 Zongpeng Li – p.3/28

Hello World y 2 1 1 2 0 x Zongpeng Li – p.4/28

The power of LP But we all know the world is nonlinear. — Harold Hotelling, 1948 Zongpeng Li – p.5/28

The power of LP But we all know the world is nonlinear. 1. If you have a problem that satisfies the axioms (of LP), then use it. If it does not, then don’t. — John von Neumann, 1948 2. Much more problems can be modelled using LPs than suggested by intuition. 3. LP constitutes building blocks for nonlinear programming Zongpeng Li – p.6/28

LP model: max-flow 5/10 B C 8/9 5/5 3/3 0/4 S T 0/6 8/10 5/5 5/8 A D • Maximum rate we can push flows from S to T in a given capacitied flow network. • flow-rate/link-capacity Zongpeng Li – p.7/28

LP model: max-flow → Maximize χ = f ( TS ) Subject to: → � → → f ( uv ) ≤ C ( uv ) ∀ uv � = TS → → � uv ) = � v ∈ N ( u ) f ( v ∈ N ( u ) f ( vu ) ∀ u → → f ( uv ) ≥ 0 ∀ uv Zongpeng Li – p.8/28

Totally unimodular LPs • Totally unimodular: every square sub-matrix of the coefficient matrix has determinant of 1 or -1. • Totally unimodular LPs always have integral optimal solutions. • The node-arc incidence matrix of a directed network is totally unimodular! Zongpeng Li – p.9/28

LP model: min-cut → � Minimize uv C ( uv ) y ( uv ) → Subject to: → � → → y ( uv ) + p ( v ) ≥ p ( u ) ∀ uv � = TS p ( T ) − p ( S ) ≥ 1 → → y ( uv ) ≥ 0 ∀ uv • Max-cut cannot be modelled as a simple LP; it is NP-hard. • Elegant approximation algorithm of max-cut based on semidefinite programming. Zongpeng Li – p.10/28

LP model: min-cost flow → → � Minimize uv w ( uv ) f ( uv ) → Subject to: →  f ( TS ) = d    → → → f ( uv ) ≤ C ( uv ) ∀ uv � = TS → →   � uv ) = � v ∈ N ( u ) f ( v ∈ N ( u ) f ( vu ) ∀ u  → → f ( uv ) ≥ 0 ∀ uv Zongpeng Li – p.11/28

LP model: shortest path → → � Minimize uv w ( uv ) f ( uv ) → Subject to: → � f ( TS ) = 1 → → � uv ) = � v ∈ N ( u ) f ( v ∈ N ( u ) f ( vu ) ∀ u → → f ( uv ) ≥ 0 ∀ uv Zongpeng Li – p.12/28

LP model: the assignment problem • Assign n objects to n persons, 1-to-1 mapping • Each object o worths v ( i, o ) to each person i • Goal: maximize “total happiness” � � Maximize o f ( i, o ) v ( i, o ) i Subject to: � � o f ( i, o ) = 1 ∀ i � i f ( i, o ) = 1 ∀ o f ( i, o ) ≥ 0 ∀ i, ∀ o • Totally unimodular LP , integral optimal solution • primal-dual algorithm design, the celebrated auction algorithm Zongpeng Li – p.13/28

LP model: max-rate multicast with network coding Given network coding, a multicast rate x is feasible in a directed network iff it is feasible as an independent unicast to every receiver. [Ahlswede et al. IT 2000][Koetter and M ´ e dard TON 2003] S S S a replication point a a T 2 T 1 T 1 T 2 T 2 T 1 S S encoding S b a point a b a+b a b a+b a+b T 1 T 1 T 2 T 2 T T 2 1 Zongpeng Li – p.14/28

LP model: max-rate multicast with network coding Maximize χ Subject to: →  χ ≤ f i ( T i S ) ∀ i (1)    → → → →   f i ( uv ) ≤ c ( uv ) ∀ i, ∀ uv � = (2) T i S → → � uv ) = � v ∈ N ( u ) f i ( v ∈ N ( u ) f i ( vu ) ∀ i, ∀ u (3)    → →  c ( uv ) + c ( vu ) ≤ C ( uv ) ∀ uv � = T i S (4)  → → → c ( uv ) , f i ( uv ) , χ ≥ 0 ∀ i, ∀ uv Zongpeng Li – p.15/28

LP model: max-rate multicast without network coding � Minimize t f ( t ) Subject to: � f ( t ) ≤ c ( e ) ∀ e t : e ∈ t f ( t ) ≥ 0 ∀ t • Don’t be misguided by the seeming simplicity of the LP . • It has exponentially many variables. • We know a network instance with 16 nodes only, having ∼ 50 million different trees. • But, what else can we do? It’s an NP-hard problem. Zongpeng Li – p.16/28

Primal and dual LPs Minimize c 1 x 1 + c 2 x 2 + c 3 x 3 Maximize b 1 y 1 + b 2 y 2 + b 3 y 3 Subject to: Subject to:   a 11 x 1 + a 12 x 2 + a 13 x 3 ≥ b 1 a 11 y 1 + a 21 y 2 + a 31 y 3 ≤ c 1 ↔ y 1 ↔ x 1       a 21 x 1 + a 22 x 2 + a 23 x 3 ≥ b 2 a 12 y 1 + a 22 y 2 + a 32 y 3 ≤ c 2 ↔ y 2 ↔ x 2     a 31 x 1 + a 32 x 2 + a 33 x 3 ≥ b 3 a 13 y 1 + a 23 y 2 + a 33 y 3 ≤ c 3 ↔ y 3 ↔ x 3   x 1 , x 2 , x 3 ≥ 0 y 1 , y 2 , y 3 ≥ 0 • Poor student vs. greedy drug store owner • Student: satisfying vitamin intaking needs with minimal budget • Store owner: maximizing revenue while maintaining competitiveness Zongpeng Li – p.17/28

LP duality • Every feasible solution in the primal (minimization) provides a lower-bound for the dual (maximization) and vice versa. • If the primal is feasible and has optimal solutions, then so does the dual; furthermore, their optimal objective function values must be the same. • Every max-min theorem (that I know of) in graph theory, combinatorial optimization and game theory can be derived as a corollary of the LP duality theorem and/or the matroid union theorem. Zongpeng Li – p.18/28

Complementary slackness • Let x ∗ and y ∗ be a pair of corresponding optimal primal and dual solutions • y ∗ 1 > 0 ⇒ a 11 x ∗ 1 + a 12 x ∗ 2 + a 13 x ∗ 3 = b 1 , and so on • The shadow price is nonzero only if the resource supply is tight • Generalization into nonlinear programming: the Karush-Kuhn-Tucker (KKT) conditions Zongpeng Li – p.19/28

An example application of LP duality and CS Enforcing minimum-cost multicast routing, Li and Williamson, 2007. • Min-cost multicast, flows selfishly route themselves through cheapest paths available • Formulate primal and dual LPs • Use shadow prices to allocate edge costs and set edge taxes • Each optimal flow can be thus enforced; proof of Nash Equilibrium based on CS conditions Zongpeng Li – p.20/28

Solving LPs: the simplex method • Walk along a sequence of vertice, on the polyhedron boundary • with improved objective value at each step • multiple “better neighbors”, which to choose? • The pivot rule Zongpeng Li – p.21/28

Solving LPs: the interior-point method • Walk within the polytope • Each step, walk towards a new feasible solution in the polytope • which had better not be too close to the boundary • being close to the optimum is naturally good • Model the above concerns using barrier functions and potential functions Zongpeng Li – p.22/28

Solving LPs: the ellipsoid method • Solve optimization by solving feasibility, through binary search. • Enclose the feasibility polytope using an ellipsoid • either verify feasibility using a separation oracle • or cut the ellipsoid into two halves and enclose the feasible half using a smaller ellipsoid • Claim infeasibility when the ellipsoid becomes small enough. • Why ellipsoid? Why not a sphere? What about other geometric shapes? Zongpeng Li – p.23/28

Solving LPs: problem specific methods • Tailor the simplex algorithm: the network simplex algorithm • Lagrange relaxation and subgradient optimization ◦ Assume a network flow LP with an extra side constraint ◦ Can relax the side constraint and solve the smaller network flow LP using highly optimized algorithms ◦ Trade-off: need to solve a sequence of these ◦ Can help in distributed protocol design Zongpeng Li – p.24/28

Solving LPs: realworld experiences • 1000 variables/constraints ? — that’s easy • 1 million variables/constraints ? — that’s OK • 1 billion variables/constraints ? — no way • For general LPs: simplex and interior-point algorithms can compete with each other • For LPs with a network background: interior-point algorithms might perform much better (personal experience) • Ellipsoid algorithms are of theoretical interest mostly Zongpeng Li – p.25/28

Solving LPs: software available • GNU glpk , http://www.gnu.org/software/glpk/ ◦ free ◦ simplex, interior-point, branch-and-cut • CPLEX, http://www.ilog.com/products/cplex/ ◦ simplex, interior-point, integer programming, quadratic programming • CVX, http://www.stanford.edu/ boyd/cvx/ ◦ free ◦ Matlab library ◦ solves “disciplined” convex programs Zongpeng Li – p.26/28

Liner integer programming: layered multicast k l k .x i � � Maximize (9) i k Subject to: → →  v ∈ N ( u ) [ f i uv ) − f i � k ( k ( vu )] = 0 ∀ k, ∀ i, ∀ u   → → →  f i k ( uv ) ≤ f k ( uv ) ∀ k, ∀ i, ∀ uv   → → → � k f k ( uv ) ≤ C ( uv ) ∀ uv   →  k ≤ f i k ( T i S )  x i k +1 ≤ x i ∀ k = 1 ..L − 1 , ∀ i  l k → → → uv ) , f i uv ) ≥ 0 , x i f k ( k ( k ∈ { 0 , 1 } ∀ k, ∀ i, ∀ uv Zongpeng Li – p.27/28