Cutting Planes and Branch and Bound Marco Chiarandini Department - PowerPoint PPT Presentation

DM545 Linear and Integer Programming Lecture 13 Cutting Planes and Branch and Bound Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Cutting Plane Algorithms Outline Branch and Bound 1. Cutting Plane Algorithms 2. Branch and Bound 2

Cutting Plane Algorithms Outline Branch and Bound 1. Cutting Plane Algorithms 2. Branch and Bound 3

Cutting Plane Algorithms Valid Inequalities Branch and Bound • IP: z = max { c T x : x ∈ X } , X = { x : A x ≤ b , x ∈ Z n + } • Proposition: conv ( X ) = { x : ˜ A x ≤ ˜ b , x ≥ 0 } is a polyhedron • LP: z = max { c T x : ˜ A x ≤ ˜ b , x ≥ 0 } would be the best formulation • Key idea: try to approximate the best formulation. Definition (Valid inequalities) ax ≤ b is a valid inequality for X ⊆ R n if ax ≤ b ∀ x ∈ X Which are useful inequalities? and how can we find them? How can we use them? 4

Cutting Plane Algorithms Example: Pre-processing Branch and Bound • X = { ( x , y ) : x ≤ 999 y ; 0 ≤ x ≤ 5 , y ∈ B 1 } x ≤ 5 y • X = { x ∈ Z n + : 13 x 1 + 20 x 2 + 11 x 3 + 6 x 4 ≥ 72 } 2 x 1 + 2 x 2 + x 3 + x 4 ≥ 13 11 x 1 + 20 11 x 2 + x 3 + 6 11 x 4 ≥ 72 11 = 6 + 6 11 2 x 1 + 2 x 2 + x 3 + x 4 ≥ 7 • Capacitated facility location: � x ij ≤ b j y j ∀ j ∈ N x ij ≤ b j y j i ∈ M � x ij = a i ∀ i ∈ M x ij ≤ a i j ∈ N x ij ≥ 0 , y j ∈ B n x ij ≤ min { a i , b j } y j 5

Cutting Plane Algorithms Chvátal-Gomory cuts Branch and Bound • X ∈ P ∩ Z n P = { x ∈ R n + : A x ≤ b } , A ∈ R m × n + , • u ∈ R m + , { a 1 , a 2 , . . . a n } columns of A CG procedure to construct valid inequalities n � 1 ) ua j x j ≤ ub valid: u ≥ 0 j = 1 n � � � 2 ) ⌊ ua j ⌋ x j ≤ ub valid: x ≥ 0 and ⌊ ua j ⌋ x j ≤ ua j x j j = 1 n � valid for X since x ∈ Z n 3 ) ⌊ ua j ⌋ x j ≤ ⌊ ub ⌋ j = 1 Theorem by applying this CG procedure a finite number of times every valid inequality for X can be obtained 6

Cutting Plane Algorithms Cutting Plane Algorithms Branch and Bound • X ∈ P ∩ Z n + • a family of valid inequalities F : a T x ≤ b , ( a , b ) ∈ F for X • we do not find them all a priori, only interested in those close to optimum Cutting Plane Algorithm Init.: t = 0 , P 0 = P z t = max { c T x : x ∈ P t } Iter. t : Solve ¯ let x t be an optimal solution if x t ∈ Z n stop, x t is opt to the IP if x t �∈ Z n solve separation problem for x t and F if ( a t , b t ) is found with a t x t > b t that cuts off x t P t + 1 = P ∩ { x : a i x ≤ b i , i = 1 , . . . , t } else stop ( P t is in any case an improved formulation) 7

Cutting Plane Algorithms Gomory’s fractional cutting plane algorithm Branch and Bound Cutting plane algorithm + Chvátal-Gomory cuts • max { c T x : A x = b , x ≥ 0 , x ∈ Z n } • Solve LPR to optimality   x u = ¯ b u − � ¯ a uj x j , u ∈ B ¯ ¯ A N = A − 1 0 I B A N b   j ∈ N   z = ¯ d + � c j x j ¯   − ¯ j ∈ N c B ¯ c N ( ≤ 0 ) ¯ 1 d • If basic optimal solution to LPR is not integer then ∃ some row u : ¯ b u �∈ Z 1 . The Chvatál-Gomory cut applied to this row is: � a uj ⌋ x j ≤ ⌊ ¯ ⌊ ¯ b u ⌋ x B u + j ∈ N ( B u is the index in the basis B corresponding to the row u ) (cntd) 8

Cutting Plane Algorithms Branch and Bound • Eliminating x B u = ¯ b u − � ¯ a uj x j in the CG cut we obtain: j ∈ N � ) x j ≥ ¯ b u − ⌊ ¯ (¯ a uj − ⌊ ¯ a uj ⌋ b u ⌋ � �� � � �� � j ∈ N 0 ≤ f uj < 1 0 < f u < 1 � f uj x j ≥ f u j ∈ N f u > 0 or else u would not be row of fractional solution. It implies that x ∗ in which x ∗ N = 0 is cut out! • Moreover: when x is integer, since all coefficient in the CG cut are integer the slack variable of the cut is also integer: � s = − f u + f uj x j j ∈ N (theoretically it terminates after a finite number of iterations, but in practice not successful.) 9

Cutting Plane Algorithms Example Branch and Bound max x 1 + 4 x 2 x 2 x 1 + 6 x 2 ≤ 18 ≤ 3 x 1 x 1 = 3 x 1 , x 2 ≥ 0 x 1 , x 2 integer x 1 + 6 x 2 = 18 x 1 x 1 + 4 x 2 = 2 | | x1 | x2 | x3 | x4 | -z | b | |---+----+----+----+----+----+----| | | 1 | 6 | 1 | 0 | 0 | 18 | | | 1 | 0 | 0 | 1 | 0 | 3 | |---+----+----+----+----+----+----| | | 1 | 4 | 0 | 0 | 1 | 0 | | | x1 | x2 | x3 | x4 | -z | b | |---+----+----+------+------+----+------| | | 0 | 1 | 1/6 | -1/6 | 0 | 15/6 | | | 1 | 0 | 0 | 1 | 0 | 3 | x 2 = 5 / 2 , x 1 = 3 |---+----+----+------+------+----+------| Optimum, not integer | | 0 | 0 | -2/3 | -1/3 | 1 | -13 | 10

Cutting Plane Algorithms Branch and Bound • We take the first row: | | 0 | 1 | 1/6 | -1/6 | 0 | 15/6 | • CG cut � j ∈ N f uj x j ≥ f u � 1 6 x 3 + 5 6 x 4 ≥ 1 2 • Let’s see that it leaves out x ∗ : from the CG proof: 1 / 6 ( x 1 + 6 x 2 ≤ 18 ) 5 / 6 ( x 1 ≤ 3 ) x 1 + x 2 ≤ 3 + 5 / 2 = 5 . 5 since x 1 , x 2 are integer x 1 + x 2 ≤ 5 • Let’s see how it looks in the space of the original variables: from the first tableau: x 3 = 18 − 6 x 2 − x 1 x 4 = 3 − x 1 1 6 ( 18 − 6 x 2 − x 1 ) + 5 6 ( 3 − x 1 ) ≥ 1 x 1 + x 2 ≤ 5 � 2 11

Cutting Plane Algorithms Branch and Bound • Graphically: x 2 x 1 = 3 x 1 + 6 x 2 = 18 x 1 + x 2 = 5 x 1 x 1 + 4 x 2 = 2 • Let’s continue: We need to apply dual-simplex | | x1 | x2 | x3 | x4 | x5 | -z | b | (will always be the case, why?) |---+----+----+------+------+----+----+------| | | 0 | 0 | -1/6 | -5/6 | 1 | 0 | -1/2 | c j | | 0 | 1 | 1/6 | -1/6 | 0 | 0 | 5/2 | ratio rule: min {| a ij | : a ij < 0 } | | 1 | 0 | 0 | 1 | 0 | 0 | 3 | |---+----+----+------+------+----+----+------| | | 0 | 0 | -2/3 | -1/3 | 0 | 1 | -13 | 12

Cutting Plane Algorithms Branch and Bound • After the dual simplex iteration: We can choose any of the three | | x1 | x2 | x3 | x4 | x5 | -z | b | rows. |---+----+----+------+----+------+----+-------| | | 0 | 0 | 1/5 | 1 | -6/5 | 0 | 3/5 | | | 0 | 1 | 1/5 | 0 | -1/5 | 0 | 13/5 | Let’s take the third: CG cut: | | 1 | 0 | -1/5 | 0 | 6/5 | 0 | 12/5 | 5 x 3 + 1 4 5 x 5 ≥ 2 |---+----+----+------+----+------+----+-------| 5 | | 0 | 0 | -3/5 | 0 | -2/5 | 1 | -64/5 | • In the space of the original variables: x 2 4 ( 18 − x 1 − 6 x 2 ) + ( 5 − x 1 − x 2 ) ≥ 2 x 1 + 5 x 2 ≤ 15 x 1 • ... 13

Cutting Plane Algorithms Outline Branch and Bound 1. Cutting Plane Algorithms 2. Branch and Bound 14

Cutting Plane Algorithms Branch and Bound Branch and Bound • Consider the problem z = max { c T x : x ∈ S } • Divide and conquer: let S = S 1 ∪ . . . ∪ S k be a decomposition of S into smaller sets, and let z k = max { c T x : x ∈ S k } for k = 1 , . . . , K . Then z = max k z k For instance if S ⊆ { 0 , 1 } 3 the enumeration tree is: S x 1 = 0 x 1 = 1 S 0 S 1 x 2 = 0 S 00 S 01 S 10 S 11 x 3 = 0 S 000 S 001 S 010 S 011 S 100 S 101 S 110 S 111 15

Cutting Plane Algorithms Bounding Branch and Bound Let’s consider a maximization problem (gurobi’s default is minimization) • Let z k be an upper bound on z k (dual bound) • Let z k be an lower bound on z k (primal bound) • ( z k ≤ z k ≤ z k ) • z = max k z k is a lower bound on z • z = max k z k is an upper bound on z 16

Cutting Plane Algorithms Pruning Branch and Bound 27 z = 25 13 z = 20 pruned by optimality 20 25 20 15 27 z = 26 13 z = 21 pruned by bounding 20 26 18 21 40 z = 37 −∞ z = 13 nothing to prune 24 37 13 −∞ 17

Cutting Plane Algorithms Pruning Branch and Bound 27 z = 26 13 z = 14 pruned by infeasibility 26 infeas. 14 18

Cutting Plane Algorithms Example Branch and Bound max x 1 + 2 x 2 x 2 x 1 + 4 x 2 ≤ 8 4 x 1 + x 2 ≤ 8 x 1 , x 2 ≥ 0 , integer x 1 + 4 x 2 = 8 x 1 x 1 + 2 x 2 = 1 4 x 1 + x 2 = 8 • Solve LP | | x1 | x2 | x3 | x4 | -z | b | |---+----+----+----+----+----+---| | | 1 | 4 | 1 | 0 | 0 | 8 | | | 4 | 1 | 0 | 1 | 0 | 8 | |---+----+----+----+----+----+---| | | 1 | 2 | 0 | 0 | 1 | 0 | | | x1 | x2 | x3 | x4 | -z | b | |--------------+----+------+----+------+----+----| | I’=I-II’ | 0 | 15/4 | 1 | -1/4 | 0 | 6 | | II’=1/4II | 1 | 1/4 | 0 | 1/4 | 0 | 2 | |--------------+----+------+----+------+----+----| | III’=III-II’ | 0 | 7/4 | 0 | -1/4 | 0 | -2 | 19

Cutting Plane Algorithms Branch and Bound • continuing x 2 = 1 + 3 / 5 = 1 . 6 | | x1 | x2 | x3 | x4 | -z | b | x 1 = 8 / 5 |----------------+----+----+-------+-------+----+---------| The optimal solution | I’=4/15I | 0 | 1 | 4/15 | -1/15 | 0 | 24/15 | | II’=II-1/4I’ | 1 | 0 | -1/15 | 4/15 | 0 | 24/15 | will not be more than |----------------+----+----+-------+-------+----+---------| 2 + 14 / 5 = 4 . 8 | III’=III-7/4I’ | 0 | 0 | -7/15 | -3/5 | 1 | -2-14/5 | • Both variables are fractional, we pick one of the two: x 2 4 . 8 x 1 = 1 x 1 ≤ 1 x 1 ≥ 2 x 1 + 4 x 2 = 8 x 1 x 1 + 2 x 2 = 1 4 x 1 + x 2 = 8 20