Column Generation By Soumitra Pal Under the guidance of Prof. A. - PowerPoint PPT Presentation

Column Generation By Soumitra Pal Under the guidance of Prof. A. G. Ranade

Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

Cutting Stock Problem 10 • Given larger raw paper rolls • Get final rolls of smaller widths 5 3

Cutting Stock Problem (2) • Raw width (W) = 10 • No of finals given below i Width (w i ) Quantity (b i ) 1 3 9 2 5 79 3 6 90 4 9 27 • Minimize total no of raws reqd. to be cut

Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

Objective/ Min -x 1 -x 2 Cost fn 5x 1 - 8x 2 ≥ -80 -5x 1 - 4x 2 ≥ -100 Constraints -5x 1 - 2x 2 ≥ -80 -5x 1 + 2x 2 ≥ -50

5x 1 - 8x 2 ≥ -80 -5x 1 - 4x 2 ≥ -100 -5x 1 - 2x 2 ≥ -80 -5x 1 + 2x 2 ≥ -50 (8,15) (0,10) (12,10) (13,5) (10,0)

Cost decreases in this direction -x1-x2 = -10 -x1-x2 = -20 -x1-x2 = -30

Simplex method (8,15) (0,10) (12,10) (13,5) (10,0)

+ + + L Min c x c x c x 1 1 2 2 n l + + + ≥ L s.t. : a x a x a x b 11 1 12 2 1 1 n l + + + ≥ L a x a x a x b 21 1 22 2 2 2 n l M + + + ≥ L a x a x a x b 1 1 2 1 m m ml l m ≥ 0 x i no. of variables l no. of contraints m

+ + + + + + L L Min 0 0 0 c x c x c x x x x + + 1 1 2 2 1 2 n l l l n + + + − = L s.t. : a x a x a x x b + 11 1 12 2 1 1 1 n l l + + + − = L a x a x a x x b + 21 1 22 2 2 2 2 n l l M L + + + − = L a x a x a x x b 1 1 2 1 m m ml l n m ≥ 0 x i

Min cx = s.t. : Ax b ≥ 0 x

Basic & non-basic 5x 1 - 8x 2 ≥ -80 -5x 1 - 4x 2 ≥ -100 -5x 1 - 2x 2 ≥ -80 -5x 1 + 2x 2 ≥ -50 (8,15,0,0,10,40) (0,10,0, 60,60,70) (12,10,60,0,0,10) (13,5,110,10,0,0) (10,0,130,50,30,0) (0,0,80, 100,80,50)

⎡ ⎤ x [ ] = B min ⎢ ⎥ cx c c B N ⎣ ⎦ 0 ⎡ ⎤ x [ ] = = ≥ B , 0 ⎢ ⎥ Ax B N b x B ⎣ ⎦ 0 => = Bx b B − => = 1 x B b B => = = − 1 cx c x c B b B B B [ ] ⎡ ⎤ x − = − 1 B 1 ⎢ ⎥ I B N B b ⎣ ⎦ 0 To find out next corner, increase one non - basic variable ⎡ ⎤ ' x = ' B ⎢ ⎥ x ' ⎣ ⎦ x N Putting it in previous equation, + − = − ' 1 ' 1 x B Nx B b B N − − => = − ' 1 1 ' x B b B Nx B N − => = − ' 1 ' x x B Nx B B N

New cost − = + = − + ' ' ' 1 ' ' ( ) cx c x c x c x B Nx c x B B N N B B N N N − => = + = + − ' ' ' 1 ' ( ) cx c x c x c x c c B N x B B N N B B N B N Change in cost − = − − ' 1 ' ( ) cx cx c c B N x N B N The term in bracket is called reduced cost − = − 1 c c c B N N B • If all elements of it are non-negative, it is already optimal • Otherwise choose the non-basic variable corresponding to most negative value to enter the basic

• How to find out outgoing basic variable? − − + = ' 1 ' 1 x B Nx B b B N − 1 Let be the column of that d B N mutilpies with new basic variable . It can be increased until one x i th ' (say k ) component of becomes 0. x B Min value of − = 1 kth component of ( ) B b x B kth component of d

Few steps of simplex algorithm − − 1 • Compute reduced cost c c B N N B • If all >= 0, optimal stop. • Otherwise choose the most negative component • Corresponding variable enters basis • Compute the ratios for finding out leaving basis • Update basic variables and B for next step − = − ' 1 ' x x B Nx B B N

Smart way of doing = gives Bx b x B B = − gives and reduced cost yB c y c yN B N = where is the column of , gives Bd a a N d x B gives info about leaving variable d

Start d y

Reduced cost vector d y

Selection of non-basic variable d y

y vector d y

Selection of leaving basic variable d y

Preparation for next step d y

Entering variable in 2 nd iteration d y

Leaving variable in 2 nd iteration d y

Preparation for 3 rd iteration d y

Final iteration d y

Done so far • Basics of (revised) Simplex algorithm – Formulation – Corners, basic variables, non-basic variables – How to move from corner to corners – Optimal value

Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

First step to solution - Formulation K ∑ Min y k = 1 k K ∑ ≥ = K s.t. : 1 , , x b i m ik i = 1 k m ∑ ≤ = K 1 , , w x Wy k K i ik k = 1 i = = K 0 or 1 1 , , y k K k ≥ 0 and integer x = = K K 1 , , 1 , , k K i m ik K no. of available raws m no. of different widths k index for a raw i index for a width th 1 if k raw is used, y k 0 otherwise Kantorovich formulation th no. of finals of i width x th ik cut from k raw

Kantorovich formulation example 1 2 3 4 = = = = = 4 , 1 , 0 K y y y y 1 3 2 4 = = = 1 , 1 , 3 x x x 11 21 13 ∑ ∑ = = 4 , 1 x x 1 2 k k k k

LP relaxation • Integer programming is hard • Convert it to a linear program K ∑ Min y k = 1 k K ∑ ≥ = K s.t. : 1 , , x b i m ik i = k 1 m ∑ ≤ = K 1 , , w x Wy k K 0 ≤ y k ≤ 1 i ik k = 1 i = = K 0 or 1 1 , , y k K k ≥ 0 and integer x = = 1 , K , 1 , K , k K i m ik K no. of available raws m no. of different widths k index for a raw i index for a width th 1 if k raw is used, y k 0 otherwise th no. of finals of i width x th ik cut from k raw

LP relaxation (2) • LP relaxation is poor • For our example, optimal LP objective is 120.5 • The optimal integer objective solution value is 157 • Gap is 36.5 • It can be as bad as ½ of the integer solution • Can we get a better LP relaxation?

Another Formulation ∑ Min x j ∈ j J ∑ ≥ = K s.t. : 1 , , a x b i m ij j i ∈ j J ≥ ∀ ∈ 0 and integer x j J j i index for a width m no. of different widths j a cutting pattern Gilmore-Gomory J set of all possible patterns Formulation no. of raws cut in pattern j x j th no. of finals of i width a cut from pattern j ij

Example formulation ⎡ ⎤ x 1 ⎢ ⎥ x ⎢ ⎥ 2 ⎢ ⎥ x = 3 ⎡ ⎤ ⎡ ⎤ ⎢ ⎥ 3 : 0 0 1 0 0 1 1 2 3 9 w 1 ⎢ ⎥ ⎢ ⎥ x ⎢ ⎥ = 4 5 : 0 0 0 1 2 1 0 0 0 79 w ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ≥ 2 x ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ 5 6 : 0 1 1 0 0 0 0 0 0 90 w 3 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ x = 6 9 : ⎣ 1 0 0 0 0 0 0 0 0 ⎦ ⎣ 27 ⎦ w ⎢ ⎥ 4 x ⎢ ⎥ 7 ⎢ ⎥ x 3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10 8 ⎢ ⎥ ⎣ ⎦ x 9

LP relaxation of Gilmore-Gomory • LP relaxation is better • For our example, 156.7 • Integer objective solution, 157 • Gap is 0.3 • Conjecture: Gap is less than 2 for practical cutting stock problems

Issues • The number of columns are exponential • Optimal value is still not integer • Gilmore-Gomory proposed an ingenuous way of working with less columns • Branch-and-price to solve the other problem

Done so far • Basics of (revised) Simplex algorithm – Formulation – Corners, basic variables, non-basic variables – How to move from corner to corners – Optimal value • Formulation of the CSP – LP relaxation – Bounds

Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

Column Generation • In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis • This decision is made from the reduced − − − cost vector 1 or c c B N c yN N B N • The component with most negative value determines the entering variable

Column Generation (2) • In terms of columns, we need to find j − arg min { | is a column of } c ya a N j j j • That is equivalent to finding a such that − min{ ( ) | is a column of } c a ya a A • For cutting stock problem, c j =1, hence it is equivalent to > max{ | 1 } ya ya • With implicit constraint wa ≤ W • Pricing sub-problem

Column Generation (3) Solve Restricted Master Problem (RMP) Update RMP with Solve New Columns Pricing Problem Any Yes New Columns? No STOP (LP Optimal)