Programming for the 0/1 Knapsack Problem Nirmal Prajapati Sanjay - PowerPoint PPT Presentation

Revisiting Sparse Dynamic Programming for the 0/1 Knapsack Problem Nirmal Prajapati Sanjay Rajopadhye Tarequl Islam Sifat Tarequl.Sifat@colostate.edu prajapati@lanl.gov Sanjay.Rajopadhye@colostate.edu Los Alamos National Laboratory Department of Computer Science Corespeq Inc Los Alamos, New Mexico, USA Colorado State University Fort Collins, Colorado, USA Fort Collins, Colorado, USA

0/1 Knapsack Problem Statement ◼ Given a set of 𝑂 items numbered from 1 up to 𝑂 , each with a weight 𝑥 𝑗 and a profit 𝑞 𝑗 , along with maximum capacity 𝐷 , we must 𝑂+1 𝑛𝑏𝑦𝑗𝑛𝑗𝑨𝑓 ෍ 𝑞 𝑗 𝑦 𝑗 𝑗=1 𝑂+1 𝑡𝑣𝑐𝑘𝑓𝑑𝑢 𝑢𝑝 ෍ 𝑥 𝑗 𝑦 𝑗 < 𝐷 𝑏𝑜𝑒 𝑦 𝑗 ∈ {0,1} 𝑗=1 2

Sparse DP Algorithm for solving 0/1- Knapsack Problem ◼ A “sparse” KPDP algorithm (SKPDP) has been known for a while. ◼ Conventional KPDP algorithm generates a DP table that contains many repeated values. ◼ SKPDP does not calculate the repeated values in the DP table. ◼ So far there has been no quantitative analysis of its benefits. 3

Contributions ◼ Quantitative analysis of Sequential SKPDP ◼ Exploration of two parallelization techniques for SKPDP and their performance analysis ◼ Comparison of SKPDP with Branch-and-Bound 4

Problem Instance Generation For Quantitative Analysis ◼ Uncorrelated : 𝑞 𝑗 = 𝑠𝑏𝑜𝑒𝑝𝑛(𝑠𝑏𝑜𝑕𝑓) ◼ Weakly Correlated : 𝑞 𝑗 = 𝛽𝑥 𝑗 + 𝑠𝑏𝑜𝑒𝑝𝑛(𝑠𝑏𝑜𝑕𝑓) ◼ Strongly Correlated: 𝑞 𝑗 = 𝛽𝑥 𝑗 + 𝛾 [Hardest Problem Instances] ◼ Subset Sum: 𝑞 𝑗 = 𝑥 𝑗 where 𝛽, 𝛾 are constants, 𝑞 𝑗 is profit and 𝑥 𝑗 is weight of each item. 5

Punch Line ◼ For KP instances with significantly large capacity than the number of items (C >> N) ◼ If the problem instance is weakly correlated, the operation count of SKPDP algorithm is invariant with respect to capacity ( 𝐷 ). ◼ If the problem instance is strongly correlated, the operation count of SKPDP algorithm is exponentially less than KPDP. 6

Punch Line Weakly Correlated Instances Strongly Correlated Instances 7

Dynamic Programming Solution 𝑁 𝑙 − 1, 𝑑 𝑗𝑔 𝑥 𝑙 > 𝑑 𝑁 𝑙, 𝑑 = ቊ max 𝑁 𝑙 − 1, 𝑑 , 𝑁 𝑙 − 1, 𝑑 − 𝑥 𝑙 + 𝑞 𝑙 𝑓𝑚𝑡𝑓 for ( k=1 ; k < N ; k++ ) { for ( c=0 ; c <= C ; c++ ) { if( c < weights[k] ) M[k,c] = M[k-1,c]; else M[k,c]= MAX (M[k-1,c], M[k-1,c-weights[k]]+profits[k]); } } 8

Example Problem Instance 𝑂 = 5 𝐷 = 11 Item No. Profits Weights 1 1 1 2 6 2 3 18 5 4 22 6 5 28 7 9

Dynamic Programming Table 𝑁 𝑙 − 1, 𝑑 𝑗𝑔 𝑥 𝑙 > 𝑑 𝑁 𝑙, 𝑑 = ቊ max 𝑁 𝑙 − 1, 𝑑 , 𝑁 𝑙 − 1, 𝑑 − 𝑥 𝑙 + 𝑞 𝑙 𝑓𝑚𝑡𝑓 c Capacity Item# (weight,profit) 0 1 2 3 4 5 6 7 8 9 10 11 0 0 0 0 0 0 0 0 0 0 0 0 0 k 1 (1,1) 0 1 1 1 1 1 1 1 1 1 1 1 2 (2,6) 0 1 6 7 7 7 7 7 7 7 7 7 3 (5,18) 0 1 6 7 7 18 19 24 25 25 25 25 4 (6,22) 0 1 6 7 7 18 22 24 28 29 29 40 5 (7,28) 0 1 6 7 7 18 22 28 29 34 35 40 10

Memory Efficient KPDP ◼ Only need the current row of the DP table to calculate the next row ◼ The whole table does not have to be stored ◼ This way we can find the optimal profit value ◼ We can find the exact solution including which items are taken in the optimal solution by using a divide-and-conquer strategy ◼ Divide-and-Conquer strategy doubles the number of computations, 2𝑂𝐷 ◼ Reduces memory requirement by a factor of 𝑂/2 11

“Sparsity” in the current context 12

“Sparsity” in the current context Capacity Item# (weight,profit) 0 1 2 3 4 5 6 7 8 9 10 11 0 0 0 0 0 0 0 0 0 0 0 0 0 1 (1,1) 0 1 1 1 1 1 1 1 1 1 1 1 2 (2,6) 0 1 6 7 7 7 7 7 7 7 7 7 3 (5,18) 0 1 6 7 7 18 19 24 25 25 25 25 4 (6,22) 0 1 6 7 7 18 22 24 28 29 29 40 5 (7,28) 0 1 6 7 7 18 22 28 29 34 35 40 Item# 0 <0,0> 1 (1,1) <0,0>,<1,1> 2 (2,6) <0,0>,<1,1>,<2,6>,<3,7> 3 (5,18) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> 4 (6,22) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,24>,<8,28>,<9,29>,<11,40> 5 (7,28) <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,22>,<7,28>,<8,29>,<9,34>,<10,35>,<11,40> 13

Building the Sparse Table Add-Merge-Kill When we include the 4 th item (Weight: 6, Profit: 22) in our choice <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40> {1,2,3,4} 14

Building the Sparse Table Add-Merge-Kill When we include the 4 th item (Weight: 6, Profit: 22) in our choice <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40> {1,2,3,4} <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> 14

Building the Sparse Table Add-Merge-Kill When we include the 4 th item (Weight: 6, Profit: 22) in our choice <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40> {1,2,3,4} <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> Add (6,22) to each pair, <6,22>,<7,23>,<8,28>,<9,29>,<11,40>,<12,41>,<13,46>,<14,47> 14

Building the Sparse Table Add-Merge-Kill When we include the 4 th item (Weight: 6, Profit: 22) in our choice <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40> {1,2,3,4} <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> Add (6,22) to each pair, <6,22>,<7,23>,<8,28>,<9,29>,<11,40> 14

Building the Sparse Table Add-Merge-Kill When we include the 4 th item (Weight: 6, Profit: 22) in our choice <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40> {1,2,3,4} <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> <6,22>,<7,23>,<8,28>,<9,29>,<11,40> 14

Building the Sparse Table Add-Merge-Kill When we include the 4 th item (Weight: 6, Profit: 22) in our choice <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,19>, <7,24>, <8,25> {1,2,3} <0,0>, <1,1>, <2,6>, <3,7>, <5,18>, <6,22>, <7,24>, <8,28>, <9,29>, <11,40> {1,2,3,4} <0,0>,<1,1>,<2,6>,<3,7>,<5,18>,<6,19>,<7,24>,<8,25> <6,22>,<7,23>,<8,28>,<9,29>,<11,40> 14

Building the Sparse Table Add-Merge-Kill 15

Generation of Problem Instances ◼ The fraction of objects that can fit in the knapsack on average, 1/𝜇 𝜇𝐷 𝑋 𝑏𝑤𝑕 = 𝑂 ◼ The set of weights, 𝑥 𝑗 is generated with a normal distribution that has a mean of 𝑋 𝑏𝑤𝑕 ◼ For weakly correlated problem instances, the correlation between the weights and profits is controlled by a noise factor, 𝜏 𝑞 𝑗 = 𝛽𝑥 𝑗 + 𝑆𝑏𝑜𝑒𝑝𝑛 𝐽𝑜𝑢𝑓𝑕𝑓𝑠 𝑐𝑓𝑢𝑥𝑓𝑓𝑜 [−𝜏𝑋 𝑏𝑤𝑕 , 𝜏𝑋 𝑏𝑤𝑕 ] ◼ For strongly correlated problem instances 𝜏 is irrelevant, 𝑞 𝑗 = 𝛽𝑥 𝑗 + 𝛾 16

Gain 𝐽𝑢𝑓𝑠𝑏𝑢𝑗𝑝𝑜𝑡 𝑗𝑜 𝑇𝐿𝑄𝐸𝑄 𝐻𝑏𝑗𝑜 = 1 − 𝐽𝑢𝑓𝑠𝑏𝑢𝑗𝑝𝑜𝑡 𝑗𝑜 𝐿𝑄𝐸𝑄 (= 2𝑂𝐷) The range of Gain is (-1, 1) ◼ A value of gain close to 1 means that the number of iterations in SKPDP is insignificant compared to KPDP ◼ A value of gain close to -1 mean that we have the worst-case scenario for SKPDP. 17

Gain 𝜇 = 2, 𝜏 = 0.1% 18

SKPDP vs KPDP Weakly Correlated Instances Strongly Correlated Instances 𝑂 = 256, 𝜇 = 8, 𝜏 = 0.1% 𝜇 = 2, 𝜏 = 0.1% 19

Impact of 𝜏 on the sparsity 𝑏𝑤𝑕 ; 𝜇 = 2 𝑞 = 𝛽𝑥 + 𝑆𝑏𝑜𝑒𝑝𝑛 𝐽𝑜𝑢. 𝐽𝑜 −𝜏𝑋 𝑏𝑤𝑕 , 𝜏𝑋 20

Impact of 𝜇 on the sparsity 𝑏𝑤𝑕 = 𝜇𝐷 𝑋 𝑂 ; 𝜏 = 0.1% 21

Impact of 𝜏 and 𝜇 on the sparsity 2 12 < C < 2 50 𝑂 = 2 10 , 22

Parallelization of SKPDP ◼ Fine-Grained Parallelization ◼ Coarse-Grained Parallelization 23