Greedy Algorithms Chapter 16 1 Optimization Problems A class of - PowerPoint PPT Presentation

Greedy Algorithms Chapter 16 1

Optimization Problems A class of problems in which we are asked to find a set (or a sequence ) of “ items ” that satisfy some constraints and simultaneously optimize (i.e., maximize or minimize ) some objective function 2

Example: Bin Packing 10 2 5 6 4 3 3

Example: Bin Packing 8 5 6 4 3 2 4

Example: Bin Packing 4 5 6 4 3 2 5

Example: Bin Packing 1 3 5 6 4 2 6

Example: Bin Packing 5 1 3 6 4 5 2 7

Example: Bin Packing 5 4 1 3 4 6 5 2 Number of bins = 3 8

Example: Bin Packing 2 4 3 6 5 (Optimal Solution) Number of bins = 2 9

The Greedy Method Applied to optimization problems Adds items to the solution one-by-one Builds up towards the final solution No backtracking Not necessarily optimal! 10

Activity Selection a.k.a. Task Scheduling 11

Activity Selection Problem Given a set of activities 𝑇 = {𝑏 1 , 𝑏 2 , … , 𝑏 𝑜 } where each activity 𝑗 has a start time 𝑡 𝑗 and a finish time 𝑔 𝑗 , where 0 ≤ 𝑡 𝑗 < 𝑔 𝑗 < ∞ . An activity 𝑏 𝑗 happens in the half-open time interval [𝑡 𝑗 , 𝑔 𝑗 ) . Two activities are said to be compatible if they do not overlap . The problem is to find a maximum-size compatible subset , i.e., a one with the maximum number of activities. 12

Example of Activity Selection 13

A Solution 14

A Better Solution 15

An Optimal (Best) Solution 16

Another Optimal Solution 17

“Greedy” Strategies Longest first Shortest first Early start first Early finish first … 18

Early Finish Greedy Strategy 1. Sort activities by finish time 2. Schedule the first activity 3. Remove all incompatible activities 4. If there are more activities, repeat 2 19

Early Finish 20

Early Finish 21

Early Finish 22

Early Finish 23

Early Finish 24

Early Finish 25

Early Finish 26

Early Finish 27

Optimality of the Greedy Choice To prove optimality of the greedy choice, we have to prove the following two properties 1. Greedy Choice: The greedy choice is part of the answer 2. Optimal Substructure: The optimal solution to the big problem contains the optimal solution to the sub-problem 28

Greedy Choice Let 𝐵 ⊆ 𝑇 be an optimal solution. Let 𝑏 𝑘 be the first element in 𝐵 and 𝑏 𝑛 be the first element in 𝑇 . We want to prove that 𝑏 𝑛 is part of an optimal solution. If 𝑏 𝑛 = 𝑏 𝑘 then we are done Otherwise, we prove that there is another optimal solution 𝐵` = 𝐵 − 𝑏 𝑘 ∪ 𝑏 𝑛 Is 𝐵` a solution? yes Is 𝐵` optimal? yes 29

Optimal Substructure We want to prove that, if 𝐵 ⊆ 𝑇 is an optimal solution to 𝑇 , then 𝐵 − 𝑏 𝑛 is an optimal solution to S` = 𝑏 𝑗 : 𝑏 𝑗 ∈ 𝑇 ∧ 𝑡 𝑗 > 𝑔 𝑛 Proof by contradiction Assume that 𝐵 − 𝑏 𝑛 is not optimal Then, there is another solution 𝐶 to 𝑇` such that 𝐶 > 𝐵 − 𝑏 𝑛 ⇒ |𝐶| ≥ |𝐵| If this is the case, then the subset 𝐵` = 𝑏 𝑛 ∪ 𝐶 is also a solution to 𝑇 𝐵` > 𝐶 ⇒ |𝐵`| > 𝐵 , which means that 𝐵 is not optimal which is a contradiction 30

Knapsack Problem 31

0-1 Knapsack Problem $4,500  15LBs $1,500  3LBs $800  2LBs $3,000 $4,000  45LBs  10LBs  20LBs 32

0-1 Knapsack Problem $4,500  15LBs $1,500  3LBs $800  2LBs $3,000 $4,000  45LBs  10LBs  20LBs 33

0-1 Knapsack Problem $4,500  15LBs $300/LB $1,500  3LBs $800 $500/LB  2LBs $400/LB $3,000 $4,000  45LBs  10LBs  20LBs $300/LB $200/LB 34

0-1 Knapsack Problem $4,500  15LBs $300/LB $1,500  3LBs $800 $500/LB  2LBs $400/LB $3,000 $4,000  45LBs  10LBs  20LBs $300/LB $200/LB Total value = $9,800 35

0-1 Knapsack Problem $4,500  15LBs $300/LB $1,500  3LBs There isn’t a known solution to $800 the 0-1 Knapsack problem using $500/LB  2LBs a greedy algorithm $400/LB $3,000 $4,000  45LBs  10LBs  20LBs $300/LB $200/LB Total value = $11,500 36

Fractional Knapsack Problem $4,500  15 oz $300/oz $1,500  3 oz $800 $500/oz  2 oz $400/oz $3,000 $4,000  45 oz  10oz  20oz $300/oz $200/oz 37

Problem Formulation Given a set 𝑇[1. . 𝑜] of items where each item 𝑗 has a weight 𝑥 𝑗 and a value 𝑤 𝑗 , we would like to find the amount 𝑦 𝑗 of each item that we can take to maximize the total value 𝑜 𝑦 𝑗 𝑊 = ෍ 𝑤 𝑗 𝑥 𝑗 𝑗=1 Under the following two constraints 0 ≤ 𝑦 𝑗 ≤ 𝑥 𝑗 𝑜 σ 𝑗=1 𝑦 𝑗 ≤ 𝑋 38

Pseudo-code Fractional-Knapsack 𝑤 𝑗 Compute the value-per-weight 𝑧 𝑗 = 𝑥 𝑗 for all items Sort items by 𝑧 𝑗 Set 𝑀 = 0 While ( 𝑀 < 𝑋 ) Select the item 𝑘 with the highest 𝑧 𝑘 Set 𝑦 𝑘 = 𝑁𝑗𝑜 𝑋 − 𝑀, 𝑥 𝑘 Remove item 𝑘 Set 𝑀 = 𝑀 + 𝑦 𝑘 Set all remaining 𝑦 𝑗 ’s to 0 39

Greedy-choice Property Given the set 𝑇 ordered by the value-per- weight ( 𝑧 ), taking as much as possible 𝑦 𝑘 from the item 𝑘 with the highest value-per- weight will lead to an optimal solution Assume there is an optimal solution 𝑌′ where we take less amount of item 𝑘 , say 𝑦 𝑘 ′ < 𝑦 𝑘 . We prove that there is another solution 𝑌′′ where we take 𝑦 𝑘 of item 𝑘 and get a similar or a higher total value 𝑊 40

Greedy-choice Property Since 𝑦 𝑘 ′ < 𝑦 𝑘 , there must be another item 𝑙 which was taken in an amount that accounts for the difference, i.e., 𝑦′ 𝑙 We create another solution 𝑌′′ by doing the following changes in 𝑌′ Reduce the amount of item 𝑙 by a value 𝑨 ′′ = 𝑦 𝑙 ′ − 𝑨 𝑦 𝑙 Increase the amount of item 𝑘 by a value 𝑨 ′′ = 𝑦 𝑘 ′ + 𝑨 = 𝑦 𝑘 ( 𝑦 𝑘 is the greedy choice) 𝑦 𝑘 41

Greedy Choice Property ′ = σ 𝑦 𝑗 ′′ ⇒ (both are valid solutions) σ 𝑦 𝑗 𝑊 ′ = σ 𝑦 𝑗 ′ 𝑧 𝑗 , 𝑊 ′′ = σ 𝑦 𝑗 ′′ 𝑧 𝑗 ′′ + 𝑧 𝑙 𝑦 𝑙 ′ + 𝑧 𝑙 𝑦 𝑙 ′′ − 𝑧 𝑘 𝑦 𝑘 𝑊 ′′ − 𝑊 ′ = 𝑧 𝑘 𝑦 𝑘 ′ (All other items are the same) ′′ − 𝑦 𝑘 ′ − 𝑧 𝑙 𝑦 𝑙 ′ − 𝑦 𝑙 𝑊 ′′ − 𝑊 ′ = 𝑧 𝑘 𝑦 𝑘 ′′ ′′ − 𝑦 𝑘 ′ = 𝑦 𝑙 ′ − 𝑦 𝑙 ′′ = 𝑨 But, x j 𝑊 ′′ − 𝑊 ′ = 𝑧 𝑘 − 𝑧 𝑙 𝑨 ′′ − 𝑦 𝑘 > 0 Since 𝑧 𝑘 ≥ 𝑧 𝑙 and 𝑨 = 𝑦 𝑘 𝑊 ′′ ≥ 𝑊 ′ , hence, 𝑌 ′′ is also optimal 42

Greedy Choice Illustration Ordered by 𝑧 𝑘 𝑥 𝑇 𝑘 Input Optimal answer that does not have 𝑘 𝑌′ ′ the greedy choice ′ 𝑦 𝑘 𝑦 𝑙 Optimal answer 𝑘 𝑘 𝑌′′ that has the ′′ 𝑦 𝑙 greedy choice ′′ = 𝑦 𝑘 43 𝑦 𝑘

Optimal Sub-structure Given the problem 𝑇 with an optimal solution 𝑌 , we want to prove that the solution 𝑌` = 𝑌 − 𝑦 𝑘 is optimal to the problem 𝑇` after removing the item 𝑘 and updating the capacity 𝑋` = 𝑋 − 𝑦 𝑘 Proof by contradiction Assume that 𝑌` is not optimal to 𝑇` There is another solution 𝑌`` to 𝑇` that has a higher total value 𝑊`` > 𝑊` This means we can have a better solution to the problem 𝑇 which is impossible because 𝑌 is optimal 44

Optimal substructure Ordered by 𝑧 𝑘 𝑥 𝑇 𝑘 Input 𝑌′ 𝑘 𝑌 Should be an optimal answer for 𝑇′ Optimal answer for 𝑇 𝑇′ Reduced problem 45

Optimal Substructure Input problem 𝑇 with an optimal answer 𝑌 of total value 𝑊 𝑌` = 𝑌 − 𝑦 𝑘 , 𝑊` = 𝑊 − 𝑦 𝑘 𝑧 𝑘 Assume 𝑌`` is an optimal solution to 𝑇` 𝑋`` = σ𝑦`` 𝑗 ≤ 𝑋 − 𝑦 𝑘 and 𝑊`` > 𝑊` 𝑌`` + 𝑦 𝑘 is a valid solution to 𝑇 because 𝑋`` + 𝑦 𝑘 ≤ 𝑋 𝑊`` = 𝑊`` + 𝑦 𝑘 𝑧 𝑘 > 𝑊 ′ + 𝑦 𝑘 𝑧 𝑘 > V This is a contradiction with the assumption that 𝑌 is an optimal solution for 𝑇 46

Huffman Codes 47

Encoding How data is represented? Fixed-size codes, e.g., ASCII A: 1000001 B: 1000010 Variable-size codes, e.g., Morse Codes A: ●▬ B: ▬●●● E: ● T: ▬ 48

Example: Morse Code 49

Prefix Codes No code is allowed to be a prefix of another code To encode, simply concatenate all the codes Decoding does not entail any ambiguity Example: Message ‘JAVA’ a = “0”, j = “11”, v = “10” Encoded message “110100” Decoding “ 110100 ” 51

Trie We can use a trie to find prefix codes the characters are stored at the external nodes a left child (edge) means 0 a right child (edge) means 1 A = 010 0 1 B = 11 0 1 0 1 C = 00 D B C D = 10 0 1 A R R = 011 52

Example of Decoding encoded text: 01011011010000101001011011010 text: ABRACADABRA ASCII: 88 bits Our encoding: 29 bits A = 010 0 1 B = 11 0 1 0 1 C = 00 D B C D = 10 0 1 A R R = 011 53