CPSC 320: Intermediate Algorithm Design and Analysis July 25, 2014 - PowerPoint PPT Presentation

CPSC 320: Intermediate Algorithm Design and Analysis July 25, 2014 1

Course Outline Introduction and basic concepts • Asymptotic notation • Greedy algorithms • Graph theory • Amortized analysis • Recursion • Divide-and-conquer algorithms • Randomized algorithms • Dynamic programming algorithms • NP-completeness • 2

Dynamic Programming 3

Dynamic Programming Greedy algorithm: combines a choice with result of taking the choice • What if right choice can’t be found easily? • 4

Coin Change Problem Assume a set of possible coins, and an amount, find the smallest number of coins • that add up to the exact amount Example: CoinChange( {1,5,10,25} , 47 )= [25, 10,10,1,1] • Is a greedy solution always optimal? • Example: GreedyCoinChange( {1,5,12,25} , 29 )= [25, 1,1,1,1] , optimal is [12,12,5] • 5

Coin Change Problem What is a subproblem that can be solved • If an optimal solution 𝑡 for 𝑂 contains 𝑑 , then an optimal solution for 𝑂 − 𝑑 is • 𝑡 − 𝑑 So the subproblem is solving for 𝑂 − 𝑑 • Let’s define the problem recursively: • For each coin 𝑑 , calculate 𝑡 𝑑 = CoinChange( 𝐷 , 𝑂 − 𝑑 ) • Select coin 𝑑 with smallest number of coins in subproblem • Attach coin 𝑑 to 𝑡 𝑑 and return • 6

Coin Change Recursive Algorithm CoinChangeRecursive( 𝐷 , 𝑂 ) – 𝐷 is set of coins, 𝑂 is amount If 𝑂 = 0 Then Return ∅ 𝑐 ← +∞ , 𝑡 ← ∅ For Each 𝑑 ∈ 𝐷 Do If 𝑑 ≤ 𝑂 Then 𝑡 𝑑 ← CoinChangeRecursive( 𝐷 , 𝑂 − 𝑑 ) If 𝑡 𝑑 + 1 < 𝑐 Then 𝑐 ← 𝑡 𝑑 + 1 𝑡 ← 𝑡 𝑑 ∪ {𝑑} Return 𝑡 7

Recursive Is this efficient? • Value for subproblems is calculated repeatedly • Example: CoinChange( 1,3,4 , 90 ), value for 86 is calculated on: • 4 + CoinChange( 86 ) • 3 + 1 + CoinChange( 86 ) • 1 + 3 + CoinChange( 86 ) • 1 + 1 + 1 + 1 + CoinChange( 86 ) • Can we save the value (“memoize”) and reuse it? • Can we calculate from the bottom-up? • 8

Dynamic Programming Approach Algorithm CoinChangeDP( 𝐷 , 𝑂 ) – 𝐷 is set of coins, 𝑂 is amount 𝑇 0 ← ∅ For 𝑗 ← 1 To 𝑂 Do 𝑇 𝑗 ← ∅ (length infinity) For Each 𝑑 ∈ 𝐷 Do If 𝑑 ≤ 𝑗 And 𝑇 𝑗 − 𝑑 Then + 1 < 𝑇 𝑗 𝑇[𝑗] ← 𝑇 𝑗 − 𝑑 ∪ {𝑑} Return 𝑇[𝑗] 9

Dynamic Programming Approach - Faster Algorithm CoinChangeDP2( 𝐷 , 𝑂 ) – 𝐷 is set of coins, 𝑂 is amount 𝑚[0] ← ∅ , 𝑐 0 ← 0 For 𝑗 ← 1 To 𝑂 Do 𝑚 𝑗 ← ∅ , 𝑐 𝑗 ← +∞ For Each 𝑑 ∈ 𝐷 Do If 𝑑 ≤ 𝑗 And 𝑐 𝑗 − 𝑑 + 1 < 𝑐[𝑗] Then 𝑚 𝑗 ← 𝑑 , 𝑐[𝑗] ← 𝑐[𝑗 − 𝑑] + 1 𝑡 ← ∅ While 𝑂 > 0 Do 𝑡 ← 𝑡 ∪ 𝑚 𝑂 , 𝑂 ← 𝑂 − 𝑚[𝑂] Return 𝑡 10

Rod Cutting Problem Problem: given a rod of length 𝑜 and a table of prices 𝑞 𝑗 for rods of length 𝑗 ≤ 𝑜 , • determine the maximum revenue 𝑠 𝑜 obtainable by cutting it in pieces Example: 𝑜 = 5, 𝑞 1 = 1, 𝑞 2 = 5, 𝑞 3 = 8, 𝑞 4 = 9, 𝑞 5 = 10 • Full rod: revenue is 10 • Cut in 5 pieces of 1: revenue is 5 • Cut in pieces of 2 and 3: revenue is 13 • Problem can be solved with dynamic programming • Choose each possible size, then solve for remainder • 11

Rod Cutting Problem Algorithm RodCutting( 𝑜 , 𝑞 ) – 𝑜 is size, 𝑞 is price array 𝑠 0 = 0 For 𝑗 ← 1 To 𝑜 Do 𝑠 𝑗 ← 𝑞[𝑗] , 𝑚 𝑗 ← 𝑗 For 𝑘 ≥ 1 To 𝑗 − 1 Do If 𝑠 𝑗 < 𝑠 𝑗 − 𝑘 + 𝑞[𝑘] Then 𝑠 𝑗 ← 𝑠 𝑗 − 𝑘 + 𝑞[𝑘] , 𝑚 𝑗 ← 𝑘 𝑡 ← ∅ While 𝑜 > 0 Do 𝑡 ← 𝑡 ∪ 𝑚 𝑜 , 𝑜 ← 𝑜 − 𝑚[𝑜] Return 𝑡 12

Weighted Interval Scheduling Problem: same as interval scheduling, but with preferences • Given a set of intervals 𝑇 , and a weight function 𝑥 , return a compatible subset • of 𝑇 with maximum total weight Greedy approach doesn’t work anymore • Define the subproblem • In optimal solution, if we remove last interval 𝑗 , remainder is optimal ending • before 𝑡 𝑗 13

Weighted Interval Scheduling Algorithm WeightedIntervalSched( 𝐽 , 𝑥 ) – 𝐽 is set of intervals, 𝑥 is weight function Sort 𝐽 by increasing finishing time 𝑋 0 ← 0 For 𝑘 ← 1 To 𝐽 Do 𝑞 ← last event that ends before 𝐽[𝑘] starts (0 if none) If 𝑋 𝑘 − 1 > 𝑥 𝐽 𝑘 + 𝑋 𝑞 Then 𝑋 𝑘 ← 𝑋 𝑘 − 1 , 𝐷 𝑘 ← false Else 𝑋 𝑘 ← 𝑥 𝐽 𝑘 + 𝑋 𝑞 , 𝐷 𝑘 ← true … 14

Weighted Interval Scheduling (cont.) … 𝑇 ← ∅ For 𝑘 ← |𝐽| DownTo 1 Do If 𝐷[𝑘] Then Add 𝐽[𝑘] to 𝑇 𝑘 ← last event that ends before 𝐽[𝑘] starts (0 if none) Return 𝑇 15

Subset Sum Problem Problem: given a set of weights, find the largest sum of weights below a limit • Applications: get a bag as full as possible, fill a server with limited resource • time as much as possible Extension: each weight has a value associated to it (knapsack problem) • Now the subproblem is slightly different • If an optimal solution contains weight 𝑥 for limit 𝑀 , then the remainder is • optimal for limit 𝑀 − 𝑥 16

Subset Sum Problem Algorithm SubsetSum( 𝑥 , 𝑁 ) – 𝑥 is array of weights, 𝑁 is limit 𝑠 0, 𝑛 ← 0 , 𝑚 0, 𝑛 ← false for 𝑛 = 0,1,2, … , 𝑁 For 𝑗 ← 1 To 𝑥 Do For 𝑛 ← 1 To 𝑁 Do If 𝑥 𝑗 > 𝑛 Or 𝑠 𝑗 − 1, 𝑛 > 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 + 𝑥[𝑗] Then 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 , 𝑚 𝑗, 𝑛 ← 𝑚[𝑗 − 1, 𝑛] Else + 𝑥[𝑗] , 𝑚 𝑗, 𝑛 ← 𝑗 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 𝑡 ← ∅ , 𝑦 ← 𝑥 While 𝑁 > 0 And 𝑚[𝑦, 𝑁] is not false Do 𝑦 ← 𝑚[𝑦, 𝑁] , 𝑡 ← 𝑡 ∪ 𝑦 , 𝑁 ← 𝑁 − 𝑥 𝑦 , 𝑦 ← 𝑦 − 1 Return 𝑡 17

Knapsack Problem Algorithm Knapsack( 𝑥 , 𝑞 , 𝑁 ) – 𝑥 is array of weights, 𝑞 is array of values, 𝑁 is limit 𝑠 0, 𝑛 ← 0 , 𝑚 0, 𝑛 ← false for 𝑛 = 0,1,2, … , 𝑁 For 𝑗 ← 1 To 𝑥 Do For 𝑛 ← 1 To 𝑁 Do If 𝑥 𝑗 > 𝑛 Or 𝑠 𝑗 − 1, 𝑛 > 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 + 𝑞[𝑗] Then 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 , 𝑚 𝑗, 𝑛 ← 𝑚[𝑗 − 1, 𝑛] Else + 𝑞[𝑗] , 𝑚 𝑗, 𝑛 ← 𝑗 𝑠 𝑗, 𝑛 ← 𝑠 𝑗 − 1, 𝑛 − 𝑥 𝑗 𝑡 ← ∅ , 𝑦 ← 𝑥 While 𝑁 > 0 And 𝑚[𝑦, 𝑁] is not false Do 𝑦 ← 𝑚[𝑦, 𝑁] , 𝑡 ← 𝑡 ∪ 𝑦 , 𝑁 ← 𝑁 − 𝑥 𝑦 , 𝑦 ← 𝑦 − 1 Return 𝑡 18