CSC373 Week 3: Dynamic Programming Nisarg Shah 373F20 - Nisarg - PowerPoint PPT Presentation

CSC373 Week 3: Dynamic Programming Nisarg Shah 373F20 - Nisarg Shah 1

Recap • Greedy Algorithms ➢ Interval scheduling ➢ Interval partitioning ➢ Minimizing lateness ➢ Huffman encoding ➢ … 373F20 - Nisarg Shah 2

Jeff Erickson on greedy algorithms… 373F20 - Nisarg Shah 3

The 1950s were not good years for mathematical research. We had a very interesting gentleman in Washington named Wilson. He was secretary of Defense, and he actually had a pathological fear and hatred of the word ‘ research ’. I’m not using the term lightly; I’m using it precisely. His face would suffuse, he would turn red, and he would get violent if people used the term ‘ research ’ in his presence. You can imagine how he felt, then, about the term ‘ mathematical ’. The RAND Corporation was employed by the Air Force, and the Air Force had Wilson as its boss, essentially. Hence, I felt I had to do something to shield Wilson and the Air Force from the fact that I was really doing mathematics inside the RAND Corporation. What title, what name, could I choose? — Richard Bellman, on the origin of his term ‘dynamic programming’ (1984) Richard Bellman’s quote from Jeff Erickson’s book 373F20 - Nisarg Shah 4

Dynamic Programming • Outline ➢ Breaking the problem down into simpler subproblems, solve each subproblem just once, and store their solutions. ➢ The next time the same subproblem occurs, instead of recomputing its solution, simply look up its previously computed solution. ➢ Hopefully, we save a lot of computation at the expense of modest increase in storage space. ➢ Also called “ memoization ” • How is this different from divide & conquer? 373F20 - Nisarg Shah 5

Weighted Interval Scheduling • Problem ➢ Job 𝑘 starts at time 𝑡 𝑘 and finishes at time 𝑔 𝑘 ➢ Each job 𝑘 has a weight 𝑥 𝑘 ➢ Two jobs are compatible if they don’t overlap ➢ Goal: find a set 𝑇 of mutually compatible jobs with highest total weight σ 𝑘∈𝑇 𝑥 𝑘 • Recall: If all 𝑥 𝑘 = 1 , then this is simply the interval scheduling problem from last week ➢ Greedy algorithm based on earliest finish time ordering was optimal for this case 373F20 - Nisarg Shah 6

Recall: Interval Scheduling • What if we simply try to use it again? ➢ Fails spectacularly! 373F20 - Nisarg Shah 7

Weighted Interval Scheduling • What if we use other orderings? ➢ By weight: choose jobs with highest 𝑥 𝑘 first ➢ Maximum weight per time: choose jobs with highest 𝑥 𝑘 /(𝑔 𝑘 − 𝑡 𝑘 ) first ➢ ... • None of them work! ➢ They’re arbitrarily worse than the optimal solution ➢ In fact, under a certain formalization, “no greedy algorithm” can produce any “decent approximation” in the worst case (beyond this course!) 373F20 - Nisarg Shah 8

Weighted Interval Scheduling • Convention ➢ Jobs are sorted by finish time: 𝑔 1 ≤ 𝑔 2 ≤ ⋯ ≤ 𝑔 𝑜 ➢ 𝑞 𝑘 = largest index 𝑗 < 𝑘 such that job 𝑗 is compatible with job 𝑘 (i.e. 𝑔 𝑗 < 𝑡 𝑘 ) Among jobs before job 𝑘 , the ones compatible with it are precisely 1 … 𝑗 E.g. 𝑞[8] = 1, 𝑞[7] = 3, 𝑞[2] = 0 373F20 - Nisarg Shah 9

Weighted Interval Scheduling • The DP approach ➢ Let OPT be an optimal solution ➢ Two options regarding job 𝑜 : o Option 1: Job 𝑜 is in OPT • Can’t use incompatible jobs 𝑞 𝑜 + 1, … , 𝑜 − 1 • Must select optimal subset of jobs from {1, … , 𝑞 𝑜 } o Option 2: Job 𝑜 is not in OPT • Must select optimal subset of jobs from {1, … , 𝑜 − 1} ➢ OPT is best of both options ➢ Notice that in both options, we need to solve the problem on a prefix of our ordering 373F20 - Nisarg Shah 10

Weighted Interval Scheduling • The DP approach ➢ 𝑃𝑄𝑈(𝑘) = max total weight of compatible jobs from 1,…,𝑘 ➢ Base case: 𝑃𝑄𝑈 0 = 0 ➢ Two cases regarding job 𝑘 : o Job 𝑘 is selected: optimal weight is 𝑥 𝑘 + 𝑃𝑄𝑈(𝑞 𝑘 ) o Job 𝑘 is not selected: optimal weight is 𝑃𝑄𝑈(𝑘 − 1) ➢ Bellman equation: 0 if 𝑘 = 0 𝑃𝑄𝑈 𝑘 = ൝ max 𝑃𝑄𝑈 𝑘 − 1 , 𝑥 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 if 𝑘 > 0 373F20 - Nisarg Shah 11

Brute Force Solution 373F20 - Nisarg Shah 12

Brute Force Solution • Q: Worst-case running time of C OMPUTE -O PT (𝑜) ? Θ(𝑜) a) Θ 𝑜 log 𝑜 b) Θ 1.618 𝑜 c) Θ(2 𝑜 ) d) 373F20 - Nisarg Shah 13

Brute Force Solution • Brute force running time ➢ It is possible that 𝑞 𝑘 = 𝑘 − 1 for each 𝑘 ➢ Calling C OMPUTE -O PT (𝑘 − 1) and C OMPUTE -O PT (𝑞[𝑘]) separately would take 2 𝑜 steps ➢ We can slightly optimize: o If 𝑞 𝑘 = 𝑘 − 1 , call it just once, else call them separately o Now, the worst case is when 𝑞 𝑘 = 𝑘 − 2 for each 𝑘 o Running time: 𝑈 𝑜 = 𝑈 𝑜 − 1 + 𝑈 𝑜 − 2 • Fibonacci, golden ratio, … ☺ • 𝑈 𝑜 = 𝑃(𝜒 𝑜 ) , where 𝜒 ≈ 1.618 373F20 - Nisarg Shah 14

Dynamic Programming • Why is the runtime high? ➢ Some solutions are being computed many, many times o E.g. if 𝑞[5] = 3 , then C OMPUTE -O PT (5) calls C OMPUTE -O PT (4) and C OMPUTE -O PT (3) o But C OMPUTE -O PT (4) in turn calls C OMPUTE -O PT (3) again • Memoization trick ➢ Simply remember what you’ve already computed, and re - use the answer if needed in future 373F20 - Nisarg Shah 15

Dynamic Program: Top-Down • Let’s store C OMPUTE -O PT (j) in 𝑁[𝑘] 373F20 - Nisarg Shah 16

Dynamic Program: Top-Down • Claim: This memoized version takes 𝑃 𝑜 log 𝑜 time ➢ Sorting jobs takes 𝑃 𝑜 log 𝑜 ➢ It also takes 𝑃(𝑜 log 𝑜) to do 𝑜 binary searches to compute 𝑞(𝑘) for each 𝑘 ➢ M-Compute-OPT( 𝑘 ) is called at most once for each 𝑘 ➢ Each such call takes 𝑃(1) time, not considering the time taken by any subroutine calls ➢ So M-Compute-OPT( 𝑜 ) takes only 𝑃 𝑜 time ➢ Overall time is 𝑃 𝑜 log 𝑜 373F20 - Nisarg Shah 17

Dynamic Program: Bottom-Up • Find an order in which to call the functions so that the sub-solutions are ready when needed 373F20 - Nisarg Shah 18

Top-Down vs Bottom-Up • Top- Down may be preferred… ➢ …when not all sub -solutions need to be computed on some inputs ➢ …because one does not need to think of the “right order” in which to compute sub-solutions • Bottom- Up may be preferred… ➢ …when all sub -solutions will anyway need to be computed ➢ …because it is faster as it prevents recursive call overheads and unnecessary random memory accesses ➢ …because sometimes we can free -up memory early 373F20 - Nisarg Shah 19

Optimal Solution • This approach gave us the optimal value • What about the actual solution (subset of jobs)? ➢ Idea: Maintain the optimal value and an optimal solution ➢ So, we compute two quantities: 0 if 𝑘 = 0 𝑃𝑄𝑈 𝑘 = ൝ max 𝑃𝑄𝑈 𝑘 − 1 , 𝑥 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 if 𝑘 > 0 ∅ if 𝑘 = 0 𝑇(𝑘 − 1) if 𝑘 > 0 ∧ 𝑃𝑄𝑈 𝑘 − 1 ≥ 𝑥 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 𝑇 𝑘 = ൞ 𝑘 ∪ 𝑇(𝑞 𝑘 ) if 𝑘 > 0 ∧ 𝑃𝑄𝑈 𝑘 − 1 < 𝑥 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 373F20 - Nisarg Shah 20

Optimal Solution 0 if 𝑘 = 0 𝑃𝑄𝑈 𝑘 = ൝ max 𝑃𝑄𝑈 𝑘 − 1 , 𝑤 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 if 𝑘 > 0 ∅ if 𝑘 = 0 𝑇(𝑘 − 1) if 𝑘 > 0 ∧ 𝑃𝑄𝑈 𝑘 − 1 ≥ 𝑤 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 𝑇 𝑘 = ൞ 𝑘 ∪ 𝑇(𝑞 𝑘 ) if 𝑘 > 0 ∧ 𝑃𝑄𝑈 𝑘 − 1 < 𝑤 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 In this problem, we can do something This works with both top-down simpler: just compute 𝑃𝑄𝑈 first, and (memoization) and bottom-up later compute 𝑇 using only 𝑃𝑄𝑈 . approaches. 373F20 - Nisarg Shah 21

Optimal Solution 0 if 𝑘 = 0 𝑃𝑄𝑈 𝑘 = ൝ max 𝑃𝑄𝑈 𝑘 − 1 , 𝑤 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 if 𝑘 > 0 ⊥ if 𝑘 = 0 𝑀 if 𝑘 > 0 ∧ 𝑃𝑄𝑈 𝑘 − 1 ≥ 𝑤 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 𝑇 𝑘 = ൞ 𝑆 if 𝑘 > 0 ∧ 𝑃𝑄𝑈 𝑘 − 1 < 𝑤 𝑘 + 𝑃𝑄𝑈 𝑞 𝑘 • Save space by storing only one bit of information for each 𝑘 : which option yielded the max weight • To reconstruct the optimal solution, start with 𝑘 = 𝑜 ➢ If 𝑇 𝑘 = 𝑀 , update 𝑘 ← 𝑘 − 1 ➢ If 𝑇 𝑘 = 𝑆 , add 𝑘 to the solution and update 𝑘 ← 𝑞[𝑘] ➢ If 𝑇 𝑘 =⊥ , stop 373F20 - Nisarg Shah 22

Optimal Substructure Property • Dynamic programming applies well to problems that have optimal substructure property ➢ Optimal solution to a problem can be computed easily given optimal solution to subproblems • Recall: divide-and-conquer also uses this property ➢ Divide-and-conquer is a special case in which the subproblems don’t “overlap” ➢ So there’s no need for memoization ➢ In dynamic programming, two of the subproblems may in turn require access to solution to the same subproblem 373F20 - Nisarg Shah 23