Memoization Greedy Algorithms Example Memoization & Greedy: Wrapping up programmatical approaches with thanks to Kosie Van Der Merwe Tslil Clingman 2nd August Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Outline Memoization 1 Greedy Algorithms 2 Example 3 Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example The problem Consider the naive implementation of fib ( n ) = fib ( n − 1 ) + fib ( n − 2 ) We take fib ( 0 ) = 0 , fib ( 1 ) = 1 Look at the case of fib ( 5 ) - about half the work is computed twice! It can actually be shown that running such a recursive call takes O ( ϕ n ) time! Surely once we have computed the value of fib ( n ) we needn’t compute it again? Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example The idea What if, instead of just blindly computing the value at every step we ... Keep track of all previously computed results in some table Look up the arguments, if it has been computed already, we needn’t do any work Otherwise, compute the result but store it in the table It’s fun to see that this doesn’t actually rely on global state at all We can also have n many base cases now, without polluting the function definition Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Pseudo-code def fib_memo (n,dct): if n in dct: return dct[n] else res = fib_memo(n-1,dct)+fib_memo(n-2,dct) dct[res] = n return res def fib (n): return fib_memo(n,{0:0, 1:1}) We now have a fib ( n ) implementation which takes only O ( n ) time! Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example That’s all folks! In principle, that’s all there is to it Just about any recursive function can be transformed into a more efficient version through memoization The general approach remains the same as we have seen above If you’re worried about duplicated computing effort, consider memoization Space-time trade-off Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Outline Memoization 1 Greedy Algorithms 2 Example 3 Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Gimme gimme gimme The idea here is, at every step where some decision is required, consume as much (or as little) as the case may be Hope that this leads to the optimal solution ... I know. I said hope. ... So then why? Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example The why By and large, greedy algorithms are very very simple to code The are comparatively easy to understand, the logic is intuitive to us Constitute the ‘obvious’ solution Far more often than not, they run in O ( n ) time. Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Enough talk! Consider the classic change problem There is a certain total which you must account for given some set of denominations You seek the solution with the least overall coins (for convenience and customer satisfaction) In specific, say denominations = { 1 , 2 , 5 , 10 } , need change for an amount of 14 By hand, we could see the solution is 1 × 10 + 2 × 2 for a total of three coins No ... this is compsci! Particulars nothing, we want to solve the class of problems, not this instance Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example The proposed solution # denoms is a sorted array of the denominations # change is the amount given in for change def make_change (change, denoms): issued = [] while change > 0: k = len (denoms)-1 while denoms[k]>change: k-=1 largest = denoms[k] change -= largest issued.append(largest) return issued change(14,[1,2,5,10]) >> [10, 2, 2] change(16,[1,2,5,10]) >> [10, 5, 1] Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example See what we have wrought! On first examination, this seems to work well enough - no obvious flaw like issuing the wrong coin However, there are two somewhat serious errors in this logic: It can fail to converge on a solution 1 It will converge on sub-optimal solutions 2 Can you see such cases? Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Oopsy change(14,[7, 10]) >> IndexError: list index out of range change(14,[2, 7, 10]) >> [10, 2, 2] but optimal is [7, 7] Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Hark, a pitfall! Greedy may be somewhat more formally described as an algorithm which attempts to find a globally optimum solution by traversing the problem space and making locally optimum choices at each step Clearly, this will not always succeed Thus bear the following in mind when opting for greedy Sub-optimal yet correct answers Failure to converge altogether Depend on proof of correctness I have a wonderful proof but the margin is not wide enough to contain it... These proofs are not simple. Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example So ... When can you use them? It’s complicated. I’m sorry. Some examples include: Kruskal’s algorithm (MST), Huffman Encoding, Dominating set ... Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example So ... When can you use them? It’s complicated. I’m sorry. Some examples include: Kruskal’s algorithm (MST), Huffman Encoding, Dominating set ... Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Outline Memoization 1 Greedy Algorithms 2 Example 3 Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection Given a set S of n activities with start time s i and finish time f i You have to find the maximal size subset of S such that no two activities are done at the same time. Any suggestions for a greedy algorithm to solve this problem? Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection (Continued) We can find a counter examples for almost all greedy algorithms except where you select activities by the least finishing time. So let’s prove this works: First, we need to show there exists an optimal solution that makes the greedy choice. Secondly, we need to show that if we combine this greedy choice with the optimal solution for the remaining sub-problem we get an optimal solution to the original problem. Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection (Proof) So let’s prove that there exists an optimal solution that makes the greedy choice: Let A be an optimal solution. Let activity 1 be the greedy choice. If 1 ∈ A , then the proof is done. ∈ A , then we need to show that A ′ = A − { a } + { 1 } is If 1 / another optimal solution that includes activity 1, where a is the activity with the least finish time in A . Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection (Moar) Since our activities are sorted by finishing time in our algorithm, we have f 1 ≤ f a . But if f 1 ≤ s a we could add 1 to A and A would thus not be optimal. So f 1 > s a and thus 1 and a overlap. Since f 1 ≤ f a we could remove a and add 1 to get another compatible solution A ′ = A − { a } + { 1 } and | A ′ | = | A | . Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection (Break it down!) Now we have to prove that combining the greedy choice and the optimal solution to the sub-problem gives us the global optimal solution: Let 1 be the greedy choice. Let S ′ be the subset of activities that do not overlap with 1. S ′ = { x | s x ≥ f 1 } Let B be an optimal solution for S ′ . From the definition of S ′ , A ′ = { 1 } + B is compatible and a solution to the original problem, but not necessarily an optimal one. Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection (Getting there...) Using Proof by Contradiction: Assume A ′ is not an optimal solution. Let A be an optimal solution that contains 1. So | A ′ | < | A | , and | A − { 1 }| > | A ′ − { 1 }| = | B | But A − { 1 } is also a solution to the problem of S ′ , contradicting the assumption that B is an optimal solution to S ′ . Thus A ′ is an optimal solution to the original problem. Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example Activity Selection, a Solution Eventually ... code: #sort the N activities by the finish time. A = [0] lastadded = 0 for i in range (1,N-1): if s[i] >= f[lastadded]: A.append(i) lastadded = i Wasn’t that easy? But merely looking at this we have no way of seeing that it is always optimal Tslil Clingman Memoization and Greedy
Memoization Greedy Algorithms Example In summary Proofs are long, tricky and difficult to create In general, we don’t bother. It pays to simply try and suss things out Attempt to find degenerate cases of the problem and try to break your greedy The more subtle tests it passes, the more likely it is to be correct Tslil Clingman Memoization and Greedy
Recommend
More recommend
