CS 270 Algorithms Oliver Week 8 Kullmann Greedy algorithms Making Greedy Algorithms change Minimum spanning trees Huffman Greedy algorithms 1 Codes Making change 2 Minimum spanning trees 3 Huffman Codes 4
CS 270 General remarks Algorithms Oliver Kullmann Greedy algorithms Making change Minimum We learn about greedy algorithms . spanning trees Huffman Reading from CLRS for week 8 Codes 1 Chapter 16.1-16.3 2 Chapter 23
CS 270 Greedy Algorithms Algorithms Oliver Kullmann Greedy algorithms This week and next week we will discuss two general methods Making change which may be used to solve optimisation problems. Minimum spanning The topic this week will be Greedy Algorithms. trees Huffman Next week we will discuss Dynamic Programming. Codes Idea of Greedy Algorithms When we have a choice to make, make the one that looks best right now . Make a locally optimal choice in hope of getting a globally optimal solution .
CS 270 The problem Algorithms Oliver Kullmann Greedy From [Anany Levitin, Introduction to the Design and Analysis of algorithms Algorithms], Chapter 9, “Greedy Technique”, with some Making change comments in square brackets: Minimum spanning trees Let us start with the change-making problem faced Huffman by millions of cashiers all over the world (at least Codes subconsciously): give change for a specific amount n with the least number of coins of the denominations d 1 > d 2 > · · · > d m used in that locale. For example, the widely [?] used coin denomination in the United States [of America] are d 1 = 25 (quarter), d 2 = 10 (dime), d 3 = 5 (nickel), and d 4 = 1 (penny).
CS 270 The problem (cont.) Algorithms Oliver How would you give change with the coins of these Kullmann denominations of, say, 48 cents? If you came up with Greedy the answer 1 quarter, 2 dimes, and 3 pennies algorithms [ 48 = 25 + 2 · 10 + 3 ], you followed [likely] — Making change consciously or not — a logical [better “reasonable”] Minimum strategy of making a sequence of best choices among spanning trees the currently available alternatives. Indeed, in the first Huffman step, you could have given one coin of any of the four Codes denominations. “Greedy” thinking leads to giving one quarter because it reduces the remaining about the most, namely, to 23 cents. In the second step, you had the same coins at your disposal, but you could not give a quarter because it would have violated the problem’s constraints. So your best selection in this step was one dime, reducing the remaining amount to 13 cents. Giving one more dime left you with 3 cents to be given with three pennies.
CS 270 Reflection Algorithms Oliver Kullmann Greedy algorithms Is this solution to the instance of the change-making Making problem optimal? Yes, it is. In fact, it is possible to change prove that the greedy algorithm yields an optimal Minimum spanning solution for every positive integer amount with these trees [!] coin denominations. At the same time, it is easy to Huffman Codes give an example of “weird” coin denominations — e.g., d 1 = 7 , d 2 = 5 , d 3 = 1 — which may not yield an optimal solution for some amounts. [The smallest amount for which the greedy strategy is not working is n = 10 . The general problem is called the subset-sum problem , and is NP-complete.]
CS 270 Reflection (cont.) Algorithms Oliver Kullmann The approach applied in the opening paragraph to the change-making problem is called greedy . [...] The Greedy algorithms greedy approach suggests constructing a solution Making change through a sequence of steps, each expanding a partially Minimum constructed solution obtained so far, until a complete spanning trees solution to the problem is reached. On each step — Huffman and this is the central point of this technique — the Codes choice made must be feasible, i.e., it has to satisfy the problem’s constraints. locally optimal, i.e., it has to be the [or one] best local choice among all feasible choices available on that step. irrevocable, i.e., once made, it cannot be changed on subsequent steps of the algorithm.
CS 270 Reflection (cont.) Algorithms Oliver Kullmann Greedy algorithms Making change Minimum These requirements explain the technique’s name: on spanning trees each step, it suggests a “greedy” grab of the the best Huffman alternative available in the hope that a sequence of Codes locally optimal choices will yield a (globally) optimal solution to the entire problem.
CS 270 Making change (again) Algorithms Oliver Kullmann Greedy algorithms Input: A list of integers representing coin denominations, plus Making change another positive integer representing an amount of money. Minimum spanning Output: A minimal collection of coins of the given trees denominations which sum to the given amount. Huffman Codes Greedy Strategy: Repeatedly include in the solution the largest coin whose value doesn’t exceed the remaining amount. E.g.: If the denominations are (25,10,5,1) and the amount is 87, then 87 = 25 + 25 + 25 + 10 + 1 + 1 .
CS 270 Applicability of greedy algorithms Algorithms Oliver Greedy algorithms don’t always work. Kullmann For example, in Making Change, if the denominations were Greedy algorithms (25,11,5,1), then Making change 15 = 11 + 1 + 1 + 1 + 1 = 5 + 5 + 5 . Minimum spanning trees However, quite often they do, or they come close enough to Huffman Codes the optimal solution to make the outcome acceptable. This, and the fact that they are quite easy to implement, make them an attractive alternative for many hard problems. Well-known instances of the use of Greedy Algorithms are the following problems: Minimum Spanning Trees (Kruskal’s Algorithm) Data Compression (Huffman Codes.) .
CS 270 Remark on coin denominations Algorithms Oliver As far as I am aware, for all coin denominations in place today Kullmann the greedy strategy works. Greedy algorithms The only coin denomination in history I am aware of for Making change which the greedy strategy doesn’t work was the British Minimum system before decimalisation (1971). spanning trees That system had Huffman Codes half crown – 30 pence two shillings – 24 pence shilling – 12 pence sixpence – 6 pence threepence – 3 pence penny – 1 pence halfpenny – 1/2 pence. Do you see an example where it doesn’t work? 48 = 30 + 12 + 6 = 2 · 24 .
CS 270 Minimum Spanning Trees (MST) Algorithms Oliver Kullmann Input: An undirected connected graph G = ( V , E ), and a (positive) weight w ( e ) ∈ R ≥ 0 on each edge e ∈ E . Greedy algorithms Output: A subset F ⊆ E of edges which connects all of the Making change vertices V of G , has no cycles, and minimises the total edge Minimum weight: spanning trees w ( F ) = � e ∈ F w ( e ) . Huffman Codes Example: 20 b 15 c 12 a 7 6 11 d 3 e 1 7 4 10 g f
CS 270 Kruskal’s algorithm Algorithms Oliver The following classic algorithm due to Kruskal is a greedy algorithm. Kullmann Greedy algorithms Kruskal-MST ( G , w ) Making / sort edges E of G by nondecreasing weight w change / 1 F = ∅ Minimum spanning 2 for each edge e ∈ E trees 3 if F ∪ { e } is acyclic Huffman Codes 4 F = F ∪ { e } 5 return F At each stage, the algorithm greedily chooses the cheapest edge and adds it to the partial solution F , provided it satisfies the acyclicity cri- terion. The running time of the algorithm is dominated by the sorting of edges in line 1 (assuming we can do the test in line 3 efficiently.) Hence, by using an optimal sorting algorithm, the running time of this algorithm is Θ( E lg E ).
CS 270 Kruskal’s algorithm illustrated Algorithms Oliver Kullmann b b b Greedy c c c a a a algorithms d d d e e e 3 3 1 1 1 4 Making g g g f f f change w ( a , f ) = 1 Minimum b b b c c c a a a spanning w ( a , d ) = 7 6 7 7 trees d d d e e 3 4 3 4 3 1 e 1 1 w ( a , b ) = 15 4 7 Huffman g g g f f f Codes w ( b , d ) = 12 w ( b , c ) = 20 b b b c c c a a a 12 7 7 7 d w ( c , g ) = 3 d 11 d e e e 4 3 3 3 1 1 1 4 4 w ( c , e ) = 6 f f f 10 g 10 g 10 g w ( d , f ) = 7 b b b 20 15 w ( d , e ) = 11 a c a c a c 12 12 12 7 7 7 d d d e 4 3 e 3 e 3 w ( e , g ) = 4 1 1 1 4 4 f f f 10 10 10 g g g w ( f , g ) = 10
CS 270 Correctness of Kruskal’s algorithm Algorithms Oliver Fact: At any time during the execution of Kruskal’s Algorithm, Kullmann F is a subset of some MST. Greedy algorithms Proof: By induction on | F | . This is clearly true (initially) when Making F = ∅ . change Suppose F � = ∅ , and let e =( a , b ) be the most recently added Minimum spanning edge. Let T be a MST which (by induction) contains trees Huffman F −{ e } . Let f be the first edge on the (unique) path in T Codes going from vertex a to vertex b which is not in F −{ e } , and assume that f � = e . We must have w ( e ) ≤ w ( f ) (for oth- b a e erwise f would have been included in F rather than e ), and hence we could get another MST by replacing f by e in T . Hence F is a subset of a MST. � f Corollary: Kruskal’s Algorithm computes a MST.
Recommend
More recommend
