COMP 3403 Algorithm Analysis Part 5 Chapter 9 Jim Diamond CAR - PowerPoint PPT Presentation

COMP 3403 — Algorithm Analysis Part 5 — Chapter 9 Jim Diamond CAR 409 Jodrey School of Computer Science Acadia University

Chapter 9 Greedy Techniques Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 160 Greedy Approaches • One technique to solve a problem is to make a series of decisions which seem to be the best at the instant each decision is made • More specifically, these decisions have three properties: – – they are locally optimal : that is, among all options available at the time the decision is made, the best (or a best) decision is made – they are irrevocable : that is, having made a decision, the decision can not be taken back or reversed (you can’t later change your mind) • Applying these criteria often provide a relatively simple algorithm – sometimes the overall solution is optimal, sometimes it isn’t – in cases where the solution is not optimal, a greedy approach can still be useful – – it provides a bound, which may be useful for other purposes Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 161 Change-Making Problem • Given an “unlimited” amounts of coins of denominations d 1 > · · · > d m , make change for an amount n using the smallest number of coins • Example: d 1 = 25 / c, d 2 = 10 / c, d 3 = 5 / c, d 4 = 1 / c and n = 62 / c • The greedy solution: – – may not be optimal for “unnormal” coin denominations e.g., suppose n = 15 / – c and the available coin denominations are 10/ c, 7/ c and 1/ c; the optimal solution uses three coins the greedy solution uses six coins • In the case of “unnormal” coin denominations, the greedy solution gives an upper bound on the number of coins needed, but the upper bound is not (in general) “tight” Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 162 Minimum(-Cost/Weight) Spanning Trees • Given a connected graph G , a spanning tree T of G is a connected, acyclic subgraph of G which contains all of G ’s vertices • Given a weighted, connected graph G , a minimum-cost spanning tree T of G is a spanning tree of G which is has minimum cost over all of G ’s spanning trees – • Example A graph and its 3 spanning trees; T 1 has min cost Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 163 Prim’s MST algorithm • Start with a tree T 1 consisting of one (any) vertex and “grow” the tree one vertex at a time to produce a MST through a series of expanding subtrees T 1 , T 2 , . . . , T n • On each iteration, construct T i +1 from T i by adding a vertex not in T i that is closest to some vertex already in T i (this is a “greedy” step!) • Stop when all vertices are included /* * Prim ’ s algorithm to find a minimum cost spanning tree. * Input: a weighted, connected graph G=(V,E) * Returns: E T , the set of edges of an MST of G */ Prim(G) V T ← { v 0 } E T ← Ø for i ← 1 to |V| - 1 do find a min-cost edge e*= { u*, v* } with v* ∈ V T and u* �∈ V T V T ← V T ∪ { u* } E T ← E T ∪ { e* } return E T Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 164 Example of Prim’s Algorithm • Arbitrarily start with the first vertex (which is a ); this gives V ( T 1 ) = { a } • Find a min-cost edge with a at one end: ( a, b ) ; this gives V ( T 2 ) = { a, b } • Find a min-cost edge with a or b at one end, the other end in V ( T 2 ) : ( b, c ) ; this gives V ( T 3 ) = { a, b, c } • Next we pick ( b, f ) ; this gives V ( T 4 ) = { a, b, c, f } • Next we pick ( e, f ) ; this gives V ( T 5 ) = { a, b, c, e, f } • Next we pick ( d, f ) ; this gives V ( T 6 ) = { a, b, c, d, e, f } • The min cost tree has E ( T 6 ) = { ( a, b ) , ( b, c ) , ( b, f ) , ( d, f ) , ( e, f ) } Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 165 Prim’s Algorithm: Is the Tree Optimal? • The algorithm obeys the criteria: feasible, locally optimal, irrevocable; but is the overall answer optimal? Yes! Proof by contradiction: Let T be a tree generated by Prim, where e 1 was the first edge added, e 2 the second, and so on. Assume T is not optimal; then there are optimal trees { T ∗ ( k ) } which have smaller costs than T . For each such k , let e i k = ( v, u ) be the first edge added to T which is not in T ∗ ( k ) (i.e., e 1 , . . . , e i k − 1 are all in T ∗ ( k ) , but e i k / ∈ T ∗ ( k ) ). Choose T ∗ to be an optimal tree which maximizes i k . Example: suppose T is Prim’s tree for some G , and T ∗ (1) , T ∗ (2) , T ∗ (3) and T ∗ (4) are MSTs for G , where T = e 1 , e 2 , e 3 , e 4 , e 5 , e 6 , e 7 , e 8 , e 9 , e 10 , e 11 , e 12 , e 13 , e 14 , e 15 , . . . , e n − 1 { e 1 , e 2 , e 3 , e 4 } ⊂ T ∗ (1) , but e 5 / ∈ T ∗ (1) thus e i 1 = e 5 { e 1 , e 2 , e 3 , . . . , e 8 } ⊂ T ∗ (2) , but e 9 / ∈ T ∗ (2) thus e i 2 = e 9 { e 1 , e 2 } ⊂ T ∗ (3) , but e 3 / ∈ T ∗ (3) thus e i 3 = e 3 { e 1 , e 2 , e 3 , . . . , e 6 } ⊂ T ∗ (4) , but e 7 / ∈ T ∗ (4) thus e i 4 = e 7 Here we would choose T ∗ = T ∗ (2) Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 166 Prim’s Algorithm: Is the Tree Optimal? (2) Consider the graph T ∗ ∪ { e i k } : u e ′ v e i k This must have some cycle C , and C must have some edge e ′ � = e i k • connecting some vertex v ∈ T i k − 1 to some vertex u / ∈ T i k − 1 . If e ′ had less cost than e i k , Prim’s algorithm would have chosen it • – By deleting e ′ from T ∗ ∪ { e i k } we produce a tree T ′ whose weight is no • larger than T ∗ Since T ∗ is assumed to be optimal, c ( T ′ ) = c ( T ∗ ) . But then T ′ is an optimal tree with a larger associated i k value than T ∗ , contradicting the choice of T ∗ . Therefore T is optimal. Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 167 Prim’s Algorithm: Efficiency We should be able to do operations like “add e ∗ to E T ” and “add u ∗ to • V T ” efficiently (in constant time?) But how about “find a min cost edge ( u ∗ , v ∗ ) with v ∗ ∈ V T and • u ∗ ∈ V T ”? – • There are various data structures we could use to solve this • E.g., we can use a priority queue where each element in the queue is an edge the book idea uses a priority queue, but for vertices – when a new vertex u ∗ is added to the tree, we add all edges from u ∗ – to non-tree vertices to the priority queue • GEQ #1: is this a valid solution? • GEQ #2: if so, how efficient is this? Is it better or worse than the description in the book? • GEQ #3: if better, does every edge removed from the priority queue end up in the tree? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 168 Kruskal’s MST Algorithm • Idea: – – grow tree one edge at a time to produce an MST through a series of forests F 1 , F 2 , . . . , F n − 1 – on each iteration, add the next edge on the sorted list unless this would create a cycle; if it would, skip the edge // Kruskal ’ s algorithm to find an MST // Input: a weighted, connected graph G = (V,E) // Returns: E T , the set of edges of an MST of G Kruskal(G) sort E in nondecreasing order of weights: e i 1 , e i 2 , e i 3 , . . . E T ← Ø edge_count ← 0 k ← 0 ; ; while edge_count < |V| - 1 k++ if E T ∪ { e i k } is acyclic E T ← E T ∪ { e i k }, edge_count++ return E T Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 169 Kruskal Example Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 170 Kruskal’s Algorithm Implementation Considerations • In some respects, this algorithm appears simpler than Prim’s algorithm • However, the question “is E T ∪ { E i k } acyclic?” is easier said than done – • Idea 1: if E i k = ( u, v ) , search in the current graph to see if v is reachable from u – – can we do better? • Idea 2: can we come up with an efficient way of answering the question “is u in the same tree as v ?” – Q: what if each tree in the forest had a unique id, and we could find the id of u ’s tree (and v ’s tree) efficiently? – Q’: when an edge is added, two trees are merged: how can we efficiently update one tree’s id? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 9 171 The Union-Find Algorithm • Suppose we have some algorithm which requires the following abstract operations: – makeset(v) — create a set with the element v find(v) — return (a unique identifier for) the set containing v – union(u,v) — move all elements of the set containing v to the set – containing u • These operations allow us to implement Kruskal’s algorithm: – this creates a forest of |V| trees, each with just one vertex – to answer the question “ if E T ∪ { e i k } is acyclic ” where – e i k = (u,v) merely compare find(u) to find(v) – to implement “ E T ← E T ∪ { e i k } ” where e i k = (u,v) merely call – union(u,v) Jim Diamond, Jodrey School of Computer Science, Acadia University