Greedy Algorithms for Minimum Spanning Trees Lecture 18 March 31, - PowerPoint PPT Presentation

Safe edge Example... Every cut identifies one safe edge... V \ S S 13 7 3 5 11 Safe edge in the cut ( S, V \ S ) ...the cheapest edge in the cut. Note: An edge e may be a safe edge for many cuts! Chandra & Lenny (UIUC) CS374 15 Spring 2015 15 / 61

Unsafe edge Example... Every cycle identifies one unsafe edge... 5 7 3 2 15 Chandra & Lenny (UIUC) CS374 16 Spring 2015 16 / 61

Unsafe edge Example... Every cycle identifies one unsafe edge... 5 7 3 2 15 15 ...the most expensive edge in the cycle. Chandra & Lenny (UIUC) CS374 16 Spring 2015 16 / 61

Example 20 1 2 23 4 15 1 9 6 7 3 36 16 3 28 25 5 4 17 Figure : Graph with unique edge costs. Safe edges are red, rest are unsafe. Chandra & Lenny (UIUC) CS374 17 Spring 2015 17 / 61

Example 20 1 2 23 4 15 1 9 6 7 3 36 16 3 28 25 5 4 17 Figure : Graph with unique edge costs. Safe edges are red, rest are unsafe. Chandra & Lenny (UIUC) CS374 17 Spring 2015 17 / 61

Example 20 1 2 23 4 15 1 9 6 7 3 36 16 3 28 25 5 4 17 Figure : Graph with unique edge costs. Safe edges are red, rest are unsafe. And all safe edges are in the MST in this case... Chandra & Lenny (UIUC) CS374 17 Spring 2015 17 / 61

Key Observation: Cut Property Lemma If e is a safe edge then every minimum spanning tree contains e . Chandra & Lenny (UIUC) CS374 18 Spring 2015 18 / 61

Key Observation: Cut Property Lemma If e is a safe edge then every minimum spanning tree contains e . Proof. Suppose (for contradiction) e is not in MST T . 1 Since e is safe there is an S ⊂ V such that e is the unique min 2 cost edge crossing S . Since T is connected, there must be some edge f with one end 3 in S and the other in V \ S Since c f > c e , T ′ = (T \ { f } ) ∪ { e } is a spanning tree of 4 lower cost! Chandra & Lenny (UIUC) CS374 18 Spring 2015 18 / 61

Key Observation: Cut Property Lemma If e is a safe edge then every minimum spanning tree contains e . Proof. Suppose (for contradiction) e is not in MST T . 1 Since e is safe there is an S ⊂ V such that e is the unique min 2 cost edge crossing S . Since T is connected, there must be some edge f with one end 3 in S and the other in V \ S Since c f > c e , T ′ = (T \ { f } ) ∪ { e } is a spanning tree of 4 lower cost! Error: T ′ may not be a spanning tree!! Chandra & Lenny (UIUC) CS374 18 Spring 2015 18 / 61

Error in Proof: Example Problematic example. S = { 1 , 2 , 7 } , e = (7 , 3) , f = (1 , 6) . T − f + e is not a spanning tree. 20 1 2 23 4 15 1 9 (A) Consider adding the edge f . 1 6 7 3 f 36 16 3 28 25 5 4 (A) 17 Chandra & Lenny (UIUC) CS374 19 Spring 2015 19 / 61

Error in Proof: Example Problematic example. S = { 1 , 2 , 7 } , e = (7 , 3) , f = (1 , 6) . T − f + e is not a spanning tree. 20 1 2 23 4 15 1 (A) Consider adding the edge f . 1 9 (B) It is safe because it is the 2 6 7 3 f 36 cheapest edge in the cut. 16 3 28 25 5 4 (B) 17 Chandra & Lenny (UIUC) CS374 19 Spring 2015 19 / 61

Error in Proof: Example Problematic example. S = { 1 , 2 , 7 } , e = (7 , 3) , f = (1 , 6) . T − f + e is not a spanning tree. 20 (A) Consider adding the edge f . 1 2 1 (B) It is safe because it is the 23 2 4 15 1 e cheapest edge in the cut. 9 (C) Lets throw out the edge e 3 6 7 3 f 36 currently in the spanning tree which is more expensive than f 16 3 28 25 and is in the same cut. Put it f instead... 5 4 (C) 17 Chandra & Lenny (UIUC) CS374 19 Spring 2015 19 / 61

Error in Proof: Example Problematic example. S = { 1 , 2 , 7 } , e = (7 , 3) , f = (1 , 6) . T − f + e is not a spanning tree. (A) Consider adding the edge f . 1 20 1 2 (B) It is safe because it is the 2 cheapest edge in the cut. 23 4 15 1 (C) Lets throw out the edge e 3 currently in the spanning tree 9 6 7 3 which is more expensive than f 36 and is in the same cut. Put it f 16 3 28 25 instead... (D) New graph of selected edges 4 5 4 (D) 17 is not a tree anymore. BUG. Chandra & Lenny (UIUC) CS374 19 Spring 2015 19 / 61

Proof of Cut Property Proof. Suppose e = (v , w) is not in MST 1 T and e is min weight edge in cut 20 (S , V \ S) . Assume v ∈ S . 1 2 23 4 15 1 9 6 7 3 36 16 25 3 28 5 4 17 Chandra & Lenny (UIUC) CS374 20 Spring 2015 20 / 61

Proof of Cut Property Proof. Suppose e = (v , w) is not in MST 1 T and e is min weight edge in cut 20 (S , V \ S) . Assume v ∈ S . 1 2 T is spanning tree: there is a unique 2 23 4 15 1 path P from v to w in T 9 6 7 3 36 e 16 25 3 28 5 4 17 Chandra & Lenny (UIUC) CS374 20 Spring 2015 20 / 61

Proof of Cut Property Proof. Suppose e = (v , w) is not in MST 1 T and e is min weight edge in cut 20 (S , V \ S) . Assume v ∈ S . 1 2 T is spanning tree: there is a unique 2 23 4 15 1 path P from v to w in T P 9 6 7 3 36 e 16 25 3 28 5 4 17 Chandra & Lenny (UIUC) CS374 20 Spring 2015 20 / 61

Proof of Cut Property Proof. Suppose e = (v , w) is not in MST 1 T and e is min weight edge in cut 20 (S , V \ S) . Assume v ∈ S . 1 2 T is spanning tree: there is a unique 2 23 4 15 e ′ 1 path P from v to w in T Let w ′ be the first vertex in P 9 3 6 7 3 belonging to V \ S ; let v ′ be the 36 e vertex just before it on P , and let 16 25 3 28 e ′ = (v ′ , w ′ ) 5 4 17 Chandra & Lenny (UIUC) CS374 20 Spring 2015 20 / 61

Proof of Cut Property Proof. Suppose e = (v , w) is not in MST 1 T and e is min weight edge in cut 20 (S , V \ S) . Assume v ∈ S . 1 2 T is spanning tree: there is a unique 2 23 4 15 1 path P from v to w in T Let w ′ be the first vertex in P 9 3 6 7 3 belonging to V \ S ; let v ′ be the 36 vertex just before it on P , and let 16 25 3 28 e ′ = (v ′ , w ′ ) T ′ = (T \ { e ′ } ) ∪ { e } is spanning 4 5 4 17 tree of lower cost. (Why?) Chandra & Lenny (UIUC) CS374 20 Spring 2015 20 / 61

Proof of Cut Property (contd) Observation T ′ = (T \ { e ′ } ) ∪ { e } is a spanning tree. Proof. T ′ is connected. T ′ is a tree Chandra & Lenny (UIUC) CS374 21 Spring 2015 21 / 61

Proof of Cut Property (contd) Observation T ′ = (T \ { e ′ } ) ∪ { e } is a spanning tree. Proof. T ′ is connected. Removed e ′ = (v ′ , w ′ ) from T but v ′ and w ′ are connected by the path P − f + e in T ′ . Hence T ′ is connected if T is. T ′ is a tree Chandra & Lenny (UIUC) CS374 21 Spring 2015 21 / 61

Proof of Cut Property (contd) Observation T ′ = (T \ { e ′ } ) ∪ { e } is a spanning tree. Proof. T ′ is connected. Removed e ′ = (v ′ , w ′ ) from T but v ′ and w ′ are connected by the path P − f + e in T ′ . Hence T ′ is connected if T is. T ′ is a tree T ′ is connected and has n − 1 edges (since T had n − 1 edges) and hence T ′ is a tree Chandra & Lenny (UIUC) CS374 21 Spring 2015 21 / 61

Safe Edges form a Tree Lemma Let G be a connected graph with distinct edge costs, then the set of safe edges form a connected graph. Proof. Suppose not. Let S be a connected component in the graph 1 induced by the safe edges. Consider the edges crossing S , there must be a safe edge among 2 them since edge costs are distinct and so we must have picked it. Chandra & Lenny (UIUC) CS374 22 Spring 2015 22 / 61

Safe Edges form an MST Corollary Let G be a connected graph with distinct edge costs, then set of safe edges form the unique MST of G . Chandra & Lenny (UIUC) CS374 23 Spring 2015 23 / 61

Safe Edges form an MST Corollary Let G be a connected graph with distinct edge costs, then set of safe edges form the unique MST of G . Consequence: Every correct MST algorithm when G has unique edge costs includes exactly the safe edges. Chandra & Lenny (UIUC) CS374 23 Spring 2015 23 / 61

Cycle Property Lemma If e is an unsafe edge then no MST of G contains e . Proof. Exercise. Note: Cut and Cycle properties hold even when edge costs are not distinct. Safe and unsafe definitions do not rely on distinct cost assumption. Chandra & Lenny (UIUC) CS374 24 Spring 2015 24 / 61

Correctness of Prim’s Algorithm Prim’s Algorithm Pick edge with minimum attachment cost to current tree, and add to current tree. Proof of correctness. If e is added to tree, then e is safe and belongs to every MST . 1 Set of edges output is a spanning tree 2 Chandra & Lenny (UIUC) CS374 25 Spring 2015 25 / 61

Correctness of Prim’s Algorithm Prim’s Algorithm Pick edge with minimum attachment cost to current tree, and add to current tree. Proof of correctness. If e is added to tree, then e is safe and belongs to every MST . 1 Let S be the vertices connected by edges in T when e is added. 1 Set of edges output is a spanning tree 2 Chandra & Lenny (UIUC) CS374 25 Spring 2015 25 / 61

Correctness of Prim’s Algorithm Prim’s Algorithm Pick edge with minimum attachment cost to current tree, and add to current tree. Proof of correctness. If e is added to tree, then e is safe and belongs to every MST . 1 Let S be the vertices connected by edges in T when e is added. 1 e is edge of lowest cost with one end in S and the other in 2 V \ S and hence e is safe. Set of edges output is a spanning tree 2 Chandra & Lenny (UIUC) CS374 25 Spring 2015 25 / 61

Correctness of Prim’s Algorithm Prim’s Algorithm Pick edge with minimum attachment cost to current tree, and add to current tree. Proof of correctness. If e is added to tree, then e is safe and belongs to every MST . 1 Let S be the vertices connected by edges in T when e is added. 1 e is edge of lowest cost with one end in S and the other in 2 V \ S and hence e is safe. Set of edges output is a spanning tree 2 Set of edges output forms a connected graph: by induction, S is 1 connected in each iteration and eventually S = V . Chandra & Lenny (UIUC) CS374 25 Spring 2015 25 / 61

Correctness of Prim’s Algorithm Prim’s Algorithm Pick edge with minimum attachment cost to current tree, and add to current tree. Proof of correctness. If e is added to tree, then e is safe and belongs to every MST . 1 Let S be the vertices connected by edges in T when e is added. 1 e is edge of lowest cost with one end in S and the other in 2 V \ S and hence e is safe. Set of edges output is a spanning tree 2 Set of edges output forms a connected graph: by induction, S is 1 connected in each iteration and eventually S = V . Only safe edges added and they do not have a cycle 2 Chandra & Lenny (UIUC) CS374 25 Spring 2015 25 / 61

Correctness of Kruskal’s Algorithm Kruskal’s Algorithm Pick edge of lowest cost and add if it does not form a cycle with existing edges. Proof of correctness. If e = (u , v) is added to tree, then e is safe 1 Set of edges output is a spanning tree : exercise 2 Chandra & Lenny (UIUC) CS374 26 Spring 2015 26 / 61

Correctness of Kruskal’s Algorithm Kruskal’s Algorithm Pick edge of lowest cost and add if it does not form a cycle with existing edges. Proof of correctness. If e = (u , v) is added to tree, then e is safe 1 When algorithm adds e let S and S ’ be the connected 1 components containing u and v respectively Set of edges output is a spanning tree : exercise 2 Chandra & Lenny (UIUC) CS374 26 Spring 2015 26 / 61

Correctness of Kruskal’s Algorithm Kruskal’s Algorithm Pick edge of lowest cost and add if it does not form a cycle with existing edges. Proof of correctness. If e = (u , v) is added to tree, then e is safe 1 When algorithm adds e let S and S ’ be the connected 1 components containing u and v respectively e is the lowest cost edge crossing S (and also S ’). 2 Set of edges output is a spanning tree : exercise 2 Chandra & Lenny (UIUC) CS374 26 Spring 2015 26 / 61

Correctness of Kruskal’s Algorithm Kruskal’s Algorithm Pick edge of lowest cost and add if it does not form a cycle with existing edges. Proof of correctness. If e = (u , v) is added to tree, then e is safe 1 When algorithm adds e let S and S ’ be the connected 1 components containing u and v respectively e is the lowest cost edge crossing S (and also S ’). 2 If there is an edge e ′ crossing S and has lower cost than e , then 3 e ′ would come before e in the sorted order and would be added by the algorithm to T Set of edges output is a spanning tree : exercise 2 Chandra & Lenny (UIUC) CS374 26 Spring 2015 26 / 61

Correctness of Bor˚ uvka’s Algorithm Proof of correctness. Argue that only safe edges are added. Chandra & Lenny (UIUC) CS374 27 Spring 2015 27 / 61

Correctness of Reverse Delete Algorithm Reverse Delete Algorithm Consider edges in decreasing cost and remove an edge if it does not disconnect the graph Proof of correctness. Argue that only unsafe edges are removed. Chandra & Lenny (UIUC) CS374 28 Spring 2015 28 / 61

When edge costs are not distinct Heuristic argument: Make edge costs distinct by adding a small tiny and different cost to each edge Chandra & Lenny (UIUC) CS374 29 Spring 2015 29 / 61

When edge costs are not distinct Heuristic argument: Make edge costs distinct by adding a small tiny and different cost to each edge Formal argument: Order edges lexicographically to break ties e i ≺ e j if either c(e i ) < c(e j ) or ( c(e i ) = c(e j ) and i < j ) 1 Lexicographic ordering extends to sets of edges. If A , B ⊆ E , 2 A � = B then A ≺ B if either c(A) < c(B) or ( c(A) = c(B) and A \ B has a lower indexed edge than B \ A ) Can order all spanning trees according to lexicographic order of 3 their edge sets. Hence there is a unique MST . Chandra & Lenny (UIUC) CS374 29 Spring 2015 29 / 61

When edge costs are not distinct Heuristic argument: Make edge costs distinct by adding a small tiny and different cost to each edge Formal argument: Order edges lexicographically to break ties e i ≺ e j if either c(e i ) < c(e j ) or ( c(e i ) = c(e j ) and i < j ) 1 Lexicographic ordering extends to sets of edges. If A , B ⊆ E , 2 A � = B then A ≺ B if either c(A) < c(B) or ( c(A) = c(B) and A \ B has a lower indexed edge than B \ A ) Can order all spanning trees according to lexicographic order of 3 their edge sets. Hence there is a unique MST . Chandra & Lenny (UIUC) CS374 29 Spring 2015 29 / 61

When edge costs are not distinct Heuristic argument: Make edge costs distinct by adding a small tiny and different cost to each edge Formal argument: Order edges lexicographically to break ties e i ≺ e j if either c(e i ) < c(e j ) or ( c(e i ) = c(e j ) and i < j ) 1 Lexicographic ordering extends to sets of edges. If A , B ⊆ E , 2 A � = B then A ≺ B if either c(A) < c(B) or ( c(A) = c(B) and A \ B has a lower indexed edge than B \ A ) Can order all spanning trees according to lexicographic order of 3 their edge sets. Hence there is a unique MST . Prim’s, Kruskal, and Reverse Delete Algorithms are optimal with respect to lexicographic ordering. Chandra & Lenny (UIUC) CS374 29 Spring 2015 29 / 61

Edge Costs: Positive and Negative Algorithms and proofs don’t assume that edge costs are 1 non-negative! MST algorithms work for arbitrary edge costs. Another way to see this: make edge costs non-negative by 2 adding to each edge a large enough positive number. Why does this work for MST s but not for shortest paths? Can compute maximum weight spanning tree by negating edge 3 costs and then computing an MST. Chandra & Lenny (UIUC) CS374 30 Spring 2015 30 / 61

Edge Costs: Positive and Negative Algorithms and proofs don’t assume that edge costs are 1 non-negative! MST algorithms work for arbitrary edge costs. Another way to see this: make edge costs non-negative by 2 adding to each edge a large enough positive number. Why does this work for MST s but not for shortest paths? Can compute maximum weight spanning tree by negating edge 3 costs and then computing an MST. Question: Why does this not work for shortest paths? Chandra & Lenny (UIUC) CS374 30 Spring 2015 30 / 61

Part II Data Structures for MST: Priority Queues and Union-Find Chandra & Lenny (UIUC) CS374 31 Spring 2015 31 / 61

Implementing Bor˚ uvka’s Algorithm No complex data structure needed. T is ∅ (* T will store edges of a MST *) while T is not spanning do X ← ∅ for each connected component S of T do add to X the cheapest edge between S and V \ S Add edges in X to T return the set T O(log n) iterations of while loop. Why? Chandra & Lenny (UIUC) CS374 32 Spring 2015 32 / 61

Implementing Bor˚ uvka’s Algorithm No complex data structure needed. T is ∅ (* T will store edges of a MST *) while T is not spanning do X ← ∅ for each connected component S of T do add to X the cheapest edge between S and V \ S Add edges in X to T return the set T O(log n) iterations of while loop. Why? Number of connected components shrink by at least half since each component merges with one or more other components. Each iteration can be implemented in O(m) time. Chandra & Lenny (UIUC) CS374 32 Spring 2015 32 / 61

Implementing Bor˚ uvka’s Algorithm No complex data structure needed. T is ∅ (* T will store edges of a MST *) while T is not spanning do X ← ∅ for each connected component S of T do add to X the cheapest edge between S and V \ S Add edges in X to T return the set T O(log n) iterations of while loop. Why? Number of connected components shrink by at least half since each component merges with one or more other components. Each iteration can be implemented in O(m) time. Running time: O(m log n) time. Chandra & Lenny (UIUC) CS374 32 Spring 2015 32 / 61

Implementing Prim’s Algorithm Implementing Prim’s Algorithm Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) while S � = V do pick e = (v , w) ∈ E such that v ∈ S and w ∈ V − S e has minimum cost T = T ∪ e S = S ∪ w return the set T Analysis Chandra & Lenny (UIUC) CS374 33 Spring 2015 33 / 61

Implementing Prim’s Algorithm Implementing Prim’s Algorithm Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) while S � = V do pick e = (v , w) ∈ E such that v ∈ S and w ∈ V − S e has minimum cost T = T ∪ e S = S ∪ w return the set T Analysis Number of iterations = O(n) , where n is number of vertices 1 Chandra & Lenny (UIUC) CS374 33 Spring 2015 33 / 61

Implementing Prim’s Algorithm Implementing Prim’s Algorithm Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) while S � = V do pick e = (v , w) ∈ E such that v ∈ S and w ∈ V − S e has minimum cost T = T ∪ e S = S ∪ w return the set T Analysis Number of iterations = O(n) , where n is number of vertices 1 Picking e is O(m) where m is the number of edges 2 Chandra & Lenny (UIUC) CS374 33 Spring 2015 33 / 61

Implementing Prim’s Algorithm Implementing Prim’s Algorithm Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) while S � = V do pick e = (v , w) ∈ E such that v ∈ S and w ∈ V − S e has minimum cost T = T ∪ e S = S ∪ w return the set T Analysis Number of iterations = O(n) , where n is number of vertices 1 Picking e is O(m) where m is the number of edges 2 Total time O(nm) 3 Chandra & Lenny (UIUC) CS374 33 Spring 2015 33 / 61

Implementing Prim’s Algorithm More Efficient Implementation Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) for v �∈ S , a(v) = min w ∈ S c(w , v) for v �∈ S , e(v) = w such that w ∈ S and c(w , v) is minimum while S � = V do pick v with minimum a(v) T = T ∪ { (e(v) , v) } S = S ∪ { v } update arrays a and e return the set T Chandra & Lenny (UIUC) CS374 34 Spring 2015 34 / 61

Implementing Prim’s Algorithm More Efficient Implementation Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) for v �∈ S , a(v) = min w ∈ S c(w , v) for v �∈ S , e(v) = w such that w ∈ S and c(w , v) is minimum while S � = V do pick v with minimum a(v) T = T ∪ { (e(v) , v) } S = S ∪ { v } update arrays a and e return the set T Chandra & Lenny (UIUC) CS374 34 Spring 2015 34 / 61

Implementing Prim’s Algorithm More Efficient Implementation Prim ComputeMST E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) for v �∈ S , a(v) = min w ∈ S c(w , v) for v �∈ S , e(v) = w such that w ∈ S and c(w , v) is minimum while S � = V do pick v with minimum a(v) T = T ∪ { (e(v) , v) } S = S ∪ { v } update arrays a and e return the set T Maintain vertices in V \ S in a priority queue with key a(v) . Chandra & Lenny (UIUC) CS374 34 Spring 2015 34 / 61

Priority Queues Data structure to store a set S of n elements where each element v ∈ S has an associated real/integer key k(v) such that the following operations makeQ : create an empty queue 1 findMin : find the minimum key in S 2 extractMin : Remove v ∈ S with smallest key and return it 3 add ( v , k(v) ): Add new element v with key k(v) to S 4 Delete ( v ): Remove element v from S 5 decreaseKey ( v , k ′ (v) ): decrease key of v from k(v) (current 6 key) to k ′ (v) (new key). Assumption: k ′ (v) ≤ k(v) meld : merge two separate priority queues into one 7 Chandra & Lenny (UIUC) CS374 35 Spring 2015 35 / 61

Prim’s using priority queues E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) for v �∈ S , a(v) = min w ∈ S c(w , v) for v �∈ S , e(v) = w such that w ∈ S and c(w , v) is minimum while S � = V do pick v with minimum a(v) T = T ∪ { (e(v) , v) } S = S ∪ { v } update arrays a and e return the set T Maintain vertices in V \ S in a priority queue with key a(v) Chandra & Lenny (UIUC) CS374 36 Spring 2015 36 / 61

Prim’s using priority queues E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) for v �∈ S , a(v) = min w ∈ S c(w , v) for v �∈ S , e(v) = w such that w ∈ S and c(w , v) is minimum while S � = V do pick v with minimum a(v) T = T ∪ { (e(v) , v) } S = S ∪ { v } update arrays a and e return the set T Maintain vertices in V \ S in a priority queue with key a(v) Requires O(n) extractMin operations 1 Chandra & Lenny (UIUC) CS374 36 Spring 2015 36 / 61

Prim’s using priority queues E is the set of all edges in G S = { 1 } T is empty (* T will store edges of a MST *) for v �∈ S , a(v) = min w ∈ S c(w , v) for v �∈ S , e(v) = w such that w ∈ S and c(w , v) is minimum while S � = V do pick v with minimum a(v) T = T ∪ { (e(v) , v) } S = S ∪ { v } update arrays a and e return the set T Maintain vertices in V \ S in a priority queue with key a(v) Requires O(n) extractMin operations 1 Requires O(m) decreaseKey operations 2 Chandra & Lenny (UIUC) CS374 36 Spring 2015 36 / 61

Running time of Prim’s Algorithm O(n) extractMin operations and O(m) decreaseKey operations Using standard Heaps, extractMin and decreaseKey take 1 O(log n) time. Total: O((m + n) log n) Using Fibonacci Heaps, O(log n) for extractMin and O(1) 2 (amortized) for decreaseKey . Total: O(n log n + m) . Chandra & Lenny (UIUC) CS374 37 Spring 2015 37 / 61

Running time of Prim’s Algorithm O(n) extractMin operations and O(m) decreaseKey operations Using standard Heaps, extractMin and decreaseKey take 1 O(log n) time. Total: O((m + n) log n) Using Fibonacci Heaps, O(log n) for extractMin and O(1) 2 (amortized) for decreaseKey . Total: O(n log n + m) . Prim’s algorithm and Dijkstra’s algorithms are similar. Where is the difference? Chandra & Lenny (UIUC) CS374 37 Spring 2015 37 / 61

Kruskal’s Algorithm Kruskal ComputeMST Initially E is the set of all edges in G T is empty (* T will store edges of a MST *) while E is not empty do choose e ∈ E of minimum cost if ( T ∪ { e } does not have cycles) add e to T return the set T Chandra & Lenny (UIUC) CS374 38 Spring 2015 38 / 61

Kruskal’s Algorithm Kruskal ComputeMST Initially E is the set of all edges in G T is empty (* T will store edges of a MST *) while E is not empty do choose e ∈ E of minimum cost if ( T ∪ { e } does not have cycles) add e to T return the set T Chandra & Lenny (UIUC) CS374 38 Spring 2015 38 / 61

Kruskal’s Algorithm Kruskal ComputeMST Initially E is the set of all edges in G T is empty (* T will store edges of a MST *) while E is not empty do choose e ∈ E of minimum cost if ( T ∪ { e } does not have cycles) add e to T return the set T Presort edges based on cost. Choosing minimum can be done in 1 O(1) time Chandra & Lenny (UIUC) CS374 38 Spring 2015 38 / 61

Kruskal’s Algorithm Kruskal ComputeMST Initially E is the set of all edges in G T is empty (* T will store edges of a MST *) while E is not empty do choose e ∈ E of minimum cost if ( T ∪ { e } does not have cycles) add e to T return the set T Presort edges based on cost. Choosing minimum can be done in 1 O(1) time Chandra & Lenny (UIUC) CS374 38 Spring 2015 38 / 61

Kruskal’s Algorithm Kruskal ComputeMST Initially E is the set of all edges in G T is empty (* T will store edges of a MST *) while E is not empty do choose e ∈ E of minimum cost if ( T ∪ { e } does not have cycles) add e to T return the set T Presort edges based on cost. Choosing minimum can be done in 1 O(1) time Do BFS / DFS on T ∪ { e } . Takes O(n) time 2 Chandra & Lenny (UIUC) CS374 38 Spring 2015 38 / 61

Kruskal’s Algorithm Kruskal ComputeMST Initially E is the set of all edges in G T is empty (* T will store edges of a MST *) while E is not empty do choose e ∈ E of minimum cost if ( T ∪ { e } does not have cycles) add e to T return the set T Presort edges based on cost. Choosing minimum can be done in 1 O(1) time Do BFS / DFS on T ∪ { e } . Takes O(n) time 2 Total time O(m log m) + O(mn) = O(mn) 3 Chandra & Lenny (UIUC) CS374 38 Spring 2015 38 / 61

Implementing Kruskal’s Algorithm Efficiently Kruskal ComputeMST Sort edges in E based on cost T is empty (* T will store edges of a MST *) each vertex u is placed in a set by itself while E is not empty do pick e = (u , v) ∈ E of minimum cost if u and v belong to different sets add e to T merge the sets containing u and v return the set T Chandra & Lenny (UIUC) CS374 39 Spring 2015 39 / 61

Implementing Kruskal’s Algorithm Efficiently Kruskal ComputeMST Sort edges in E based on cost T is empty (* T will store edges of a MST *) each vertex u is placed in a set by itself while E is not empty do pick e = (u , v) ∈ E of minimum cost if u and v belong to different sets add e to T merge the sets containing u and v return the set T Chandra & Lenny (UIUC) CS374 39 Spring 2015 39 / 61

Implementing Kruskal’s Algorithm Efficiently Kruskal ComputeMST Sort edges in E based on cost T is empty (* T will store edges of a MST *) each vertex u is placed in a set by itself while E is not empty do pick e = (u , v) ∈ E of minimum cost if u and v belong to different sets add e to T merge the sets containing u and v return the set T Need a data structure to check if two elements belong to same set and to merge two sets. Chandra & Lenny (UIUC) CS374 39 Spring 2015 39 / 61

MST for really sparse graphs? Given a graph G with n vertices, and n + 20 edges, its MST can be computed in (A) O(n 2 ) . (B) O(n log n) . (C) O(n log log n) . (D) O(n log ∗ n) . (E) O(n) . Chandra & Lenny (UIUC) CS374 40 Spring 2015 40 / 61