Mat 3770 Prim Kruskal Heaps Heapsort Method 1 Method 2 Spring - PowerPoint PPT Presentation

Mat 3770 Week 9 Spanning Trees Mat 3770 Prim Kruskal Heaps Heapsort Method 1 Method 2 Spring 2014 Disjoint Sets

Week 9 — Student Responsibilities Mat 3770 Reading: Chapter 3.3–3.4 (Tucker), 10.4–10.5 (Rosen) Week 9 Spanning Homework Trees Prim Due date Tucker Rosen Kruskal 3/21 3.2 10.3 Heaps Heapsort 3/21 DFS & BFS Worksheets Method 1 3/26 3.3 10.4, 10.5 Method 2 Disjoint Sets 3/28 Heapify worksheet Attendance Truly, Madly, Deeply Encouraged

3.3 Spanning Trees Mat 3770 A spanning tree of a graph G is a subgraph of G that is a tree containing all vertices of G. Week 9 Spanning Trees A minimal spanning tree is a spanning tree whose sum of Prim the edge weights (lengths) is as small as possible. Kruskal Heaps Heapsort Problem Statement : Given a graph G = (V, E) with Method 1 positive edge weights (cost: E → ℜ + ), find the cheapest Method 2 connected spanning subgraph H of G. Disjoint Sets Note : If H = (V, E H ), then cost(H) = � e ∈ E ( H ) cost(e), i.e., cost of subgraph is sum of costs of edges in subgraph.

Example: Minimal Spanning Tree Mat 3770 E 3 B Week 9 Spanning 4 Trees 3 Prim 5 Kruskal 4 D Heaps A 6 5 Heapsort Method 1 8 Method 2 4 F 6 Disjoint Sets C

Observations Mat 3770 H must be a tree (if H exists). Why? Week 9 1. must span and be connected Spanning Trees 2. if cycle, then extra edge with positive weight, which could be Prim removed to reduce cost Kruskal Heaps Heapsort If G has n vertices ( | V | = n), then any minimal spanning Method 1 tree of G has N − 1 edges. Method 2 Disjoint Sets A graph with no cycles is called a forest A connected forest is called a tree

Prim’s Minimal Spanning Tree Mat 3770 Idea: “Grow” a Tree Week 9 1. Pick an arbitrary vertex in the graph, place in V H Spanning Trees Prim 2. From among the edges going from V H to vertices not in V H , Kruskal choose a cheapest one, say edge e to vertex x Heaps Heapsort Method 1 3. Add vertex x to V H and e to E H Method 2 Disjoint Sets 4. Repeat process from step 2 until no more vertices remain to be added to V H , which is equivalent to saying | E H | = | V G | − 1

Prim’s Minimal Spanning Tree, Start at A Mat 3770 A A Week 9 8 5 Spanning B B C C Trees 10 Prim D D 2 5 Kruskal Heaps 18 3 30 Heapsort 12 Method 1 14 E F E F Method 2 Disjoint Sets 4 26 G G

Prim’s MST Algorithm Mat 3770 Procedure Prim (G): H Week 9 // PRE: G is connected Spanning // POST: H is an MST of G Trees Prim begin Kruskal pick an arbitrary vertex x in the vertices of G, Heaps and add it to VH, the vertices in H Heapsort from among the edges incident to x, select the Method 1 cheapest and add it to EH, the edges in H Method 2 while |EH| < |VG| - 1 Disjoint Sets find the cheapest edge <a, b> where a is in VH, and b is in VG - VH add <a, b> to EH, add b to VH end

Kruskal’s MST Mat 3770 Idea: connect forests until one tree Week 9 Spanning 1. Place the vertices of V G into | V G | = n individual subtrees Trees Prim Kruskal 2. Find the minimum cost edge, e ∈ E G , which doesn’t cause a Heaps cycle in the spanning forest Heapsort Method 1 3. Add e to E H , joining two of the subtrees Method 2 Disjoint Sets 4. repeat process from step 2 until a single spanning tree exists, which is equivalent to saying | E H | = | V G | − 1

Kruskal’s MST Mat 3770 A A Week 9 8 5 B B C C Spanning 10 Trees D D Prim 2 5 Kruskal 18 3 Heaps 30 Heapsort 12 14 Method 1 E F E F Method 2 4 26 Disjoint Sets G G Edge weights: 2, 3, 4, 5, 5, 8, 10, 12, 14, 18, 26, and 30

Kruskal’s MST Algorithm Mat 3770 Procedure Kruskal (G): H // PRE: G is connected Week 9 // POST: H is an MST of G Spanning Trees begin Prim put n vertices into n singleton trees Kruskal H = { } Heaps Heapsort Method 1 // repeat until tree has n-1 edges Method 2 while |EH| < |VG| - 1 Disjoint Sets a) find the min_cost_edge e in EG b) if H remains a forest when e is added add e to VH c) delete e from EG end

Implementing Kruskal’s Algorithm Mat 3770 Week 9 We need to be able to ( quickly ) find the next cheapest Spanning edge Trees Prim Kruskal Using a min-heap vs sorted list Heaps Heapsort Heap is better since not every edge may be examined / Method 1 removed Method 2 Disjoint Sets But, what’s a heap?

Priority Queues and Heaps Mat 3770 A Priority Queue is a data structure supporting the operations: Week 9 Spanning 1. insert() and Trees 2. removeMin() (or removeMax() , depending upon the Prim problem) Kruskal Heaps Heapsort One way to implement priority queues is with a heap Method 1 Method 2 Disjoint Sets A heap is an essentially complete binary tree, i.e.: 1. all levels are full except possibly bottom level (leaves) 2. all bottom nodes are in left–most positions.

Heap Implementation Heaps can be efficiently implemented with arrays, which form Mat 3770 an implicit (vs explicit) representation of a tree. For example: Week 9 i = 1 2 3 4 5 6 7 8 9 10 11 12 Spanning Trees L 20 10 15 8 7 14 3 5 6 4 2 1 Prim Kruskal Where Children of L[i] are in positions 2*i and (2*i)+1. Heaps Heapsort Method 1 20 Method 2 Disjoint Sets 10 15 8 7 14 3 5 6 4 2 1

The Min–heap Property Mat 3770 An array L[k..n] has the Min–heap property if Week 9 Spanning Trees ∀ i ∋ k ≤ i < n Prim 2 : L [ i ] ≤ L [2 i ] and L [ i ] ≤ L [2 i + 1] Kruskal Heaps L [ n if n is even, then 2 ] ≤ L [ n ] Heapsort Method 1 Method 2 Disjoint Sets In other words: Parents are smaller than their children , or child in the case n is even.

removeMin() (Max) Algorithm Mat 3770 Send out the first value in heap as min (max) Week 9 Spanning Trees Put last value (x) of heap in position 1: L [1] = L [ size ] Prim Kruskal Heaps Decrement heap size Heapsort Method 1 Method 2 Trickle–down x through heap by swapping it with the smaller Disjoint Sets (larger) child until smaller (larger) child is larger (smaller) than x.

Example — removeMax() Mat 3770 20 Week 9 10 15 Spanning Trees 8 7 14 3 Prim Kruskal 5 6 4 2 1 Heaps Heapsort 1 max is deleted, Method 1 last child is moved up, Method 2 and trickled down 10 15 Disjoint Sets 8 7 14 3 5 6 4 2

Insert() Algorithm Mat 3770 Increment heap size Week 9 Put new value (x) in L[size] Spanning Trees While x is smaller (bigger) than its parent, swap them (aka Prim percolate– or bubble–up) Kruskal Heaps Heapsort 15 Method 1 Method 2 10 14 Disjoint Sets 1 8 7 3 5 6 4 2 17

Complexity of Heap Operations Mat 3770 Observation: if a heap of height h has n nodes, then ≤ 2 h +1 2 h ≤ n Week 9 Spanning one leaf at level h versus a complete tree Trees Prim Take the log of each part of the inequality: Kruskal Heaps ≤ log n ≤ h + 1 h Heapsort Method 1 Subtracting 1, we find: Method 2 Disjoint Sets log n − 1 ≤ ≤ log n h Hence, to traverse the heap from root to leaf, or in reverse, takes O (log n ) time. Thus, both Insert() and Delete() take O (log n ) time.

What if we merely kept an unordered list? Mat 3770 Delete: find, delete item, and fill in array: O ( n ) + O (1) + O (1) = O ( n ) Week 9 Spanning linked list: O ( n ) + O (1) = O ( n ) Trees Prim Insert: array / linked list: O (1) Kruskal Heaps An ordered list? Heapsort Method 1 Delete: find, delete item, and fill in Method 2 array: O (1) + O (1) + O ( n ) = O ( n ) Disjoint Sets linked list: O (1) + O (1) = O (1) Insert: find position, insert (move) array: O (log n ) + O ( n ) = O ( n ) linked list: O ( n ) + O (1) = O ( n )

An Aside: Heapsort Mat 3770 An array, L, can be sorted as follows: Week 9 1. Turn L[1..n] into a heap (aka heapify ) Spanning 2. Remove() n times, storing the removed (min or max) value at Trees the end of the heap, then decrement size of heap Prim Kruskal Heaps How fast is this sort? Heapsort Method 1 Step 2 takes time: Method 2 � Disjoint Sets log ( n )+log ( n − 1)+ · · · +log (1) = log i ∈ O ( n log n ) i =1 .. n Step 1? It depends . . .

Heapifying — Method 1: Top–down Mat 3770 for i = 2 to n BubbleUp(L[1..i]) Week 9 // invariant: L[1..i] is a heap Spanning Trees (Max)–Heapify: 1, 2, 3, 4, 5, 6, 7, 8, 10, 14, 15, 20 Prim Kruskal Heaps Heapsort Method 1 Method 2 Disjoint Sets

Method 1 Complexity Analysis Mat 3770 Complexity? All nodes at depth j take j swaps in worst case, so: � Week 9 T ( n ) = j × number of nodes at depth j Spanning j =0 .. h Trees We have at most 2 j nodes at depth j , so: Prim Kruskal ( h − 1)2 h +1 + 2 � ( j × 2 j ) T ( n ) ≤ = Heaps j =0 .. h Heapsort Method 1 We know h ≤ log n , so: Method 2 (log ( n − 1))2 log n +1 + 2 T ( n ) ≤ Disjoint Sets ≤ log n ( n ) + 2 ≤ n log n + 2 ∈ O ( n log n )

Can We Create Heaps Any Faster? Mat 3770 Week 9 Method 1 : Top–down moves many (about n 2 ) elements by Spanning log n in the worst case (if already in order, all n 2 last inserts Trees must percolate to the top!) Prim Kruskal Heaps Heapsort Consider Method 2 : Bottom–up , where we assume the Method 1 leaves are in place and use sift–down on the top n 2 elements. Method 2 Disjoint Sets This moves fewer elements by the height of the tree.