Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 29/103
8 13 b c d 2 9 5 4 14 11 a i e 7 6 12 10 h g f 1 3 Q: Which edge can be safely included in the MST? A: The edge with the smallest weight (lightest edge). 30/103
Lemma It is safe to include the lightest edge: there is a minimum spanning tree, that contains the lightest edge. Proof. Take a minimum spanning tree T Assume the lightest edge e ∗ is not in T There is a unique path in T connecting u and v Remove any edge e in the path to obtain tree T ′ ⇒ w ( T ′ ) ≤ w ( T ) : T ′ is also a MST w ( e ∗ ) ≤ w ( e ) = lightest edge e ∗ v u 31/103
Is the Residual Problem Still a MST Problem? 13 8 b c d 2 5 9 4 14 11 a i e 7 6 12 10 g ∗ h g f 1 3 Residual problem: find the minimum spanning tree that contains edge ( g, h ) Contract the edge ( g, h ) Residual problem: find the minimum spanning tree in the contracted graph 32/103
Contraction of an Edge ( u, v ) 13 8 b c d 2 9 5 4 14 11 a i e 7 6 12 10 g ∗ h g f 1 3 Remove u and v from the graph, and add a new vertex u ∗ Remove all edges parallel connecting u to v from E For every edge ( u, w ) ∈ E, w � = v , change it to ( u ∗ , w ) For every edge ( v, w ) ∈ E, w � = u , change it to ( u ∗ , w ) May create parallel edges! E.g. : two edges ( i, g ∗ ) 33/103
Greedy Algorithm Repeat the following step until G contains only one vertex: Choose the lightest edge e ∗ , add e ∗ to the spanning tree 1 Contract e ∗ and update G be the contracted graph 2 Q: What edges are removed due to contractions? A: Edge ( u, v ) is removed if and only if there is a path connecting u and v formed by edges we selected 34/103
Greedy Algorithm MST-Greedy ( G, w ) F = ∅ 1 sort edges in E in non-decreasing order of weights w 2 for each edge ( u, v ) in the order 3 if u and v are not connected by a path of edges in F 4 F = F ∪ { ( u, v ) } 5 return ( V, F ) 6 35/103
Kruskal’s Algorithm: Example 13 8 b c d 2 5 9 4 14 11 a i e 7 6 12 10 h g f 1 3 Sets: { a, b, c, i, f, g, h, d, e } 36/103
Kruskal’s Algorithm: Efficient Implementation of Greedy Algorithm MST-Kruskal( G , w ) F ← ∅ 1 S ← {{ v } : v ∈ V } 2 sort the edges of E in non-decreasing order of weights w 3 for each edge ( u, v ) ∈ E in the order 4 S u ← the set in S containing u 5 S v ← the set in S containing v 6 if S u � = S v 7 F ← F ∪ { ( u, v ) } 8 S ← S \ { S u } \ { S v } ∪ { S u ∪ S v } 9 10 return ( V, F ) 37/103
Running Time of Kruskal’s Algorithm MST-Kruskal( G , w ) F ← ∅ 1 S ← {{ v } : v ∈ V } 2 sort the edges of E in non-decreasing order of weights w 3 for each edge ( u, v ) ∈ E in the order 4 S u ← the set in S containing u 5 S v ← the set in S containing v 6 if S u � = S v 7 F ← F ∪ { ( u, v ) } 8 S ← S \ { S u } \ { S v } ∪ { S u ∪ S v } 9 10 return ( V, F ) Use union-find data structure to support 2 , 5 , 6 , 7 , 9 . 38/103
Union-Find Data Structure V : ground set We need to maintain a partition of V and support following operations: Check if u and v are in the same set of the partition Merge two sets in partition 39/103
V = { 1 , 2 , 3 , · · · , 16 } Partition: { 2 , 3 , 5 , 9 , 10 , 12 , 15 } , { 1 , 7 , 13 , 16 } , { 4 , 8 , 11 } , { 6 , 14 } 7 8 6 3 4 11 1 14 16 10 2 13 12 9 15 5 par [ i ] : parent of i , ( par [ i ] = nil if i is a root). 40/103
Union-Find Data Structure 7 8 6 3 4 11 1 14 16 10 2 13 12 9 15 5 Q: how can we check if u and v are in the same set? A: Check if root( u ) = root( v ). root( u ): the root of the tree containing u Merge the trees with root r and r ′ : par [ r ] ← r ′ . 41/103
Union-Find Data Structure root( v ) root( v ) if par [ v ] = nil then 1 if par [ v ] = nil then 1 return v 2 return v 2 else 3 else 3 par [ v ] ← root( par [ v ] ) 4 return root( par [ v ] ) 4 return par [ v ] 5 Problem: the tree might too deep; running time might be large Improvement: all vertices in the path directly point to the root, saving time in the future. 42/103
Union-Find Data Structure root( v ) if par [ v ] = nil then 1 return v 2 else 3 par [ v ] ← root( par [ v ] ) 4 return par [ v ] 5 7 8 6 3 11 4 1 14 16 10 2 13 9 12 15 5 43/103
MST-Kruskal( G , w ) F ← ∅ 1 S ← {{ v } : v ∈ V } 2 sort the edges of E in non-decreasing order of weights w 3 for each edge ( u, v ) ∈ E in the order 4 S u ← the set in S containing u 5 S v ← the set in S containing v 6 if S u � = S v 7 F ← F ∪ { ( u, v ) } 8 S ← S \ { S u } \ { S v } ∪ { S u ∪ S v } 9 10 return ( V, F ) 44/103
MST-Kruskal( G , w ) F ← ∅ 1 for every v ∈ V : let par [ v ] ← nil 2 sort the edges of E in non-decreasing order of weights w 3 for each edge ( u, v ) ∈ E in the order 4 u ′ ← root ( u ) 5 v ′ ← root ( v ) 6 if u ′ � = v ′ 7 F ← F ∪ { ( u, v ) } 8 par [ u ′ ] ← v ′ 9 10 return ( V, F ) 2 , 5 , 6 , 7 , 9 takes time O ( mα ( n )) α ( n ) is very slow-growing: α ( n ) ≤ 4 for n ≤ 10 80 . Running time = time for 3 = O ( m lg n ) . 45/103
Assumption Assume all edge weights are different. Lemma An edge e ∈ E is not in the MST, if and only if there is cycle C in G in which e is the heaviest edge. 13 8 b c d 2 9 5 4 14 11 a i e 7 6 12 10 h g f 1 3 ( i, g ) is not in the MST because of cycle ( i, c, f, g ) ( e, f ) is in the MST because no such cycle exists 46/103
Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 47/103
Two Methods to Build a MST Start from F ← ∅ , and add edges to F one by one until we 1 obtain a spanning tree Start from F ← E , and remove edges from F one by one 2 until we obtain a spanning tree 8 b c d 2 5 9 4 a i e 7 6 10 h g f 1 3 48/103
Lemma It is safe to exclude the heaviest non-bridge edge: there is a MST that does not contain the heaviest non-bridge edge. 49/103
Reverse Kruskal’s Algorithm MST-Greedy ( G, w ) F ← E 1 sort E in non-increasing order of weights 2 for every e in this order 3 if ( V, F \ { e } ) is connected then 4 F ← F \ { e } 5 return ( V, F ) 6 50/103
Reverse Kruskal’s Algorithm: Example 8 b c d 2 5 9 4 a i e 10 h g f 1 3 51/103
Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 52/103
Design Greedy Strategy for MST Recall the greedy strategy for Kruskal’s algorithm: choose the edge with the smallest weight. 13 8 b c d 2 9 5 4 14 11 a i e 7 6 12 10 h g f 1 3 Greedy strategy for Prim’s algorithm: choose the lightest edge incident to a . 53/103
Lemma It is safe to include the lightest edge incident to a . lightest edge e ∗ incident to a C a Proof. Let T be a MST Consider all components obtained by removing a from T Let e ∗ be the lightest edge incident to a and e ∗ connects a to component C Let e be the edge in T connecting a to C T ′ = T \ e ∪ { e ∗ } is a spanning tree with w ( T ′ ) ≤ w ( T ) 54/103
Prim’s Algorithm: Example 13 8 b c d 2 5 9 4 14 11 a i e 7 6 12 10 h g f 1 3 55/103
Greedy Algorithm MST-Greedy1( G, w ) S ← { s } , where s is arbitrary vertex in V 1 F ← ∅ 2 while S � = V 3 ( u, v ) ← lightest edge between S and V \ S , 4 where u ∈ S and v ∈ V \ S S ← S ∪ { v } 5 F ← F ∪ { ( u, v ) } 6 return ( V, F ) 7 Running time of naive implementation: O ( nm ) 56/103
Prim’s Algorithm: Efficient Implementation of Greedy Algorithm For every v ∈ V \ S maintain d ( v ) = min u ∈ S :( u,v ) ∈ E w ( u, v ) : the weight of the lightest edge between v and S π ( v ) = arg min u ∈ S :( u,v ) ∈ E w ( u, v ) : ( π ( v ) , v ) is the lightest edge between v and S (13 , c ) 8 13 b c d 2 9 5 (10 , f ) 4 14 11 a i e 7 6 12 10 h g f 1 3 (3 , f ) (7 , i ) 57/103
Prim’s Algorithm: Efficient Implementation of Greedy Algorithm For every v ∈ V \ S maintain d ( v ) = min u ∈ S :( u,v ) ∈ E w ( u, v ) : the weight of the lightest edge between v and S π ( v ) = arg min u ∈ S :( u,v ) ∈ E w ( u, v ) : ( π ( v ) , v ) is the lightest edge between v and S In every iteration Pick u ∈ V \ S with the smallest d ( u ) value Add ( π ( u ) , u ) to F Add u to S , update d and π values. 58/103
Prim’s Algorithm MST-Prim( G, w ) s ← arbitrary vertex in G 1 S ← ∅ , d ( s ) ← 0 and d ( v ) ← ∞ for every v ∈ V \ { s } 2 while S � = V , do 3 u ← vertex in V \ S with the minimum d ( u ) 4 S ← S ∪ { u } 5 for each v ∈ V \ S such that ( u, v ) ∈ E 6 if w ( u, v ) < d ( v ) then 7 d ( v ) ← w ( u, v ) 8 π ( v ) ← u 9 10 return � � ( u, π ( u )) | u ∈ V \ { s } 59/103
Example 13 8 b c d 2 9 5 4 14 11 a i e 7 6 12 10 h g f 1 3 60/103
Prim’s Algorithm For every v ∈ V \ S maintain d ( v ) = min u ∈ S :( u,v ) ∈ E w ( u, v ) : the weight of the lightest edge between v and S π ( v ) = arg min u ∈ S :( u,v ) ∈ E w ( u, v ) : ( π ( v ) , v ) is the lightest edge between v and S In every iteration Pick u ∈ V \ S with the smallest d ( u ) value extract min Add ( π ( u ) , u ) to F Add u to S , update d and π values. decrease key Use a priority queue to support the operations 61/103
Def. A priority queue is an abstract data structure that maintains a set U of elements, each with an associated key value, and supports the following operations: insert ( v, key value ) : insert an element v , whose associated key value is key value . decrease key ( v, new key value ) : decrease the key value of an element v in queue to new key value extract min () : return and remove the element in queue with the smallest key value · · · 62/103
Prim’s Algorithm MST-Prim( G, w ) s ← arbitrary vertex in G 1 S ← ∅ , d ( s ) ← 0 and d ( v ) ← ∞ for every v ∈ V \ { s } 2 3 while S � = V , do 4 u ← vertex in V \ S with the minimum d ( u ) 5 S ← S ∪ { u } 6 for each v ∈ V \ S such that ( u, v ) ∈ E 7 if w ( u, v ) < d ( v ) then 8 d ( v ) ← w ( u, v ) 9 π ( v ) ← u 10 11 return � � ( u, π ( u )) | u ∈ V \ { s } 63/103
Prim’s Algorithm Using Priority Queue MST-Prim( G, w ) s ← arbitrary vertex in G 1 S ← ∅ , d ( s ) ← 0 and d ( v ) ← ∞ for every v ∈ V \ { s } 2 Q ← empty queue, for each v ∈ V : Q. insert ( v, d ( v )) 3 while S � = V , do 4 u ← Q. extract min() 5 S ← S ∪ { u } 6 for each v ∈ V \ S such that ( u, v ) ∈ E 7 if w ( u, v ) < d ( v ) then 8 d ( v ) ← w ( u, v ) , Q. decrease key ( v, d ( v )) 9 π ( v ) ← u 10 11 return � � ( u, π ( u )) | u ∈ V \ { s } 64/103
Running Time of Prim’s Algorithm Using Priority Queue O ( n ) × (time for extract min) + O ( m ) × (time for decrease key) concrete DS extract min decrease key overall time heap O (log n ) O (log n ) O ( m log n ) Fibonacci heap O (log n ) O (1) O ( n log n + m ) 65/103
Assumption Assume all edge weights are different. Lemma ( u, v ) is in MST, if and only if there exists a cut ( U, V \ U ) , such that ( u, v ) is the lightest edge between U and V \ U . 8 13 b c d 2 5 9 4 14 11 a i e 7 6 12 10 h g f 1 3 � � ( c, f ) is in MST because of cut { a, b, c, i } , V \ { a, b, c, i } ( i, g ) is not in MST because no such cut exists 66/103
“Evidence” for e ∈ MST or e / ∈ MST Assumption Assume all edge weights are different. e ∈ MST ↔ there is a cut in which e is the lightest edge e / ∈ MST ↔ there is a cycle in which e is the heaviest edge Exactly one of the following is true: There is a cut in which e is the lightest edge There is a cycle in which e is the heaviest edge Thus, the minimum spanning tree is unique with assumption. 67/103
Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 68/103
s - t Shortest Paths Input: (directed or undirected) graph G = ( V, E ) , s, t ∈ V w : E → R ≥ 0 Output: shortest path from s to t 1 16 s 4 1 5 3 2 4 10 t 3 3 3 69/103
Single Source Shortest Paths Input: directed graph G = ( V, E ) , s ∈ V w : E → R ≥ 0 Output: shortest paths from s to all other vertices v ∈ V Reason for Considering Single Source Shortest Paths Problem We do not know how to solve s - t shortest path problem more efficiently than solving single source shortest path problem Shortest paths in directed graphs is more general than in undirected graphs: we can replace every undirected edge with two anti-parallel edges of the same weight 70/103
Shortest path from s to v may contain Ω( n ) edges There are Ω( n ) different vertices v Thus, printing out all shortest paths may take time Ω( n 2 ) Not acceptable if graph is sparse 71/103
Shortest Path Tree O ( n ) -size data structure to represent all shortest paths For every vertex v , we only need to remember the parent of v : second-to-last vertex in the shortest path from s to v (why?) 2 7 5 a b 6 2 8 9 4 0 13 10 16 s c d 12 1 3 4 5 7 14 4 7 e t 3 f 72/103
Single Source Shortest Paths Input: directed graph G = ( V, E ) , s ∈ V w : E → R ≥ 0 Output: π ( v ) , v ∈ V \ s : the parent of v d ( v ) , v ∈ V \ s : the length of shortest path from s to v 73/103
Q: How to compute shortest paths from s when all edges have weight 1? A: Breadth first search (BFS) from source s 1 7 2 3 4 5 8 6 74/103
Assumption Weights w ( u, v ) are integers (w.l.o.g). An edge of weight w ( u, v ) is equivalent to a pah of w ( u, v ) unit-weight edges 4 1 1 1 1 u v u v Shortest Path Algorithm by Running BFS replace ( u, v ) of length w ( u, v ) with a path of w ( u, v ) 1 unit-weight edges, for every ( u, v ) ∈ E run BFS virtually 2 π ( v ) = vertex from which v is visited 3 d ( v ) = index of the level containing v 4 Problem: w ( u, v ) may be too large! 75/103
Shortest Path Algorithm by Running BFS Virtually S ← { s } , d ( s ) ← 0 1 while | S | ≤ n 2 find a v / ∈ S that minimizes u ∈ S :( u,v ) ∈ E { d ( u ) + w ( u, v ) } min 3 S ← S ∪ { v } 4 d ( v ) ← min u ∈ S :( u,v ) ∈ E { d ( u ) + w ( u, v ) } 5 76/103
Virtual BFS: Example 9 4 0 4 5 s a b 2 6 3 4 3 7 2 10 5 4 c d e Time 10 77/103
Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 78/103
Dijkstra’s Algorithm Dijkstra( G, w, s ) S ← ∅ , d ( s ) ← 0 and d ( v ) ← ∞ for every v ∈ V \ { s } 1 while S � = V do 2 u ← vertex in V \ S with the minimum d ( u ) 3 add u to S 4 for each v ∈ V \ S such that ( u, v ) ∈ E 5 if d ( u ) + w ( u, v ) < d ( v ) then 6 d ( v ) ← d ( u ) + w ( u, v ) 7 π ( v ) ← u 8 return ( d, π ) 9 Running time = O ( n 2 ) 79/103
2 7 5 a b 6 8 2 9 4 0 13 16 10 s c d 12 1 3 4 5 7 14 4 7 e t 3 f u 80/103
Improved Running Time using Priority Queue Dijkstra ( G, w, s ) 1 S ← ∅ , d ( s ) ← 0 and d ( v ) ← ∞ for every v ∈ V \ { s } 2 Q ← empty queue, for each v ∈ V : Q. insert ( v, d ( v )) 3 while S � = V , do 4 u ← Q. extract min () 5 S ← S ∪ { u } 6 for each v ∈ V \ S such that ( u, v ) ∈ E 7 if d ( u ) + w ( u, v ) < d ( v ) then 8 d ( v ) ← d ( u ) + w ( u, v ) , Q. decrease key ( v, d ( v )) 9 π ( v ) ← u 10 11 return ( π, d ) 81/103
Recall: Prim’s Algorithm for MST MST-Prim( G, w ) s ← arbitrary vertex in G 1 S ← ∅ , d ( s ) ← 0 and d ( v ) ← ∞ for every v ∈ V \ { s } 2 Q ← empty queue, for each v ∈ V : Q. insert ( v, d ( v )) 3 while S � = V , do 4 u ← Q. extract min() 5 S ← S ∪ { u } 6 for each v ∈ V \ S such that ( u, v ) ∈ E 7 if w ( u, v ) < d ( v ) then 8 d ( v ) ← w ( u, v ) , Q. decrease key ( v, d ( v )) 9 π ( v ) ← u 10 11 return � � ( u, π ( u )) | u ∈ V \ { s } 82/103
Improved Running Time Running time: O ( n ) × ( time for extract min ) + O ( m ) × ( time for decrease key ) Priority-Queue extract min decrease key Time Heap O (log n ) O (log n ) O ( m log n ) Fibonacci Heap O (log n ) O (1) O ( n log n + m ) 83/103
Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 84/103
Encoding Symbols Using Bits assume: 8 symbols a, b, c, d, e, f, g, h in a language need to encode a message using bits idea: use 3 bits per symbol a b c d e f g h 000 001 010 011 100 101 110 111 deacfg → 011100000010101110 Q: Can we have a better encoding scheme? Seems unlikely: must use 3 bits per symbol Q: What if some symbols appear more frequently than the others in expectation? 85/103
Q: If some symbols appear more frequently than the others in expectation, can we have a better encoding scheme? A: Maybe. Using variable-length encoding scheme. Idea using fewer bits for symbols that are more frequently used, and more bits for symbols that are less frequently used. Need to use prefix codes to guarantee a unique decoding. 86/103
Prefix Codes Def. A prefix code for a set S of symbols is a function γ : S → { 0 , 1 } ∗ such that for two distinct x, y ∈ S , γ ( x ) is not a prefix of γ ( y ) . 0 1 a b c d 001 0000 0001 100 0 1 0 1 e h e f g h 1 0 1 0 11 1010 1011 01 a d 1 0 1 0 c g f b 0001/001/100/0000/01/01/11/1010/0001/001/ cadbhhefca 87/103
Properties of Encoding Tree Rooted binary tree 0 1 Left edges labelled 0 and right edges labelled 1 1 0 0 1 A leaf corresponds to a code e h for some symbol 0 1 0 1 If coding scheme is not a d 0 1 0 1 wasteful: a non-leaf has c g exactly two children f b Best Prefix Codes Input: frequencies of letters in a message Output: prefix coding scheme giving the shortest encoding for the message 88/103
example symbols a b c d e frequencies 18 3 4 6 10 scheme 1 length 2 3 3 2 2 total = 89 scheme 2 length 1 3 3 3 3 total = 87 scheme 3 length 1 4 4 3 2 total = 84 a a e a d e d b c b c d e b c scheme 1 scheme 2 scheme 3 89/103
Example Input: ( a : 18, b : 3, c : 4, d : 6, e : 10) Q: What types of decisions should we make? the code for some letter? hard to design a strategy; residual problem is complicated. a partition of letters into left and right sub-trees? not clear how to design the greedy algorithm A: Choose two letters and make them brothers in the tree. 90/103
Which Two symbols Can Be Safely Put Together As Brothers? Focus a tree structure, without leaf labeling There are two deepest leaves that are brothers It is safe to make the two least frequent symbols brothers! best to put the two least frenquent symbols here! 91/103
It is safe to make the two least frequent symbols brothers! Lemma There is an optimum encoding tree, where the two least frequent symbols are brothers. So we can make the two least frequent symbols brothers; the decision is irrevocable. Q: Is the residual problem an instance of the best prefix codes problem? A: Yes, although the answer is not immediate. 92/103
f x : the frequency of the symbol x in the support. x 1 and x 2 : the two symbols we decided to put together. d x the depth of symbol x in our output encoding tree. � f x d x x ∈ S � = f x d x + f x 1 d x 1 + f x 2 d x 2 encoding tree for x ∈ S \{ x 1 ,x 2 } S \ { x 1 , x 2 } ∪ { x ′ } � = f x d x + ( f x 1 + f x 2 ) d x 1 x ∈ S \{ x 1 ,x 2 } x ′ � = f x d x + f x ′ ( d x ′ + 1) x 1 x 2 x ∈ S \{ x 1 ,x 2 } � = f x d x + f x ′ x ∈ S \{ x 1 ,x 2 }∪{ x ′ } Def: f x ′ = f x 1 + f x 2 93/103
In order to minimize � f x d x , x ∈ S we need to minimize � f x d x , x ∈ S \{ x 1 ,x 2 }∪{ x ′ } subject to that d is the depth function for an encoding tree of S \ { x 1 , x 2 } . This is exactly the best prefix codes problem, with symbols S \ { x 1 , x 2 } ∪ { x ′ } and frequency vector f ! 94/103
Huffman codes: Recursive Algorithm Huffman ( S, f ) if | S | > 1 then 1 let x 1 , x 2 be the two symbols with the smallest f values 2 introduce a new symbol x ′ and let f x ′ = f x 1 + f x 2 3 S ′ ← S \ { x 1 , x 2 } ∪ { x ′ } 4 call Huffman ( S ′ , f | S ′ ) to build an encoding tree T ′ 5 let T be obtained from T ′ by adding x 1 , x 2 as two children 6 of x ′ return T 7 else 8 let x be the symbol in S 9 return a tree with a single node labeled x 10 95/103
Huffman codes: Iterative Algorithm Huffman ( S, f ) while | S | > 1 do 1 let x 1 , x 2 be the two symbols with the smallest f values 2 introduce a new symbol x ′ and let f x ′ = f x 1 + f x 2 3 let x 1 and x 2 be the two children of x ′ 4 S ← S \ { x 1 , x 2 } ∪ { x ′ } 5 return the tree constructed 6 96/103
Example 75 A : 00 B : 10 1 0 C : 010 D : 011 47 28 E : 110 1 F : 111 0 1 0 20 13 0 1 0 1 27 15 11 9 8 5 A B C D E F 97/103
Algorithm using Priority Queue Huffman ( S, f ) Q ← build-priority-queue( S ) 1 while Q. size > 1 do 2 x 1 ← Q. extract-min() 3 x 2 ← Q. extract-min() 4 introduce a new symbol x ′ and let f x ′ = f x 1 + f x 2 5 let x 1 and x 2 be the two children of x ′ 6 Q. insert( x ′ ) 7 return the tree constructed 8 98/103
Outline Toy Examples 1 Interval Scheduling 2 Minimum Spanning Tree 3 Kruskal’s Algorithm Reverse-Kruskal’s Algorithm Prim’s Algorithm Single Source Shortest Paths 4 Dijkstra’s Algorithm Data Compression and Huffman Code 5 Summary 6 99/103
Summary for Greedy Algorithms Design a “reasonable” strategy 1 Interval scheduling problem: schedule the job j ∗ with the earliest deadline Kruskal’s algorithm for MST: select lightest edge e ∗ Inverse Kruskal’s algorithm for MST: drop the heaviest non-bridge edge e ∗ Prim’s algorithm for MST: select the lightest edge e ∗ incident to a specified vertex s Huffman codes: make the two least frequent symbols brothers 100/103
Recommend
More recommend
