CS 473: Algorithms, Fall 2016 Dynamic Programming on Trees Lecture 4 September 2, 2016 Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 46
What is Dynamic Programming? Every recursion can be memoized. Automatic memoization does not help us understand whether the resulting algorithm is e ffi cient or not. Dynamic Programming: A recursion that when memoized leads to an e ffi cient algorithm. Key Questions: Given a recursive algorithm, how do we analyze the complexity when it is memoized? How do we recognize whether a problem admits a dynamic programming based e ffi cient algorithm? How do we further optimize time and space of a dynamic programming based algorithm? Chandra & Ruta (UIUC) CS473 2 Fall 2016 2 / 46
Dynamic Programming Template Come up with a recursive algorithm to solve problem 1 Understand the structure/number of the subproblems generated 2 by recursion Memoize the recursion 3 set up compact notation for subproblems set up a data structure for storing subproblems Iterative algorithm 4 Understand dependency graph on subproblems Pick an evaluation order (any topological sort of the dependency dag) Analyze time and space 5 Optimize 6 Chandra & Ruta (UIUC) CS473 3 Fall 2016 3 / 46
Dynamic Programming on Trees Fact: Many graph optimization problems are NP-Hard Fact: The same graph optimization problems are in P on trees. Why? Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 46
Dynamic Programming on Trees Fact: Many graph optimization problems are NP-Hard Fact: The same graph optimization problems are in P on trees. Why? A significant reason: DP algorithm based on decomposability Powerful methodology for graph algorithms via a formal notion of decomposability called treewidth (beyond the scope of this class) Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 46
Maximum Independent Set in a Graph Definition Given undirected graph G = ( V , E ) a subset of nodes S ✓ V is an independent set (also called a stable set) if for there are no edges between nodes in S . That is, if u , v 2 S then ( u , v ) 62 E . B C A F E D Some independent sets in graph above: { D } , { A , C } , { B , E , F } Chandra & Ruta (UIUC) CS473 5 Fall 2016 5 / 46
Maximum Independent Set Problem Input Graph G = ( V , E ) Goal Find maximum sized independent set in G B C A F E D Chandra & Ruta (UIUC) CS473 6 Fall 2016 6 / 46
Maximum Weight Independent Set Problem Input Graph G = ( V , E ) , weights w ( v ) � 0 for v 2 V Goal Find maximum weight independent set in G 5 B 15 2 C A F 2 E D 10 20 Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 46
Maximum Weight Independent Set Problem No one knows an e ffi cient (polynomial time) algorithm for this 1 problem Problem is NP-Hard and it is believed that there is no 2 polynomial time algorithm Brute-force algorithm: Try all subsets of vertices. Chandra & Ruta (UIUC) CS473 8 Fall 2016 8 / 46
A Recursive Algorithm Let V = { v 1 , v 2 , . . . , v n } . For a vertex u let N ( u ) be its neighbors. Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 46
A Recursive Algorithm Let V = { v 1 , v 2 , . . . , v n } . For a vertex u let N ( u ) be its neighbors. Observation v 1 : vertex in the graph. One of the following two cases is true Case 1 v 1 is in some maximum independent set. Case 2 v 1 is in no maximum independent set. We can try both cases to “reduce” the size of the problem Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 46
A Recursive Algorithm Let V = { v 1 , v 2 , . . . , v n } . For a vertex u let N ( u ) be its neighbors. Observation v 1 : vertex in the graph. One of the following two cases is true Case 1 v 1 is in some maximum independent set. Case 2 v 1 is in no maximum independent set. We can try both cases to “reduce” the size of the problem G 1 = G � v 1 obtained by removing v 1 and incident edges from G G 2 = G � v 1 � N ( v 1 ) obtained by removing N ( v 1 ) [ v 1 from G MIS ( G ) = max { MIS ( G 1 ) , MIS ( G 2 ) + w ( v 1 ) } Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 46
A Recursive Algorithm RecursiveMIS ( G ): if G is empty then Output 0 a = RecursiveMIS ( G � v 1 ) b = w ( v 1 ) + RecursiveMIS ( G � v 1 � N ( v n ) ) Output max( a , b ) Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 46
Example Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 46
Recursive Algorithms ..for Maximum Independent Set Running time: ⇣ ⌘ T ( n ) = T ( n � 1) + T n � 1 � deg ( v 1 ) + O (1 + deg ( v 1 )) where deg ( v 1 ) is the degree of v 1 . T (0) = T (1) = 1 is base case. Worst case is when deg ( v 1 ) = 0 when the recurrence becomes T ( n ) = 2 T ( n � 1) + O (1) Solution to this is T ( n ) = O (2 n ) . Chandra & Ruta (UIUC) CS473 12 Fall 2016 12 / 46
Memoization We can memoize the recursive algorithm. Question: Does it lead to an e ffi cient algorithm? Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 46
Memoization We can memoize the recursive algorithm. Question: Does it lead to an e ffi cient algorithm? What is number of subproblems if started on graph with n nodes? Exercise: Show that even when G is a cycle the number of subproblems is exponential in n . Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 46
Part I Maximum Weighted Independent Set in Trees Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 46
Maximum Weight Independent Set in a Tree Input Tree T = ( V , E ) and weights w ( v ) � 0 for each v 2 V Goal Find maximum weight independent set in T r 10 5 8 a b 11 e g c 4 d f 4 9 3 8 2 h i j 7 Maximum weight independent set in above tree: ?? Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 46
A Recursive Algorithm For an arbitrary graph G : Number vertices as v 1 , v 2 , . . . , v n 1 Find recursively optimum solutions without v n (recurse on 2 G � v n ) and with v n (recurse on G � v n � N ( v n ) & include v n ). Saw that if graph G is arbitrary there was no good ordering that 3 resulted in a small number of subproblems. What about a tree? Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 46
A Recursive Algorithm For an arbitrary graph G : Number vertices as v 1 , v 2 , . . . , v n 1 Find recursively optimum solutions without v n (recurse on 2 G � v n ) and with v n (recurse on G � v n � N ( v n ) & include v n ). Saw that if graph G is arbitrary there was no good ordering that 3 resulted in a small number of subproblems. What about a tree? Natural candidate for v n is root r of T ? Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 46
Towards a Recursive Solution Natural candidate for v n is root r of T ? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r . Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Towards a Recursive Solution Natural candidate for v n is root r of T ? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r . Case r 2 O : None of the children of r can be in O . O � { r } contains an optimum solution for each subtree of T hanging at a grandchild of r . Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Towards a Recursive Solution Natural candidate for v n is root r of T ? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r . Case r 2 O : None of the children of r can be in O . O � { r } contains an optimum solution for each subtree of T hanging at a grandchild of r . Subproblems? Subtrees of T rooted at nodes in T . Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Towards a Recursive Solution Natural candidate for v n is root r of T ? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r . Case r 2 O : None of the children of r can be in O . O � { r } contains an optimum solution for each subtree of T hanging at a grandchild of r . Subproblems? Subtrees of T rooted at nodes in T . How many of them? Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Towards a Recursive Solution Natural candidate for v n is root r of T ? Let O be an optimum solution to the whole problem. Case r 62 O : Then O contains an optimum solution for each subtree of T hanging at a child of r . Case r 2 O : None of the children of r can be in O . O � { r } contains an optimum solution for each subtree of T hanging at a grandchild of r . Subproblems? Subtrees of T rooted at nodes in T . How many of them? O ( n ) Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 46
Example r 10 5 8 a b 11 c e g 4 f d 4 9 3 8 2 i j h 7 Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 46
A Recursive Solution T ( u ) : subtree of T hanging at node u OPT ( u ) : max weighted independent set value in T ( u ) OPT ( u ) = Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 46
A Recursive Solution T ( u ) : subtree of T hanging at node u OPT ( u ) : max weighted independent set value in T ( u ) (P v child of u OPT ( v ) , OPT ( u ) = max w ( u ) + P v grandchild of u OPT ( v ) Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 46
Recommend
More recommend
