Algorithms Theory Algorithms Theory 14 Dynamic Programming (3) - PowerPoint PPT Presentation

Algorithms Theory Algorithms Theory 14 – Dynamic Programming (3) Programming (3) Optimal binary search trees P.D. Dr. Alexander Souza Winter term 11/12

Method of dynamic programming Recursive approach: Solve a problem by solving several smaller analogous subproblems of the same type. Then combine these solutions to generate a solution to the original problem. l ti t t l ti t th i i l bl Drawback: Repeated computation of solutions Drawback: Repeated computation of solutions Dynamic-programming method: Once a subproblem has been solved, store its solution in a table so that it can be retrieved later by simple table lookup. 2 Winter term 11/12

Optimal substructure Dynamic programming is typically applied to optimization problems . An optimal solution to the original problem contains optimal solutions to smaller subproblems. ti l l ti t ll b bl 3 Winter term 11/12

Construction of optimal binary search trees (- ∞ ,k 1 ) k 1 (k 1 ,k 2 ) k 2 (k 2 ,k 3 ) k 3 (k 3 ,k 4 ) k 4 (k 4 , ∞ ) 4 1 0 3 0 3 0 3 10 4 1 0 3 0 3 0 3 10 3 k 2 1 3 k 1 k 4 4 0 3 10 (- ∞ , k 1 ) ( 1 ) ( k 1 , k 2 ) ( 1 2 ) k 3 ( k 4 , ∞ ) ( 4 ) 3 0 0 ( k 2 , k 3 ) ( k k ) 0 0 ( k ( k 3 , k 4 ) k ) weighted path length: 3 ⋅ 1 +2 ⋅ (1 + 3) + 3 ⋅ 3 + 2 ⋅ (4 + 10) 3 ⋅ 1 +2 ⋅ (1 + 3) + 3 ⋅ 3 + 2 ⋅ (4 + 10) 4 Winter term 11/12

Construction of optimal binary search trees Given : set S of keys Given : set S of keys S = { k 1 ,....k n } - ∞ = k 0 < k 1 < ... < k n < k n+1 = ∞ a i : (absolute) frequency of requests to key k i b j : (absolute) frequency of requests to x ∈ ( k j , k j+1 ) b j : (absolute) frequency of requests to x ∈ ( k j , k j+1 ) Weighted path length P(T) of a binary search tree T for S : n n ∑ ∑ P ( T ) = ( depth ( k ) + 1 ) a + depth (( k , k )) b i i j j + 1 j i = 1 j = 0 Goal : Binary search tree with minimum weighted path length P for S . 5 Winter term 11/12

Construction of optimal binary search trees 4 4 2 2 k 2 k k 1 k 2 5 5 1 4 k 1 k 1 k 2 k 2 1 1 3 3 3 3 5 5 P(T 2 ) = 27 ( 2 ) P(T 1 ) = 21 P(T 1 ) 21 6 Winter term 11/12

Construction of optimal binary search trees a i k i b j k j , k j + 1 An optimal binary search tree is a binary search tree with minimum weighted path length. 7 Winter term 11/12

Construction of optimal binary search trees T a k P ( T ) = P( T l ) + W( T l ) + P( T r ) + W( T r ) + a k k = P( T l ) + P(T r ) + W(T) where W(T) := total weight of all nodes in T T l T r If T is a tree with minimum weighted path length for S , then subtrees T l and T r are trees with minimum weighted path length for subsets of S . 8 Winter term 11/12

Construction of optimal binary search trees Let T(i, j): optimal binary search tree for (k i , k i+1 ) k i+1 ... k j (k j , k j+1 ), W(i, j): weight of T(i, j), i.e. W(i, j) = b i + a i+1 + ... + a j + b j , P(i j) P(i, j): weighted path length of T(i, j). i ht d th l th f T(i j) 9 Winter term 11/12

Construction of optimal binary search trees T(i, j) = k l T(i, l - 1) T(l, j) ( k i , k i + 1 ) k i + 1 ..... k l – 1 (k l –1 , k l ) ( k l , k l + 1 ) k l + 1 ..... k j (k j , k j + 1 ) request frequency: b i a i + 1 a l – 1 b l – 1 a l b l a l + 1 a j b j 10 Winter term 11/12

Construction of optimal binary search trees W(i, i) = b i , for 0 ≤ i ≤ n W(i, j) = W(i, j – 1) + a j + b j , for 0 ≤ i < j ≤ n P(i, i) = 0 , for 0 ≤ i ≤ n P(i, j) = W(i, j) + min { P(i, l –1 ) + P(l, j) }, for 0 ≤ i < j ≤ n P(i j) W(i j) + i { P(i l 1 ) + P(l j) } f 0 ≤ i < j ≤ (*) (*) i < l ≤ j r(i, j) = the index l for which the minimum is achieved in (*) 11 Winter term 11/12

Construction of optimal binary search trees Base cases Case 1: s = j – i = 0 T(i, i) = (k i , k i +1 ) W(i, i) = b i P(i, i) = 0, r(i, i) not defined 12 Winter term 11/12

Construction of optimal binary search trees Case 2: s j Case 2: s = j – i = 1 i 1 T(i, i+1) k i 1 k i+1 a i+1 k k k i , k i+1 k i+1 , k i+2 k k b i b i+1 W(i ,i +1) = b i + a i+1 + b i+1 = W(i, i) + a i+1 + W(i+1, i+1) P(i, i+1) = W(i, i + 1) r(i, i +1) = i + 1 13 Winter term 11/12

Computing the minimum weighted path length using dynamic programming using dynamic programming Case 3: s = j - i > 1 Case 3: s j i 1 for s = 2 to n do for i = 0 to (n – s) do { j = i + s; determine (greatest) l, i < l ≤ j, s.t. P(i, l – 1) + P(l, j) is minimal d t i ( t t) l i < l ≤ j t P(i l 1) + P(l j) i i i l P(i, j) = P(i, l –1) + P(l, j) + W(i, j); r(i, j) = l; r(i, j) l; } 14 Winter term 11/12

Construction of optimal binary search trees Define: ⎫ ⎫ P P ( ( i , j ) ) : min. i weighted i h d path h l length h f for = ⎬ K b a b a b i i + 1 i + 1 j j ⎭ W ( i , j ) : = sum of Then: ⎧ ⎧ b b if if i i j j = i ⎨ W ( i , j ) = ⎩ W ( i , j − 1 ) + a + W ( j , j ) otherwise j ⎧ 0 if i = j ⎪ ⎨ ⎨ P ( ( i , , j j ) ) = { { } } W W ( ( i i , , j j ) ) + + min min P P ( ( i i , , l l − 1 1 ) ) + + P P ( ( l l , , j j ) ) otherwise otherwise ⎪ ⎩ i < l ≤ j � Computing the solution P (0, n ) takes O ( n 3 ) time. and requires O ( n 2 ) space. 15 Winter term 11/12

Construction of optimal binary search trees Theorem Theorem An optimal binary search tree for n keys and n + 1 intervals with known request frequencies can be constructed in O ( n 3 ) time. 16 Winter term 11/12