Algorithm : Design & Analysis Recursion [3]
In the last class… � Asymptotic growth rate � The Sets Ο , Ω and Θ � Complexity Class � An Example: Maximum Subsequence Sum � Improvement of Algorithm � Comparison of Asymptotic Behavior � Another Example: Binary Search � Binary Search Is Optimal
Recursion � Recursive Procedures � Proving Correctness of Recursive Procedures � Deriving recurrence equations � Solution of the Recurrence equations � Guess and proving � Recursion tree � Master theorem � Divide-and-conquer
Recursion as a Thinking Way � Cutting the plane � How many sections can be generated at most by n straight lines with infinite length. L(0) = 1 L(0) = 1 Intersecting all n-1 L(n) = L(n-1) +n L(n) = L(n-1) +n existing lines to get as most sections as possible Line n
Recursion for Algorithm Recurrence relations � Computing n! � if n=1 then return 1 else return Fac(n-1)*n M(1)=0 and M(n)=M(n-1)+1 for n>0 (critical operation: multiplication) � Hanoi Tower � if n=1 then move d(1) to peg3 else { Hanoi(n-1, peg1, peg2); move d(n) to peg3; Hanoi(n-1, peg2, peg3) M(1)=1 and M(n)=2M(n-1)+1 for n>1 (critical operation: move)
Counting the Number of Bit � Input: a positive decimal integer n � Output: the number of binary digits in n ’s binary representation Int BinCounting (int n ) Int BinCounting (int n ) 1. if ( n ==1) return 1; 1. if ( n ==1) return 1; 2. else 2. else 3. return BinCounting(n div 2)+1; 3. return BinCounting(n div 2)+1;
Correctness of BinCounting � Proof by induction � Base case: if n =1, trivial by line 1. � Inductive hypothesis: for any 0< k < n , BinCounting( k ) return the correct result. � Induction � If n ≠ 1 then line 3 is excuted � ( n div 2) is a positive decimal integer (so, the precondition for BinCounting is still hold), and � 0<( n div 2)< n , so, the inductive hypothesis applies � So, the correctness (the number of bit of n is one more the that of ( n div 2)
Complexity Analysis of BinCounting � The critical operation: addition � The recurrence relation = ⎧ 0 1 n = ⎨ T ( n ) ( ) ⎣ ⎦ + > ⎩ T n / 2 1 n 1
Solution by backward substitutions ⎛ ⎞ ⎢ ⎥ n = ⎜ ⎟ + ： By the recursion equation T ( n ) T 1 ⎜ ⎟ ⎢ ⎥ ⎣ ⎦ ⎝ ⎠ 2 = k For simplicity , let n 2 ( k is a nonnegativ e integer ) , = that is, k log n ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ n n n = + = + + = + + + = ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ T ( n ) T 1 T 1 1 T 1 1 1 ...... ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ 2 4 8 ⎛ ⎞ n = + = ⎜ ⎟ T ( n ) T log n log n ( T (1)=0 ) ⎝ ⎠ k 2
Smooth Functions � Let f ( n ) be a nonnegative eventually non- decreasing function defined on the set of natural numbers, f ( n ) is called smooth if f (2 n ) ∈ Θ ( f ( n )). � Note: log n , n , n log n and n α ( α≥ 0) are all smooth. � For example: 2 n log2 n = 2 n (log n +log2) ∈ Θ ( n log n )
Even Smoother � Let f ( n ) be a smooth function, then, for any fixed integer b ≥ 2, f ( bn ) ∈Θ ( f ( n )). � That is, there exist positive constants c b and d b and a nonnegative integer n 0 such that d b f ( n ) ≤ f ( bn ) ≤ c b f ( n ) for n ≥ n 0 . = k It is easy to prove that the result holds for b 2 , for the second inequality : ≤ = ≥ k k f ( 2 n ) c f ( n ) for k 1 , 2 , 3 ... and n n . 2 0 ≥ ≤ ≤ k - 1 k For an arbitrary integer b 2 , 2 b 2 ≤ ≤ k k k Then, f ( bn ) f ( 2 n ) c f ( n ), we can use c as c . 2 2 b
Smoothness Rule � Let T ( n ) be an eventually nondecreasing function and f ( n ) be a smooth function. If T ( n ) ∈ Θ ( f ( n )) for values of n that are powers of b ( b ≥ 2), then T ( n ) ∈ Θ ( f ( n )). Just proving the big - Oh part : ≤ ≥ k k k By the hypothsis : ( ) ( ) for . T b cf b b n 0 ≤ ≥ By the prior result : f ( bn ) c f ( n ) for n n . b 0 ≤ ≤ ≤ + k k 1 Let n b n b , 0 + + ≤ ≤ = ≤ ≤ k 1 k 1 k k T ( n ) T ( b ) cf ( b ) cf ( bb ) cc f ( b ) cc f ( n ) b b Non-decreasing Non-decreasing Prior result hypothesis
Computing the n th Fibonacci Number f 1 =0 f 1 =0 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...... f 2 =1 f 2 =1 f n = f n -1 + f n -2 f n = f n -1 + f n -2 = + + + L a r a r a r a − − − n 1 n 1 2 n 2 m n k is called linear homogeneous relation of degree k . For the special case of Fibonacci: a n = a n-1 + a n-2 , r 1 = r 2 =1
Characteristic Equation � For a linear homogeneous recurrence relation of degree k = + + + L a r a r a r a − − − n 1 n 1 2 n 2 k n k the polynomial of degree k − − = + + + k k 1 k 2 L x r x r x r 1 2 k is called its characteristic equation. � The characteristic equation of linear homogeneous recurrence relation of degree 2 is: − − = 2 x r x r 0 1 2
Solution of Recurrence Relation − − = 2 � If the characteristic equation of the x r x r 0 1 2 = − + recurrence relation has two a r a r a − n 1 n 1 2 n 2 distinct roots s 1 and s 2 , then = 1 + n n a us vs n 2 where u and v depend on the initial conditions, is the explicit formula for the sequence. � If the equation has a single root s , then, both s 1 and s 2 in the formula above are replaced by s
Proof of the Solution − − = 2 Remember equation : x r x r 0 the 1 2 + = + n n We need prove that : us vs r a r a − − 1 2 1 n 1 2 n 2 − − + = + n n n 2 2 n 2 2 us vs us s vs s 1 2 1 1 2 2 − − = + + + n 2 n 2 us ( r s r ) vs ( r s r ) 1 1 1 2 2 1 2 2 − − − − = + + + n 1 n 2 n 1 n 2 r us r us r vs r vs 1 1 2 1 1 2 2 2 − − − − = + + + n 1 n 1 n 2 n 2 ( ) ( ) r us vs r us vs 1 1 2 2 1 2 = + r a r a − − 1 n 1 2 n 2
Return to Fibonacci Sequence f 1 =1 f 1 =1 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...... f 2 =1 f 2 =1 f n = f n -1 + f n -2 f n = f n -1 + f n -2 Explicit formula for Fibonacci Sequence The characteristic equation is x 2 - x -1=0, which has roots: + − 1 5 1 5 = = s and s 1 2 2 2 = + = = + = 2 2 f us vs 1 and f us vs 1 Note: (by initial conditions) 1 1 2 2 1 2 n n ⎛ + ⎞ ⎛ − ⎞ which results: 1 1 5 1 1 5 ⎜ ⎟ ⎜ ⎟ = − f ⎜ ⎟ ⎜ ⎟ n 2 2 ⎝ ⎠ ⎝ ⎠ 5 5
Guess the Solutions � Example: T(n)=2T( ⎣ n/2 ⎦ ) +n Try to prove T ( n ) ≤ cn : � Guess T ( n ) = 2 T ( ⎣ n /2 ⎦ )+ n T ( n )=2 T ( ⎣ n /2 ⎦ )+ n ≤ 2 c ( ⎣ n /2 ⎦ )+ n ≤ 2( c ⎣ n /2 ⎦ log ( ⎣ n /2 ⎦ ))+ n However: � T ( n ) ∈ O ( n )? ≤ 2 c ( n /2)+ n = ( c +1)n, Fail! ≤ cn log ( n /2)+ n T ( n ) = 2 T ( ⎣ n /2 ⎦ )+ n ≥ 2 c ⎣ n /2 ⎦ + n � T ( n ) ≤ cn , to be proved for c large enough = cn log n – cn log 2 + n ≥ 2 c [( n -1)/2]+ n = cn +( n - c ) ≥ cn � T ( n ) ∈ O ( n 2 )? = cn log n – cn + n ≤ cn log n for c ≥ 1 � T ( n ) ≤ cn 2 , to be proved for c large enough � Or maybe , T ( n ) ∈ O ( n log n )? � T ( n ) ≤ cn log n , to be proved for c large enough
Recursion Tree T ( size ) nonrecursive cost T ( n ) n T ( n / 2 ) n / 2 T ( n / 2 ) n / 2 T ( n / 4 ) n / 4 T ( n /4) n/4 T ( n / 4 ) T ( n / 4 ) n/4 n/4 The recursion tree for T(n)=T(n/2)+T(n/2)+n
Recursion Tree Rules � Construction of a recursion tree � work copy: use auxiliary variable � root node � expansion of a node: � recursive parts: children � nonrecursive parts: nonrecursive cost � the node with base-case size
Recursion tree equation � For any subtree of the recursion tree, � size field of root = Σ nonrecursive costs of expanded nodes + Σ size fields of incomplete nodes � Example: divide-and-conquer: T ( n ) = bT ( n / c ) + f ( n ) � After k th expansion: − ⎛ ⎞ ⎛ ⎞ k 1 n n ∑ − = + ⎜ ⎟ ⎜ ⎟ k i 1 T ( n ) b T b f − ⎝ ⎠ ⎝ ⎠ k i 1 c c = i 1
Evaluation of a Recursion Tree � Computing the sum of the nonrecursive costs of all nodes. � Level by level through the tree down. � Knowledge of the maximum depth of the recursion tree, that is the depth at which the size parameter reduce to a base case.
Recursion Tree Work copy: T(k)=T(k/2)+T(k/2)+k Work copy: T(k)=T(k/2)+T(k/2)+k T ( n )= n lg n T ( n ) n T ( n / 2 ) n / 2 T ( n / 2 ) n / 2 n /2 d (size → 1) T ( n / 4 ) n / 4 T ( n /4) n/4 T ( n / 4 ) T ( n / 4 ) n/4 n/4 At this level: T ( n )= n +2( n /2)+4 T ( n /4)=2 n +4 T ( n /4) At this level: T ( n )= n +2( n /2)+4 T ( n /4)=2 n +4 T ( n /4)
Recursion Tree for T ( n )=3 T ( ⎣ n /4 ⎦ )+ Θ ( n 2 ) cn 2 cn 2 3 cn 2 c ( ¼ n ) 2 c ( ¼ n ) 2 c ( ¼ n ) 2 16 log 4 n 2 ⎛ ⎞ 3 c ( (1/16) n ) 2 c ( (1/16) n ) 2 …… …… c ( (1/16) n ) 2 c ( (1/16) n ) 2 ⎜ ⎟ 2 cn c ((1/16) n ) 2 ⎝ ⎠ 16 …… …… … ( ) Θ log 4 3 n T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) n = log log 3 Total: Θ (n 2 ) 3 n 4 4 Note:
Recommend
More recommend
