2. Recurrences http://aofa.cs.princeton.edu A N A L Y T I C C O M - PowerPoint PPT Presentation

A N A L Y T I C C O M B I N A T O R I C S P A R T O N E 2. Recurrences http://aofa.cs.princeton.edu

A N A L Y T I C C O M B I N A T O R I C S P A R T O N E 2. Recurrences •Computing values •Telescoping •Types of recurrences OF •Mergesort •Master Theorem http://aofa.cs.princeton.edu 2a.Recur.Values

What is a recurrence? Def. A recurrence is an equation that recursively defines a sequence. Familiar example 1: Fibonacci numbers recurrence � � = � � − � + � � − � ��� � ≥ � ���� � � = � ��� � � = � MUST specify for all N with sequence initial conditions 0, 1, 1, 2, 3, 5, 8, 13, 21, ... Q. Simple formula for sequence (function of N )? 3

What is a recurrence? Recurrences directly model costs in programs. Familiar example 2: Quicksort (see lecture 1) program recurrence � public class Quick � � � = � + � + � ( � � + � � − � − � ) { private static int partition(Comparable[] a, int lo, int hi) � ≤ � ≤ � − � { int i = lo, j = hi+1; while (true) ��� � ≥ � ���� � � = � { while (less(a[++i], a[lo])) if (i == hi) break; while (less(a[lo], a[--j])) if (j == lo) break; if (i >= j) break; sequence exch(a, i, j); } exch(a, lo, j); 0, 2, 5, 8 2/3, 12 5/6, 17 2/5,... return j; } private static void sort(Comparable[] a, int lo, int hi) { if (hi <= lo) return; int j = partition(a, lo, hi); sort(a, lo, j-1); sort(a, j+1, hi); } } 4

Common-sense rule for solving any recurrence � � = � � − � + � � − � ��� � ≥ � ���� � � = � ��� � � = � Use your computer to compute values. public static void F(int N) { ✘ Use a recursive program? if (N == 0) return 0; if (N == 1) return 1; return F(N-1) + F(N-2); F(50) } F(49) F(48) NO, NO, NO: Takes exponential time! F(48) F(47) F(47) F(46) F(47) F(46) F(46) F(45) F(46) F(45) F(45) F(44) Top of recursion tree for naïve Fibonacci function long[] F = new long[51]; F[0] = 0; F[1] = 1; ✓ Instead, save all values in an array. if (N == 1) return 1; for (int i = 2; i <= 50; i++) F[i] = F[i-1] + F[i-2]; 5

Common-sense starting point for solving any recurrence Use your computer to compute initial values. First step: Download "standard model" from Algorithms, 4th edition booksite. StdIn Standard Input StdOut Standard Output StdDraw Standard Drawings StdRandom Random Numbers ... (Several other libraries) http://algs4.cs.princeton.edu 6

Common-sense starting point for solving any recurrence Use your computer to compute initial values (modern approach). Ex. 1: Fibonacci � � = � � − � + � � − � ���� � � = � ��� � � = � Sequence.java public interface Sequence Fib.java public class Fib implements Sequence { { public double eval(int N); private final double[] F; } public Fib(int maxN) { % java Fib 15 Compute all values F = new double[maxN+1]; 0.0 in the constructor F[0] = 0; F[1]= 1; 1.0 for (int N = 2; N <= maxN; N++) 1.0 F[N] = F[N-1] + F[N-2]; 2.0 } 3.0 5.0 public double eval(int N) 8.0 { return F[N]; } 13.0 21.0 public static void main(String[] args) 34.0 { 55.0 int maxN = Integer.parseInt(args[0]); 89.0 Fib F = new Fib(maxN); for (int i = 0; i < maxN; i++) 144.0 StdOut.println(F.eval(i)); 233.0 } 377.0 } 7

Common-sense starting point for solving any recurrence �� � = ( � + � ) � � − � + � � Ex. 2: Quicksort QuickSeq.java public class QuickSeq implements Sequence { private final double[] c; % java QuickSeq 15 0.000000 public QuickSeq(int maxN) 2.000000 { 5.000000 c = new double[maxN+1]; 8.666667 c[0] = 0; 12.833333 for (int N = 1; N <= maxN; N++) 17.400000 c[N] = (N+1)*c[N-1]/N + 2; 22.300000 } 27.485714 32.921429 public double eval(int N) 38.579365 { return c[N]; } 44.437302 50.477056 public static void main(String[] args) 56.683478 { 63.043745 // Similar to Fib.java. 69.546870 } } 8

Common-sense starting point for solving any recurrence QuickSeq.java Use your computer to plot initial values. public class QuickSeq implements Sequence { // Implementation as above. Values.java public class Values public static void main(String[] args) { { int maxN = Integer.parseInt(args[0]); public static void show(Sequence f, int maxN) QuickSeq q = new QuickSeq(maxN); { Values.show(q, maxN); double max = 0; } for (int N = 0; N < maxN; N++) } if (f.eval(N)>max) max = f.eval(N); for (int N = 0; N < maxN; N++) { double x = 1.0*N/maxN; double y = 1.0*f.eval(N)/max; StdDraw.filledCircle(x, y, .002); } StdDraw.show(); } } % java QuickSeq 1000 9

A N A L Y T I C C O M B I N A T O R I C S P A R T O N E 2. Recurrences •Computing values •Telescoping •Types of recurrences OF •Mergesort •Master Theorem http://aofa.cs.princeton.edu 2a.Recur.Values

A N A L Y T I C C O M B I N A T O R I C S P A R T O N E 2. Recurrences •Computing values •Telescoping •Types of recurrences OF •Mergesort •Master Theorem http://aofa.cs.princeton.edu 2b.Recur.Telescope

Telescoping a (linear first-order) recurrence Linear first-order recurrences telescope to a sum. Example 1. � � = � � − � + � ���� � � = � = � � − � + ( � − � ) + � Apply equation for n − 1 = � � − � + ( � − � ) + ( � − � ) + � Do it again = � � + � � Continue, leaving a sum � ≤ � ≤ � = ( � + � ) � Evaluate sum Check. � ( � + � ) � = � ( � − � ) + � � � Challenge: Need to be able to evaluate the sum. 12

Elementary discrete sums � � = � − � � � geometric series � − � � ≤ � < � � = � ( � − � ) � � � � = arithmetic series � � � ≤ � < � � � � � + � � � � = binomial (upper) � � + � � ≤ � ≤ � see Knuth volume 1 for many more � � � � � � � − � = ( � + � ) � � binomial theorem � � ≤ � ≤ � � � � = � � Harmonic numbers � ≤ � ≤ � �� � � � � � + � � � � = Vandermonde convolution � � − � � � ≤ � ≤ � 13

Telescoping a (linear first-order) recurrence (continued) When coefficients are not 1, multiply/divide by a summation factor . Example 2. � � = � � � − � + � � ���� � � = � � � � � = � � − � � � − � + � Divide by 2 n � � � � = � Telescope to a sum � � = � ≤ � ≤ � � � = � � � Check. � � � = � ( � − � ) � � − � + � � Challenge: How do we find the summation factor? 14

Telescoping a (linear first-order) recurrence (continued) � � = � � � � − � + . . . Q. What’s the summation factor for ? � � � � − � � � − � . . . � � A. Divide by Example 3. � + � � � � � = � � − � + � ��� � > � ���� � � = � � summation factor: � + � � � − � � − � . . . � � � = � + � � � − � � � + � = � � − � � � � Divide by n +1 + � � + � � � � + � = � � � + � − � = � Telescope � ≤ � ≤ � � � = � ( � + � )( � � + � − � ) Challenge: Still need to be able to evaluate sums. 15

In-class exercise 1. Verify the solution for Example 3 . Check initial values � + � � � � � = � ( � + � )( � � + � − � ) � � = � � − � + � ��� � > � ���� � � = � � � � = � ( � � − � ) = � � � = � � � + � = � � � = � � � = � ( � � − � ) = � � � � + � = � � � = � ( � � − � ) � � = � � � � + � = �� / � = � ( � / � + � / � + � / � ) = �� / � Proof � � − � � + � � � ( � � − � ) + � = � ( � + � )( � � − � ) + � � � � = � ( � + � )( � � + � − � ) � � 16

In-class exercise 2. Solve this recurrence: �� � = ( � − � ) � � − � + � ��� � > � ���� � � = � Hard way: � − � � − � � − � � � − � · · · = summation factor: � � − � � ( � − � ) � � � = � �� � � = � Easy way: ��������� � � = � WHY? 17

A N A L Y T I C C O M B I N A T O R I C S P A R T O N E Recurrences •Computing values •Telescoping •Types of recurrences OF •Mergesort •Master Theorem http://aofa.cs.princeton.edu 2b.Recur.Telescope

A N A L Y T I C C O M B I N A T O R I C S P A R T O N E Recurrences •Computing values •Telescoping •Types of recurrences OF •Mergesort •Master Theorem http://aofa.cs.princeton.edu 2c.Recur.Types

Types of recurrences linear a n = na n − 1 − 1 first first order order a n = 1 / (1 + a n − 1 ) nonlinear a n = a n − 1 + 2 a n − 2 linear a n = a n − 1 a n − 2 + √ a n − 2 second order nonlinear variable a n = na n − 1 + ( n − 1) a n − 2 + 1 coefficients higher order higher order a n = f ( a n − 1 , a n − 2 , . . . , a n − t ) full history full history a n = n + a n − 1 + a n − 2 . . . + a 1 divide-and-conquer divide-and-conquer a n = a b n/ 2 c + a d n/ 2 e + n 20