Algorithm Analysis Part II Tyler Moore CS 2123, The University of - PowerPoint PPT Presentation

Why it matters Algorithm Analysis Part II Tyler Moore CS 2123, The University of Tulsa Some slides created by or adapted from Dr. Kevin Wayne. For more information see http://www.cs.princeton.edu/~wayne/kleinberg-tardos . Some slides adapted from Dr. Steven Skiena. For more information see http://www.algorist.com 9 3 / 32 Implications of dominance Testing dominance Definition f ( n ) Dominance g ( n ) dominates f ( n ) iff lim n →∞ g ( n ) = 0 Definition Exponential algorithms get hopeless fast. Little oh notation f ( n ) is o ( g ( n )) iff g ( n ) dominates f ( n ). Quadratic algorithms get hopeless at or before 1,000,000. O ( n log n ) is possible to about one billion. In other words, little oh means “grows strictly slower than”. Q: is 3 n o ( n 2 )? A: Yes, since lim n →∞ 3 n n 2 = 3 n = 0 Q: is 3 n 2 o ( n 2 )? A: 4 / 32 5 / 32

Useful facts Asymptotic bounds for some common functions f ( n ) Proposition. If , then f ( n ) is Θ ( g ( n )) . Polynomials. Let T ( n ) = a 0 + a 1 n + … + a d n d with a d > 0 . Then, T ( n ) is Θ ( n d ) . lim g ( n ) = c > 0 n →∞ a 0 + a 1 n + . . . + a d n d Pf. By definition of the limit, there exists n 0 such such that for all n ≥ n 0 Pf. lim = a d > 0 n d n →∞ 1 2 c < f ( n ) g ( n ) < 2 c no need to specify base Logarithms. Θ (log a n ) is Θ (log b n ) for any constants a , b > 0 . (assuming it is a constant) ・ Thus, f ( n ) ≤ 2 c g ( n ) for all n ≥ n 0 , which implies f ( n ) is O ( g ( n ) ) . ・ Similarly, f ( n ) ≥ ½ c g ( n ) for all n ≥ n 0 , which implies f ( n ) is Ω ( g ( n ) ) . Logarithms and polynomials. For every d > 0 , log n is O ( n d ) . f ( n ) Proposition. If , then f ( n ) is O ( g ( n )) . lim g ( n ) = 0 n →∞ Exponentials and polynomials. For every r > 1 and every d > 0 , n d is O ( r n ) . n d lim r n = 0 Pf. n →∞ 15 16 6 / 32 7 / 32 Exercises Linear time: O(n) Linear time. Running time is proportional to input size. Computing the maximum. Compute maximum of n numbers a 1 , …, a n . Using the limit formula and results from earlier slides, answer the following: Q: Is 5 n 2 + 3 n o ( n )? max ← a 1 A: No, since lim n →∞ 5 n 2 +3 n = lim n →∞ 5 n + 3 = ∞ for i = 2 to n { n Q: is 3 n 3 + 5 Θ( n 3 )? if (a i > max) max ← a i A: } Q: is n log n + n 2 O ( n 3 )? A: 19 8 / 32 10 / 32

Linear time: O(n) Linearithmic time: O(n log n) Merge. Combine two sorted lists A = a 1 , a 2 , …, a n with B = b 1 , b 2 , …, b n into sorted O(n log n) time. Arises in divide-and-conquer algorithms. whole. Sorting. Mergesort and heapsort are sorting algorithms that perform O ( n log n ) compares. Largest empty interval. Given n time-stamps x 1 , …, x n on which copies of a i = 1, j = 1 file arrive at a server, what is largest interval when no copies of file arrive? while (both lists are nonempty) { if (a i ≤ b j ) append a i to output list and increment i O(n log n) solution. Sort the time-stamps. Scan the sorted list in order, else(a i ≤ b j )append b j to output list and increment j identifying the maximum gap between successive time-stamps. } append remainder of nonempty list to output list Claim. Merging two lists of size n takes O ( n ) time. Pf. After each compare, the length of output list increases by 1 . 20 21 11 / 32 12 / 32 Quadratic time: O(n 2 ) Cubic time: O(n 3 ) Ex. Enumerate all pairs of elements. Cubic time. Enumerate all triples of elements. Closest pair of points. Given a list of n points in the plane ( x 1 , y 1 ), …, ( x n , y n ) , Set disjointness. Given n sets S 1 , …, S n each of which is a subset of find the pair that is closest. 1, 2, …, n , is there some pair of these which are disjoint? O(n 2 ) solution. Try all pairs of points. O(n 3 ) solution. For each pair of sets, determine if they are disjoint. min ← (x 1 - x 2 ) 2 + (y 1 - y 2 ) 2 foreach set S i { for i = 1 to n { foreach other set S j { for j = i+1 to n { foreach element p of S i { d ← (x i - x j ) 2 + (y i - y j ) 2 determine whether p also belongs to S j if (d < min) } min ← d if (no element of S i belongs to S j ) } report that S i and S j are disjoint } } Remark. Ω ( n 2 ) seems inevitable, but this is just an illusion. [see Chapter 5] } 22 23 13 / 32 14 / 32

Polynomial time: O(n k ) Exponential time Independent set of size k. Given a graph, are there k nodes such that no Independent set. Given a graph, what is maximum cardinality of an two are joined by an edge? independent set? k is a constant O(n k ) solution. Enumerate all subsets of k nodes. O(n 2 2 n ) solution. Enumerate all subsets. foreach subset S of k nodes { S* ← φ check whether S in an independent set foreach subset S of nodes { if (S is an independent set) check whether S in an independent set report S is an independent set if (S is largest independent set seen so far) } update S* ← S } } } ・ Check whether S is an independent set takes O ( k 2 ) time. ≤ n k � n � = n ( n − 1)( n − 2) × · · · × ( n − k + 1) ・ Number of k element subsets = k k ( k − 1)( k − 2) × · · · × 1 k ! ・ O ( k 2 n k / k !) = O ( n k ) . poly-time for k=17, but not practical 24 25 15 / 32 16 / 32 Common algorithm dominance classes Sublinear time Search in a sorted array. Given a sorted array A of n numbers, is a given number x in the array? Dominance class Example problem types O(log n) solution. Binary search. 1 Operations independent of input size (e.g., addition, min(x,y), etc.) log n Binary search lo ← 1 , hi ← n while (lo ≤ hi) { Operating on every element in an array n mid ← (lo + hi) / 2 n log n Quicksort, mergesort if (x < A[mid]) hi ← mid - 1 n 2 Operating on every pair of items else if (x > A[mid]) lo ← mid + 1 n 3 Operating on every triple of items else return yes } 2 n Enumerating all subsets of n items return no n ! Enumerating all orderings of n items 26 17 / 32 18 / 32

Python Algorithm Development Process Debugging in Python 1 Think hard about the problem you’re trying to solve. Specify the expected inputs for which you’d like to provide a solution, and the expected outputs. 1 Main strategy: run code in the interpreter to get instant feedback on 2 Describe a method to solve the problem using English and/or errors pseudo-code 2 Backup: Generous use of print statements 3 Start coding 3 Once code is running in functions: pdb.pm() (Python debugger Development/Debugging phase 1 post-mortem) Testing phase (for correctness) 2 Evaluation phase (performance) 3 Let’s use the insertion sort as an example of the development process in Python 20 / 32 21 / 32 Main strategy: run code in the interpreter Second strategy: print variables out during execution >>> for i in range(len(s)): >>> s = [2,7,4,5,9] ... minidx = i >>> ... for j in range(i,len(s)): >>> for i in range(s): ... print ’list: %s, i: %i, j: %i, minidx: %i’%(s,i,j,minidx) ... minidx = i ... if s[j]<s[minidx]: ... for j in range(i,len(s)): ... print "reassigning minidx %i < %i" %(s[j],s[minidx]) ... if s[j]<s[minidx]: ... minidx=j ... minidx=i ... s[i],s[minidx]=s[minidx],s[i] ... s[i],s[minidx]=s[minidx],s[i] ... ... list: [2, 7, 4, 5, 9], i: 0, j: 0, minidx: 0 Traceback (most recent call last): list: [2, 7, 4, 5, 9], i: 0, j: 1, minidx: 0 File "<stdin>", line 1, in <module> list: [2, 7, 4, 5, 9], i: 0, j: 2, minidx: 0 TypeError: range() integer end argument expected, got list. list: [2, 7, 4, 5, 9], i: 0, j: 3, minidx: 0 >>> s list: [2, 7, 4, 5, 9], i: 0, j: 4, minidx: 0 [2, 7, 4, 5, 9] list: [2, 7, 4, 5, 9], i: 1, j: 1, minidx: 1 >>> range(s) list: [2, 7, 4, 5, 9], i: 1, j: 2, minidx: 1 Traceback (most recent call last): reassigning minidx 4 < 7 File "<stdin>", line 1, in <module> list: [2, 4, 7, 5, 9], i: 1, j: 3, minidx: 2 TypeError: range() integer end argument expected, got list. reassigning minidx 5 < 7 >>> len(s) list: [2, 5, 7, 4, 9], i: 1, j: 4, minidx: 3 5 list: [2, 5, 7, 4, 9], i: 2, j: 2, minidx: 2 >>> range(len(s)) list: [2, 5, 7, 4, 9], i: 2, j: 3, minidx: 2 [0, 1, 2, 3, 4] reassigning minidx 4 < 7 list: [2, 5, 4, 7, 9], i: 2, j: 4, minidx: 3 list: [2, 5, 4, 7, 9], i: 3, j: 3, minidx: 3 22 / 32 23 / 32 list: [2, 5, 4, 7, 9], i: 3, j: 4, minidx: 3 list: [2, 5, 4, 7, 9], i: 4, j: 4, minidx: 4