ECE 242 Data Structures Lecture 2 Algorithm Analysis September 11, 2009 ECE242 L2: Algorithm Analysis Overview ° Problem: How can we quantify if one algorithm is better than another? • Primarily concerned about growth of runtime as input set becomes larger • Important concept for programmers ° Complexity analysis is critical to allowing for scalable software ° Lewis and Chase text has nice discussion on this topic. • Note examples of nested loops September 11, 2009 ECE242 L2: Algorithm Analysis
Analysis of Algorithms ° How do we compare two implementations of the same interface? ° We need to account for all possible inputs. ° Our performance measure should be conservative ° Need metrics which are independent of target computer, data input set size, other applications. September 11, 2009 ECE242 L2: Algorithm Analysis Algorithm Running Time How do we characterize the running time of a method? public boolean contains (int x) { int i = 0; while ((i < data.length) && (data[i] != x)) { i++; } return (i != data.length); } • Count number of time units utilized • Parameter is the size of the collection. September 11, 2009 ECE242 L2: Algorithm Analysis
Big-O Notation ° “ “ “ “ O ” stands for “ the order of ” ° Also called asymptotic analysis ° O(N): linear to N ° O(logN): order of log N • E.g. N and N/2 ° O(N) > O(logN) • E.g. log 2 N and log 3 N September 11, 2009 ECE242 L2: Algorithm Analysis Defining Running Time We can parameterize the running time of a method by its input. public int sum(int n) { int sum = 0; n steps for (int i = 1; i <= n; i++) O(n) sum += i; return sum; } public int sum(int n) { 3 steps O(1) return (int) n*(n+1)/2; } September 11, 2009 ECE242 L2: Algorithm Analysis
Defining Running Time We can parameterize the running time of a method by its input. public int sum(int n) { int sum = 0; public int sum(int n) { for (int i = 1; i <= n; i++) return (int) n*(n+1)/2; sum += i; } return sum; } 3 operations n operations September 11, 2009 ECE242 L2: Algorithm Analysis Worst-case Analysis ° Which implementation is better? ° Any single measure of running time should describe performance over all possible inputs. ° Worst-case reasoning is conservative • Can serve as a baseline measure for comparing implementations. ° Also can consider average case and best case September 11, 2009 ECE242 L2: Algorithm Analysis
Growth Functions ° It's not usually necessary to know the exact growth function ° The key issue is the asymptotic complexity of the function – how it grows as n increases ° Determined by the dominant term in the growth function ° This is referred to as the order of the algorithm ° We often use Big-O notation to specify the order, such as O(n 2 ) September 11, 2009 ECE242 L2: Algorithm Analysis Growth Functions ° Analysis is defined in general terms, based on: • the problem size (ex: number of items to sort) • key operation (ex: comparison of two values) ° A growth function shows the relationship between the size of the problem (n) and the time it takes to solve the problem t(n) = 15n 2 + 45 n O( n 2 ) Growth here is dominated by the n 2 term for addition and subtraction September 11, 2009 ECE242 L2: Algorithm Analysis
Some growth functions and their asymptotic complexity Locate worst growth factor in each equation Note some equation can have multiplicative terms (e.g. n* logn) September 11, 2009 ECE242 L2: Algorithm Analysis Analyzing Loop Execution ° A loop executes a certain number of times (say n) ° Thus the complexity of a loop is n times the complexity of the body of the loop ° When loops are nested, the body of the outer loop includes the complexity of the inner loop ° The following loop is O(n) because the loop executes n times and the body of the loop is O(1): for (int i=0; i<n; i++) { x = x + 1; } September 11, 2009 ECE242 L2: Algorithm Analysis
Analyzing Loop Execution ° The following loop is O(n 2 ) because the loop executes n times and the body of the loop, including a nested loop, is O(n): for (int i=0; i<n; i++) { x = x + 1; for (int j=0; j<n; j++) { y = y - 1; } } September 11, 2009 ECE242 L2: Algorithm Analysis Asymptotic Notation ° Method A is 10 n ² - 5 milliseconds to process n elements. ° Method B is 100 n + 200 milliseconds. September 11, 2009 ECE242 L2: Algorithm Analysis
Big-O notation ° O(1): constant ° O(N) > O(logN) > O(1) • E.g. initialize i ° Ignore less dominating term. ° Ignore constant • E.g. N/2+1 is O(N) ° O(N): < C · N linear time ° O(logN): < C · log 2 N time September 11, 2009 ECE242 L2: Algorithm Analysis O(1) Example i=0; j=0; ° Number of statements k=0; ° Cost • Constant • O(1) Algorithms which take time which is independent of the input size are O(1) September 11, 2009 ECE242 L2: Algorithm Analysis
O (N) Example int sum = 0; for (int i=0; i<N;i++) ° Number of statements: sum = sum + a[i]; ° Cost • C · N+1 • O(N) Most loops are O(n), where n is the index limit September 11, 2009 ECE242 L2: Algorithm Analysis O (N 2 ) Example int sum = 0; for (int i=0; i<N; i++) for (int j=0; j<N; j++) ° Number of statements sum = sum + a[i][j]; ° Cost • Inner loop: C · N • N · C · N = O(N 2 ) Nested loops – note that constants are ignored September 11, 2009 ECE242 L2: Algorithm Analysis
O (logN) Example int sum = 0; Multiplier/divider in modifier for (i=N;i>0;i=i/2) sum = sum + a[i]; ° Cost • O(logN) ° log 2 3 · log 3 N = log 2 N ° log 2 3 is a constant ° log 2 N, log 3 N, log 4 N are all O(logN) September 11, 2009 ECE242 L2: Algorithm Analysis Another O (N 2 ) Example int sum = 0; for (int i=0; i<N; i++) for (int j=0; j<i; j++) ° Number of statements in inner loop sum = sum + a[i][j]; i 0 1 2 3 … # of statements in 0 1 2 3 … inner loop ° Cost • Summing all “ i ” s: 0+1+2+3+ … +(N-1)=N(N-1)/2=1/2N 2 -1/2N • O(N 2 ) September 11, 2009 ECE242 L2: Algorithm Analysis
O (NlogN) Example int sum = 0; for (i=0;i<N;i=++) for (j = N; j>0; j=j/2) sum = sum + a[i][j]; ° Number of statements in inner loop: log 2 N+1 ° Total: N(log 2 N+1) ° Cost • O(NlogN) September 11, 2009 ECE242 L2: Algorithm Analysis More Examples: O(N) sum = 0; for (i=1; i<=N; i++) { sum = sum + i ; for( j=1; j<10000; j++) sum = sum + i*j ; } ° Number of statements: • C · N+1 ° Cost • O(N) September 11, 2009 ECE242 L2: Algorithm Analysis
More Examples: O(N 2 ) sum1 = 0; sum2 = 0; for (i=1; i<=N; i++) { sum1 = sum1 + i ; for(int j=20*N; j<200*N; j++) sum2 = sum2 + i; } ° Number of statements: • C1 · N + C2 · N 2 + 1 ° Cost • O(N 2 ) September 11, 2009 ECE242 L2: Algorithm Analysis More Examples: O (N 3 ) int sum = 0; for (int i=0; i<N; i++) for (int j=0; j<N; j++) for (int k=0; k<j; k++) sum = sum + a[i][j][k]; ° Cost for loop j and loop k • O(N 2 ), we talked about this before ° So, finally cost is • N * O(N 2 ) = O(N 3 ) September 11, 2009 ECE242 L2: Algorithm Analysis
More Examples: O (logN) int sum = 0; for (int i=1; i<=N; i=i*5) sum = sum + a[i]; ° Cost • O(logN) Note multiplication in the loop modifier September 11, 2009 ECE242 L2: Algorithm Analysis More Examples: Multiple Inputs int sum = 0; for (int i=0;i<M;i++) for (int j = N; j>0; j=j/10) sum = sum + a[i]+b[j]; ° Number of statements in inner loop: log 10 N+1 ° Total: M(log 10 N+1) ° Cost • O(MlogN) September 11, 2009 ECE242 L2: Algorithm Analysis
Asymptotic Notation ° � ( n ): • This notation indicates a “tight” bound on the complexity of a function • For a function f, f is an element of O(g) if an only if, for some constant c > 0, there is a constant p >= 0 such that f(n) < cg(n) for any n >= p. 3 n ² and 0.2 n ² are both in � ( n ²) and O( n ²) • • O(n) indicates an “upper” bound ° A function's order is not changed by adding or subtracting a function in a lower order. 2 � – n + log n is in � (2 � ) September 11, 2009 ECE242 L2: Algorithm Analysis Asymptotic Notation ° Factorial takes time which is at least in O (2 � ). ° Therefore we can prove that n ! is in � (2 � ). N! = 1*2*3*4…..*N 2 � = 1*2*2*2*….. We won’t be using � notation very much in this course September 11, 2009 ECE242 L2: Algorithm Analysis
Summary ° Complexity analysis mostly involves examining loops ° Attempt to characterize growth of functions. • This will be important when we examine searching and sorting ° Big O notation helps us simplify complexity definitions – worst case • Ignore constant and lower order terms • Big Omega ( � ) examines lower bounds • Big Theta ( � ) examines specific case ° Computer scientists often use proofs to defend complexity claims September 11, 2009 ECE242 L2: Algorithm Analysis
Recommend
More recommend
