cs 171 introduction to computer science ii algorithm
play

CS 171: Introduction to Computer Science II Algorithm Analysis Li - PowerPoint PPT Presentation

CS 171: Introduction to Computer Science II Algorithm Analysis Li Xiong Announcement/Reminders Hw3 due Friday Quiz 2 on October 17, Wednesday (after Spring break, based on class poll) Linked List, Algorithm Analysis Road Map


  1. CS 171: Introduction to Computer Science II Algorithm Analysis Li Xiong

  2. Announcement/Reminders � Hw3 due Friday � Quiz 2 on October 17, Wednesday (after Spring break, based on class poll) – Linked List, Algorithm Analysis

  3. Road Map � Algorithm Analysis � Simple sorting

  4. Algorithm Analysis � An algorithm is a method for solving a problem expressed as a sequence of steps that is suitable for execution by a computer (machine) � E.g. Search in an ordered array � E.g. N-Queens problem � E.g. N-Queens problem � We are interested in designing good algorithms � Linear search vs. binary search � Brute-force search vs. backtracking � Good algorithms � Running time � Space usage (amount of memory required)

  5. Running time of an algorithm � Running time typically increases with the input size (problem size) � Also affected by hardware and software environment � We would like to focus on the relationship between the running time and the input size ����� ��������� ������

  6. Experimental Studies � Write a program implementing the algorithm ���� � Run the program with inputs ���� of varying size and ���� composition ���� ��������� ���� � Use a method like ���� �������������������������� to get ���� ���� an accurate measure of the ���� actual running time ���� � Plot the results � � �� ��� ���������� � E.g. Stopwatch.java

  7. Limitations of Experiments � It is necessary to implement the algorithm, which may be difficult � In order to compare two algorithms, the same hardware and software environments must be hardware and software environments must be used used � Results may not be indicative of the running time on other inputs not included in the experiment.

  8. Algorithm Analysis - insight � Total running time of a program is determined by two primary factors: � Cost of executing each statement (property of computer, Java compiler, OS) � Frequency of execution of each statement � Frequency of execution of each statement (property of program and input)

  9. Algorithm Analysis � Algorithm analysis: � Determine frequency of primitive operations � Characterizes it as a function of the input size � A primitive operation: � corresponds to a low-level (basic) computation with � corresponds to a low-level (basic) computation with a constant execution time � E.g. Evaluating an expression; assigning a value to a variable; indexing into an array � Benefit: � Takes into account all possible inputs � independent of the hardware/software environment

  10. Average-case vs. worst-case � An algorithm may run faster on some inputs than it does on others (with the same input size) � Average case: taking the average over all possible inputs of the same size � Depends on input distribution � Depends on input distribution � Best case � Worst case � Easier analysis � Typically leads to better algorithms

  11. Misleading Average A statistician who put her head in the oven and her feet in the refrigerator. She said, “On average, I feel just fine.”

  12. Misleading Average ������������������������������������������������������������

  13. Loop Analysis � Programs typically use loops to enumerate through input data items � Count number of operations or steps in loops � Each statement within the loop is counted as a step a step

  14. Example 1 ������ ��� � ���� ��� ���� � � �� � � �� � ��� � ��� �� ��������� � How many steps? Only count the loop statements (update to the loop variable i is ignored)

  15. Example 1 ������ ��� � ���� ��� ���� � � �� � � �� � ��� � ��� �� ��������� � How many steps? Only count the loop statements (update to the loop variable i is ignored) n

  16. Example 2 ������ ��� � ���� ��� ���� � � �� � � �� � �� �� � ��� �� ��������� � How many steps?

  17. Example 2 ������ ��� � ���� ��� ���� � � �� � � �� � �� �� � ��� �� ��������� � How many steps? Loops will be executed n/2 times: n/2

  18. Example 3 – Multiple Loops ��� ���� � � �� � � �� � ��� � ��� ���� � � �� � � �� � ��� � ��� � � ���� ��� �� �� � � How many steps?

  19. Example 3 – Multiple Loops ��� ���� � � �� � � �� � ��� � ��� ���� � � �� � � �� � ��� � ��� � � ���� ��� �� �� � � How many steps? Nested loops, each loop will be executed n times: n 2

  20. Increase of Cost w.r.t. n � Example 1: n � Example 2: n/2 � Example 3: 2n 2 � What if n is 100? What if n is 3 times larger? What if n is 100? What if n is 3 times larger? � Example 1 and 2 are linear to the input size � Example 3 is quadratic to the input size

  21. Mathematical notations for algorithm analysis � The cost function can be complicated and lengthy mathematical expressions � E.g. 3n 3 + 20n 2 + 5 � We care about how the cost increases w.r.t. � We care about how the cost increases w.r.t. the problem size, rather than the absolute cost � Use simplified mathematical notions � Tilde notation � Big O notation

  22. Tilde Notation � Tilde notation: ignore insignificant terms � Definition: we write f(n) ~ g(n) if f(n)/g(n) approaches 1 as n grows � 2n+ 10 ~ 2n � 3n 3 + 20n 2 + 5 ~ 3n 3

  23. Big-Oh Notation � Given functions � � � �� and ������ � � � � , we say that � � � �� is �� � � � � � �� if there are ����� ����� positive constants � � and � � such that � and � � such that ��� � � � � ≤ �� � � ��� for �� ≥ � � �� � Example: � � + �� is � � � � � � �� ��� ����� � pick �� = �� and � �� = �� �

  24. Big-Oh Example ��������� � Example: the function ��� � � is not � � � � ���� ������� ��� � � � ≤ �� � ������ � �� ≤ � � �� ≤ � ����� ����� � The above ��� inequality cannot �� be satisfied since � must be a constant � � �� ��� ����� �

  25. Big-Oh and Growth Rate � The big-Oh notation gives an upper bound on the growth rate of a function � The statement “ � � � �� is � � � � � �� ” means that the growth rate of � � � �� is no more than the growth rate of � � � � rate of � � � � � We can use the big-Oh notation to rank functions according to their growth rate

  26. Important Functions in Big-Oh Analysis � Constant: � � Logarithmic: log � � Linear: � � N-Log-N: �� log � � Quadratic: � � � Cubic: � � � Cubic: � � � Polynomial: � � � Exponential: � � � Factorial: ��

  27. Growth Rate � � �� � � In terms Exponential � � �� � of the order : � � � � � exponentials > Polynomial polynomials > polynomials > � � � � � � � � � � logarithms > � � � ��� � � Log-linear constant. � � � � Linear Increasing order � ���� � � Log � ��� Constant

  28. Big-Oh Analysis � Write down cost function � � � �� 1.Look for highest-order term (tilde notation) 2.Drop constant factors � Examples � �� � ����� � ��� � ������������

  29. Useful Approximations � Harmonic sum 1 + 1/2 + 1/3 + … + 1/N ~ lnN � Triangular sum 1 + 2 + 3 + … + N = N(N+1)/2 ~ N 2 /2 1 + 2 + 3 + … + N = N(N+1)/2 ~ N /2 � Geometric sum 1 + 2 + 4 + … + N = 2N -1 ~ 2N when N = 2 n � Stirling’s approximation lg N! = lg1 + lg2 + lg3 + … + lgN ~ NlgN

Recommend


More recommend