Program Analysis, Python and Logarithms Tyler Moore CSE 3353, SMU, - PDF document

Notes Program Analysis, Python and Logarithms Tyler Moore CSE 3353, SMU, Dallas, TX January 29, 2013 These slides have been adapted from the slides written by Prof. Steven Skiena at SUNY Stony Brook, author of Algorithm Design Manual. For more information see http://www.cs.sunysb.edu/~skiena/ Problem of the Day Notes Find two functions f ( n ) and g ( n ) that satisfy the following relationship. If no such f and g exist, write “None”. 1 f ( n ) = o ( g ( n )) and f ( n ) � = Θ( g ( n )) 2 f ( n ) = Θ( g ( n )) and f ( n ) = o ( g ( n )) 3 f ( n ) = Θ( g ( n )) and f ( n ) � = O ( g ( n )) 4 f ( n ) = Ω( g ( n )) and f ( n ) � = O ( g ( n )) Hint: what does “little oh” notation mean? f ( n ) = o ( g ( n )) ⇐ ⇒ g ( n ) dominates f ( n ) So n 2 = o ( n 3 ) since n 3 dominates n 2 . 2 / 20 Notes You should come to accept the dominance ranking of the basic functions: n ! ≫ 2 n ≫ n 3 ≫ n 2 ≫ n log n ≫ n ≫ √ n ≫ log n ≫ 1 3 / 20 Reasoning About Efficiency Notes Grossly reasoning about the running time of an algorithm is usually easy given a precise-enough written description of the algorithm. When you really understand an algorithm, this analysis can be done in your head. However, recognize there is always implicitly a written algorithm/program we are reasoning about. 4 / 20

Selection sort (C) Notes Outer loop goes around s e l e c t i o n s o r t ( int s [ ] , int n) 1 n times { 2 int i , j ; Inner loop goes around 3 int min ; at most n − 1 times for 4 for ( i =0; i < n ; i++) { 5 each iteration of the min=i ; 6 outer loop for ( j=i +1; j < n ; j++) 7 Thus selection sort i f ( s [ j ] < s [ min ] ) min=j ; 8 takes at most swap(&s [ i ] ,& s [ min ] ) ; 9 n × ( n − 1) → O ( n 2 ) } 10 time in the worst case. 5 / 20 But what if we were more careful about the analysis? Notes An exact count of the number of times the if statement is executed is given by: n − 1 n − 1 � � S ( n ) = 1 i =0 j = i +1 n − 1 � = n − i − 1 i =0 S ( n ) = ( n − 1) + ( n − 2) + · · · + 2 + 1 + 0 = ( n − 1)( n − 2) 2 Thus the worst-case running-time is Θ( n 2 ). 6 / 20 Using upper and lower bounds to establish running time Notes First let’s find an upper bound We loop through at most n terms For each term, the nested loop iterates at most n − 1 times Hence, S ( n ) ≤ n ( n − 1) = O ( n 2 ) Next let’s find a lower bound For at least n / 2 terms, the nested loop iterates at least n / 2 times 4 n 2 = Ω( n 2 ) Hence, S ( n ) ≥ n / 2 × n / 2 = 1 Since S ( n ) = O ( n 2 ) and S ( n ) = Ω( n 2 ), S ( n ) = Θ( n 2 ) 7 / 20 Efficiency Analysis of Insertion Sort (C) Notes i n s e r t i o n s o r t ( int s [ ] , int n) 1 Outer loop goes around { 2 n − 1 times int i , j ; 3 int min ; 4 What about the inner for ( i =1; i < n ; i++) { 5 while loop? j = i ; 6 Depends on the input, while (( j > 0) && 7 so we can assume the ( s [ j ] < s [ j − 1])) { 8 swap(&s [ j ] ,& s [ j − 1]); worst case i times 9 j = j − 1; 10 Since i < n , we can } 11 assume i = n and so } 12 O ( n 2 ) } 13 8 / 20

We already know Java and C++. Why learn Python? Notes Python has far less overhead than Java/C++ for the programmer. Python is closer to psuedo-code on the English-pseudocode-code spectrum than Java or C/C++, but it actually executes! Python is handy for data manipulation and transformation, and anything ”quick and dirty.” Python is very powerful, thanks to all those extension modules. 9 / 20 Python Resources Notes Download from python.org for all major platforms I’ve written an online tutorial: http://lyle.smu.edu/~tylerm/courses/cse3353/python.html Beginning Python, Magnus Lie Hetland: http://link.springer. com/book/10.1007/978-1-4302-0634-7/page/1 10 / 20 Back to the Selection Sort (C) Notes s e l e c t i o n s o r t ( int s [ ] , int n) 1 { 2 int i , j ; 3 int min ; 4 for ( i =0; i < n ; i++) { 5 min=i ; 6 for ( j=i +1; j < n ; j++) 7 i f ( s [ j ] < s [ min ] ) min=j ; 8 swap(&s [ i ] ,& s [ min ] ) ; 9 } 10 11 / 20 Selection Sort, now in Python Notes 1 def s e l e c t i o n s o r t ( s ) : Whitespace ””” 2 matters! Input : l i s t s to be s o r t e d 3 Dynamic, implicit Output : s o r t e d l i s t 4 typing ””” 5 for i in range ( l e n ( s ) ) : Iterators make 6 #don ’ t name min s i n c e r e s e r v e d word 7 looping easy minidx=i 8 Has built-in lists, for j in range ( i +1, l e n ( s ) ) : 9 dictionaries i f s [ j ] < s [ minidx ] : 10 minidx=j 11 Multiple variable s [ i ] , s [ minidx ]= s [ minidx ] , s [ i ] 12 assignment per return s 13 line 12 / 20

Problem of the Day due Thursday at beginning of class Notes 1 Install Python on your computer 2 Write Python code to implement an insertion sort. You may refer to the C code on p. 4 of the ADM You may also refer to the Python code implementing selection sort at http://lyle.smu.edu/~tylerm/courses/cse3353/code/l3.zip Submit the code via Blackboard (link made available later today) 13 / 20 Logarithms Notes It is important to understand deep in your bones what logarithms are and where they come from. A logarithm is simply an inverse exponential function. Saying b x = y is equivalent to saying that x = log b y . Logarithms reflect how many times we can double something until we get to n , or halve something until we get to 1. 14 / 20 Binary Search and Logarithms Notes In binary search we throw away half the possible number of keys after each comparison. Thus twenty comparisons suffice to find any name in the million-name Manhattan phone book! Question: how many times can we halve n before getting to 1? 15 / 20 Logarithms and Binary Trees Notes How tall a binary tree do we need until we have n leaves? → The number of potential leaves doubles with each level. How many times can we double 1 until we get to n ? 16 / 20

Logarithms and Bits Notes How many bits do you need to represent the numbers from 0 to 2 i − 1? 17 / 20 Logarithms and Multiplication Notes Recall that log a ( xy ) = log a ( x ) + log a ( y ) This is how people used to multiply before calculators, and remains useful for analysis. What if x = a ? 18 / 20 The Base is not Asymptotically Important Notes Recall the definition, c log c x = x and that log b a = log c a log c b So for a = 2 and c = 100: log 2 n = log 100 n log 100 2 1 Since log 100 2 = 6 . 643 is a constant, we can ignore it when calculating Big Oh 19 / 20 Federal Sentencing Guidelines Notes F1.1. Fraud and Deceit; Forgery; Offenses Involving Altered or Counterfeit Instruments other than Counterfeit Bearer Obligations of the United States. (a) Base offense Level: 6 (b) Specific offense Characteristics (1) If the loss exceeded $2,000, increase the offense level as follows: Loss(Apply the Greatest) Increase in Level (A) $2,000 or less no increase (B) More than $2,000 add 1 (C) More than $5,000 add 2 The increase in punishment level (D) More than $10,000 add 3 (E) More than $20,000 add 4 grows logarithmically in the (F) More than $40,000 add 5 amount of money stolen. (G) More than $70,000 add 6 (H) More than $120,000 add 7 Thus it pays to commit one big (I) More than $200,000 add 8 crime rather than many small (J) More than $350,000 add 9 (K) More than $500,000 add 10 crimes totaling the same (L) More than $800,000 add 11 amount. (M) More than $1,500,000 add 12 (N) More than $2,500,000 add 13 In other words, Make the (O) More than $5,000,000 add 14 Crime Worth the Time (P) More than $10,000,000 add 15 (Q) More than $20,000,000 add 16 (R) More than $40,000,000 add 17 (Q) More than $80,000,000 add 18 20 / 20