Example The sorting problems is defined as follows: n Input set: - PDF document

Advanced Programming Complexity Algorithms and Complexity Algorithm An algorithm is a calculation procedure (composed by a finite number of steps) which solves a certain problem, working on a set of input values and producing a set of output values 2

Advanced Programming Complexity Example The sorting problems is defined as follows: n Input set: sequence of numbers <a 1 , a 2 ,…,a n > n Output set: permutation <a’ 1 , a’ 2 , …, a’ n > of input sequence so that a’ 1 ≤ a’ 2 ≤ … ≤ a’ n . 3 Insertion Sort Insert A[ j ] in the sorted sequence A[1.. j - - 1] 4

Advanced Programming Complexity Esempio 5 Analysis of algorithms § Analysing algorithms means to predict the amount of resources (I/O, memory, time) required by an algorithm during its execution. § Such analysis should be independent from the kind of hardware platform on which the algorithm is executed. § We will assume in the rest of the course that the hardware platform is a common machine with a single CPU ( Random Access Machine or RAM). 6

Advanced Programming Complexity Problem Dimension Analysis of algorithms is usually performed with respect to one or more parameters which characterize the dimensions of the problem. Example n In the sorting algorithm such parameter is the number of elements of the input sequence. n In the multiplication between integer numbers, dimension is given by the number of bits used to represent the operands. 7 Hypothesis n Each statement in the pseudo-code requires a fixed time n Each statement has a different execution time. 8

Advanced Programming Complexity t j is the number of times the statement is repeated, for a certain value of j Analysis of insertion sort times Insert A[ j ] in the ordered sequence A[1 .. j -1 ] 9 Execution time of insertion sort 10

Advanced Programming Complexity Best Case If the array is already sorted, the t j = 1 for every j. Then: Which can be written as follows T(n) = an + b 11 Worst Case If an array is sorted with inverse order at step j, j comparisons and swaps: t j = j. Remember that: 12

Advanced Programming Complexity Worst Case (2) Therefore In the worst case: T(n) = an 2 +bn+c 13 Average Case Worst case analysis is important as it defines an upper bound to resources required by an algorithm. In some cases the average case (or mean value ) is worthy of being analyzed. 14

Advanced Programming Complexity Importance of complexity analysis Analysis of resources of an algorithm (also known as complexity analysis) allows describing performance of the algorithm depending on problem dimension. Choosing an algorithm with lower complexity is the best choice independently from the used technology and it can make a difference for problems with big dimensions. 15 Example Suppose you can get the solution of a problem with 2 algorithms: n One with complexity T(n) = 2n 2 n The other with complexity T(n) = 50n log 2 n Suppose you have these two machines: n Intel Core i7 Extreme 965EE performing 76000 MIPS at 3.2 GHz for the first algorithm n AMD Athlon FX-60 performing 18000 MIPS at 2.6 GHz for the second algorithm. n MIPS = Millions of Instructions Per Second 16

Advanced Programming Complexity Example (II) Execution time is proportional to T(n) / MIPS: n When n = 1 K ( 10 3 ) Intel - T(n)/76000 = 2*10 6 / 76000 = 0.026 ms AMD – T(n)/18000 = 50*10 3 *log 2 (10 3 ) / 18000 =0.027 ms n When n = 1 M ( 10 6 ) Intel – 76000 MIPS : 26310 ms AMD – 18000 MIPS : 55,36 ms n When n = 1 G ( 10 9 ) Intel – 76000 MIPS : 26,3 Ms = 7305 hours ! = 304 days !!! AMD – 18000 MIPS : 83,04 s 17 Asymptotic Notation In complexity analysis the asymptotic notation is often used to discover the complexity of an algorithm when problem dimension increases. The asymptotic notation is based on 3 notations: n Theta Notation Θ n “Big O” Notation O n Omega Notation Ω . 18

Advanced Programming Complexity Θ Notation Given an algorithm of complexity T(n). T(n) = Θ ( g(n) ) if and only if there exist three positive constants c 1 , c 2 e n 0 so that 0 ≤ c 1 g(n) ≤ T(n) ≤ c 2 g(n) for each n ≥ n 0 . In this case g(n) is the asymptotic limit for T(n). 19 “Big O” Notation T(n) = O( g(n) ) if and only if there exist two positive constants c and n 0 so that: 0 ≤ T(n) ≤ c  g(n) for each n ≥ n 0 . In this case g(n) is a upper asymptotic limit for T(n). 20

Advanced Programming Complexity Ω Notation T(n) = Ω ( g(n) ) if and only if there exist two positive constants c and n 0 so that 0 ≤ c  g(n) ≤ T(n) for each n ≥ n 0 . In this case g(n) is a lower asymptotic limit for T(n). 21 22

Advanced Programming Complexity Theorem Given two functions g(n) and T(n), T(n) = Θ ( g(n) ) if and only if n T(n) = O( g(n) ) and T(n) = Ω ( g(n) ) 23 Problems Complexity 24

Advanced Programming Complexity Empirical complexity measure Using the clock() function: #include <time.h> int main(){ clock_t b= clock(); test_function(); clock_t e = clock(); double elapsed=(double)(e – b)/CLOCKS_PER_SEC; printf("Elapsed: %f seconds\n",elapsed); return 0; } 25 Example strcpy(destination, source); Vs. mycpy(destination, source); 26

Advanced Programming Complexity Example O(n 2 ) strcpy(destination, source); void mycpy(char* d,char* s){ char* src; for(src=s;*src!='\0';++src){ Vs. char* org=s; char* dst=d; mycpy(destination, source); for(; origin<=src; ++org,++dst){ *dst = *origin; } } }; 27 Measurement code int main(){ int i,j; printf("Function,n,Clocks\n"); int n[10]={100,200,300,/*…*/,2000,2500,3000}; for(j=1;j<10;++j){ for(i=0; i<N_MEASURE;++i){ clock_t b= clock(); test_strcpy(n[j]); clock_t e = clock(); printf("strcpy(),%d,%d\n",n[j],(int)(e-b)); }} /*…*/ } 28

Advanced Programming Complexity Results 14 s 16000000 14000000 y = 1.5644x 2 - 47.556x R ² = 0.99922 12000000 10000000 strcpy() 8000000 mycpy() 6000000 4000000 463 μ s 2000000 y = 0.1498x R ² = 0.95331 0 0 500 1000 1500 2000 2500 3000 29