Analysis of Algorithms What to analyze [Lewis/Denenberg 2.1, - PowerPoint PPT Presentation

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 1 C. Kessler, IDA, Link¨ opings Universitet, 2001. Analysis of Algorithms What to analyze [Lewis/Denenberg 2.1, Goodrich/Tamassia 3.5] � correctness � termination � efficiency Time efficiency [Lewis/Denenberg 2.2, Goodrich/Tamassia 3.6+3.7] � growth rate � worst case, expected case, amortized � analysis techniques for iterative algorithms � analysis techniques for recursive algorithms Mathematical background [Lewis/Denenberg 1.3 (except of pp. 26-32); Goodrich/Tamassia 3.3]

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 2 C. Kessler, IDA, Link¨ opings Universitet, 2001. Correctness “An algorithm must not give the wrong answer.” [Lewis/Denenberg] A function fact for computing factorial must not return 6 for the call fact ( 2 ) . Which answers are wrong? � the user knows that, or � a specification of legal inputs and corresponding correct answers is needed. An algorithm is correct iff for any legal input � the computation terminates , and � the answer is as specified .

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 3 C. Kessler, IDA, Link¨ opings Universitet, 2001. Termination (1) An algorithm should � produce an answer in a finite number of steps � for any legal input Example: Algorithm for squaring an integer using n 2 ) 2 � 1 + 2 n � 1 ( n 8 n = 2 N ) : integer function Square ( integer n = 0 return 0 if n 6 = 0 return Square � 1 + 2 � 1 + 1 if n ( n ( n ) � ) < 0 . does not terminate for n

C. Kessler, IDA, Link¨ TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 4 opings Universitet, 2001. Termination (2) Termination is a difficult problem: ) : integer function OddEven ( integer m [Lewis/Denenberg, Algorithm 2.1] n m > 1 do while n if n is even then = 2 n n else 3 n + 1 n return m � 1 ? Does this algorithm compute the identity function for all m

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 5 C. Kessler, IDA, Link¨ opings Universitet, 2001. Efficiency Different algorithms may solve the same problem. How to compare them? � Resources used by an algorithm: – memory – time � Analysis of time efficiency should be: – machine-independent – valid for all legal data � We compare: – time growth-rate for growing size of (input) data (scalability) – mostly for worst-case problem instances

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 6 C. Kessler, IDA, Link¨ opings Universitet, 2001. Efficiency (2) [ 0 � 1 ) : integer function TableSearch ( table ] , key K < key > T :: n (1) for i from 0 to n � 1 do (2) if T = K then return i [ i ] � 1 (3) if T [ i ] > K then return � 1 (4) return What is the worst-case problem instance? Worst case time: n ( t 1 + t 2 + t 3 + t 4 � )

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 7 C. Kessler, IDA, Link¨ opings Universitet, 2001. Efficiency (3) [ 0 � 1 ) : integer function BinSearch ( table T ] ; key K :: n � 0 then return � 1 (0) if n 0 ; � 1 (1) l u n (2) while l < u do ) = 2 (3) mid b ( l + u c (4) if K ] then return mid = T [ mid � 1 else l + 1 (5) if K < T [ mid ] then u mid mid � 1 (6) if K ] then return l else return = T [ l Worst case time: t 0 + t 1 + maxit ( t 2 + t 3 + t 4 + t 5 + t 6 � ) where maxit = maximal number of iterations of the while loop

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 8 C. Kessler, IDA, Link¨ opings Universitet, 2001. Efficiency (4) = 1 ; 2 n How to compute maxit for n ::: ? ; ( 1 = 0 , maxit n - n/2 - 1 n/2 ) ( 2 = 1 , maxit ) ( 3 = 1 , maxit n/2 ) ( 4 = 2 , maxit ) ( 5 = 2 , maxit ) ( 6 = 2 n/2 - 1 maxit ) ; 1 � � � = 1 = 2 maxit ( n + maxit ( b n ) c ) b log 2 n maxit ( n ) = c

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 9 C. Kessler, IDA, Link¨ opings Universitet, 2001. Estimating execution time for iterative programs Elementary operation takes / can be bound by a constant time Sequence of operations takes the sum of the times of its components Loop ( for... and while... ) the time of the body multiplied by number of repetitions (in the worst case) Conditional statement ( if...then...else... ) the time for evaluating and checking the condition plus maximum of the times for then and else parts.

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 10 C. Kessler, IDA, Link¨ opings Universitet, 2001. Example: Independent Nested Loops Matrix-vector product (here, for a quadratic matrix) > 0 R n , R n ; n , given: vector matrix A with n x ~ 2 2 R n with compute: vector y ~ 2 n ∑ = 1 that is, y = A x y i a ij x j i :::; n ~ � ~ ; = ; ; = 1 j < real > x [ 1 [ 1 ; 1 [ 1 : n ] , A ) : array procedure matvec ( array :: n :: n :: n < real > y ] ] (1) for i from 1 to n do 0 : 0 (2) y [ i ] for j from 1 to n do (3) (4) y [ i ] y [ i ] + A [ i ; j � x [ j ] ] return y + n 2 Time: n ( t 1 + t 2 ( t 3 + t 4 ) )

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 11 C. Kessler, IDA, Link¨ opings Universitet, 2001. Example: Dependent Nested Loops Prefix-Sums N n , given: Vector x i ~ 2 ∑ N n with = 1 compute: “Prefix-sums” vector y y i x j i :::; n ~ 2 = ; ; = 1 j A straightforward algorithm follows directly from the definition: > x [ 1 [ 1 : n ] ) : array procedure prefixsum ( array < integer :: n < integer > y ] (1) for i from 1 to n do 0 : 0 (2) y [ i ] for j from 1 to i do (3) (4) y [ i ] y [ i ] + x [ j ] return y ( 1 + 2 � 1 Total time: t ( n = n ( t 1 + t 2 ( n + n )( t 3 + t 4 ) ) + + ::: + ) ) + 1 + n ( n = n ( t 1 + t 2 ( t 3 + t 4 2 ) Remark: There exists a better, linear-time algorithm! ) )

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 12 C. Kessler, IDA, Link¨ opings Universitet, 2001. Principles of Algorithm Analysis An algorithm should work for (input) data of any size. (Example TableSearch : input size is the size of the table.) Show the resource (time/memory) used as an increasing function of input size . Focus on the worst case performance. Ignore constant factors analysis should be machine-independent; more powerful computers introduce speed-up by constant factors. Study scalability / asymptotic behaviour for large problem sizes: ignore lower-order terms, focus on dominating terms.

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 13 C. Kessler, IDA, Link¨ opings Universitet, 2001. Commonly used increasing functions ; α be real numbers. Let x ; y ; a ; b > 0 of x > 0 Logarithm to the base b = log b x iff b y y = x > 1 . We consider only cases where a ; b Changing base – multiplication by a constant factor: ( a log a x log b x = log b = log a x log b a ) Power function of x x α where α > 0 , such as x , x 1 = 2 , x 2 , ... Exponential function of x c x for some c > 1 Combinations of these, e.g. x log 2 x

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 14 C. Kessler, IDA, Link¨ opings Universitet, 2001. How functions grow log 2 n n log 2 n n 2 2 n n n 2 1 2 2 4 4 � 10 4 16 4 16 64 256 6 : 5 � 10 19 64 6 64 384 4096 1 : 84 � 10 19 µ sec = 2 � 10 8 days = 5845 centuries 1 : 84 : 14

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 15 C. Kessler, IDA, Link¨ opings Universitet, 2001. Asymptotic analysis: Dominance relation Consider two growing functions f , g from natural numbers to positive real numbers: f(n) c g(n) g(n) n n 0 ! ∞ f dominates g iff f ) increases without bounds for n ( n ) = g ( n > 0 , that is, for a given constant factor c there is some threshold value n 0 2 N = n 2 dominates g ( n ) = 7 n .) such that f ) for all n > n 0 . ( n > c � g ( n (Ex.: f ( n ) )

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 16 C. Kessler, IDA, Link¨ opings Universitet, 2001. Asymptotic analysis: Order Notation (1) Motivation: + comparing growth rates of increasing functions + estimating efficiency of algorithms by reference to simple functions + abstraction from constant factors ! classes of functions

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 17 C. Kessler, IDA, Link¨ opings Universitet, 2001. Asymptotic analysis: Order Notation (2) f , g growing functions from natural numbers to positive real numbers > 0 , n 0 � 0 such that ) iff there exist c f is (in) O ( g for all n f ( n � c g ( n > n 0 ) ) Intuition: Apart from constant factors, f grows at most as quickly as g > 0 , n 0 � 0 such that f is (in) Ω ) iff there exist c ( g for all n f ( n � c g ( n > n 0 ) ) Intuition: Apart from constant factors, f grows at least as quickly as g Ω () is the converse of O , i.e. f is in Ω ) iff g is in O ( g ( f ) f is (in) Θ ( g ) iff f )) and g ( n 2 O ( g ( n ( n 2 O ( f ( n ) ) )) Intuition: Apart from constant factors, f grows exactly as quickly as g

TDDB56 DALGOPT-D – Lecture 2: Analysis of algorithms. Page 18 C. Kessler, IDA, Link¨ opings Universitet, 2001. Asymptotic analysis: Order Notation (3) f(n) 4 g(n) f(n) 3 g(n) 3 g(n) 2 g(n) 2 g(n) g(n) g(n) n n n 0 n n n 0 0 0 4 g(n) 3 g(n) 2 g(n) f(n) g(n) n n 0 n 0