Defining Efficiency CSE 421: Intro Algorithms “Runs fast on typical real problem instances” 2: Analysis Pro: sensible, bottom-line-oriented Summer 2007 Larry Ruzzo Con: moving target (diff computers, compilers, Moore’s law) highly subjective (how fast is “fast”? what’s “typical”?) 1 2 Efficiency Measuring efficiency Time ≈ # of instructions executed in a simple Our correct TSP algorithm was incredibly slow programming language Basically slow no matter what computer you have only simple operations (+,*,-,=,if,call,…) We want a general theory of “efficiency” that is each operation takes one time step Simple each memory access takes one time step Objective no fancy stuff (add these two matrices, copy this long Relatively independent of changing technology string,…) built in; write it/charge for it as above But still predictive - “theoretically bad” algorithms should No fixed bound on the memory size be bad in practice and vice versa (usually) 3 4 1
Complexity We left out things but... T analysis n Things we’ve dropped Problem size n memory hierarchy Worst-case complexity: max # steps algorithm disk, caches, registers have many orders of magnitude takes on any input of size n differences in access time not all instructions take the same time in practice Best-case complexity: min # steps algorithm different computers have different primitive instructions takes on any input of size n However, Average-case complexity: avg # steps algorithm the RAM model is useful for designing algorithms and takes on inputs of size n measuring their efficiency one can usually tune implementations so that the hierarchy etc. is not a huge factor 5 6 Pros and cons: Why Worst-Case Analysis? Best-case Appropriate for time-critical applications, e.g. avionics unrealistic oversell Average-case Unlike Average-Case, no debate about what the right definition is over what probability distribution? (different people may have different “average” problems) If worst >> average, then (a) alg is doing something pretty subtle, & (b) are hard instances really that rare? analysis often hard Analysis often easier Worst-case a fast algorithm has a comforting guarantee Result is often representative of "typical" problem instances maybe too pessimistic Of course there are exceptions… 7 8 2
General Goals Complexity Characterize growth rate of (worst-case) run time as The complexity of an algorithm associates a number a function of problem size, up to a constant factor T(n), the worst-case time the algorithm takes, with Why not try to be more precise? each problem size n. Technological variations (computer, compiler, OS, …) easily 10x or more Mathematically, Being more precise is a ton of work T: N+ → R+ A key question is “scale up”: if I can afford to do it today, how much longer will it take when my business problems that is T is a function that maps positive integers (giving are twice as large? (E.g. today: cn 2 , next year: c(2n) 2 = problem sizes) to positive real numbers (giving number 4cn 2 : 4 x longer.) of steps). 9 10 Complexity Complexity T(n) T(n) 2n log 2 n Time Time n log 2 n Problem size Problem size 11 12 3
O-notation etc Examples Given two functions f and g:N → R 10n 2 -16n+100 is O(n 2 ) also O(n 3 ) f(n) is O(g(n)) iff there is a constant c>0 so that 10n 2 -16n+100 ≤ 11n 2 for all n ≥ 10 f(n) is eventually always ≤ c g(n) 10n 2 -16n+100 is Ω (n 2 ) also Ω (n) f(n) is Ω (g(n)) iff there is a constant c>0 so that 10n 2 -16n+100 ≥ 9n 2 for all n ≥ 16 f(n) is eventually always ≥ c g(n) Therefore also 10n 2 -16n+100 is Θ (n 2 ) f(n) is Θ (g(n)) iff there is are constants c 1 , c 2 >0 so that eventually always c 1 g(n) ≤ f(n) ≤ c 2 g(n) 10n 2 -16n+100 is not O(n) also not Ω (n 3 ) 13 14 Properties “One-Way Equalities” Transitivity. 2 + 2 is 4 2n 2 + 5 n is O(n 3 ) If f = O(g) and g = O(h) then f = O(h). 2 + 2 = 4 2n 2 + 5 n = O(n 3 ) If f = Ω (g) and g = Ω (h) then f = Ω (h). 4 = 2 + 2 O(n 3 ) = 2n 2 + 5 n If f = Θ (g) and g = Θ (h) then f = Θ (h). All dogs are mammals All mammals are dogs Additivity. Bottom line: If f = O(h) and g = O(h) then f + g = O(h). If f = Ω (h) and g = Ω (h) then f + g = Ω (h). OK to put big-O in R.H.S. of equality, but not left. If f = Θ (h) and g = O(h) then f + g = Θ (h). [Better, but uncommon, notation: T(n) ∈ O(f(n)).] 15 16 4
Working with O- Ω - Θ notation Working with O- Ω - Θ notation Claim: For any a, and any b>0, (n+a) b is Θ (n b ) Claim: For any a, b>1 log a n is Θ (log b n) (n+a) b ≤ (2n) b for n ≥ |a| log a b = x means a x = b = 2 b n b a log a b = b = cn b for c = 2 b so (n+a) b is O(n b ) ( a log a b ) log b n = b log b n = n (log a b )(log b n ) = log a n (n+a) b ≥ (n/2) b for n ≥ 2|a| (even if a <0) = 2 -b n b c log b n = log a n for the constant c = log a b = c’n for c’ = 2 -b So : so (n+a) b is Ω (n b ) log b n = � (log a n ) = � (log n ) 17 18 Big-Theta, etc. not always “nice” A Possible Misunderstanding? Insertion Sort: We have looked at Ω (n 2 ) (worst case) type of complexity analysis � � f ( n ) = n 2 , n even worst-, best-, average-case � � O(n) (best case) types of function bounds n , n odd � � O, Ω , Θ These two considerations are independent of each f(n) ≠ Θ ( n a ) for any a . other one can do any type of function bound with any type of Fortunately, such complexity analysis - measuring different things with nasty cases are rare same yardstick f(n log n) ≠ Θ ( n a ) for any a, either, but at least it ’ s simpler . 19 20 5
Asymptotic Bounds for Some Asymptotic Bounds for Some Common Functions Common Functions Polynomials: a 0 + a 1 n + … + a d n d is Θ (n d ) if a d > 0 Exponentials. 1.01 n For all r > 1 n 100 Logarithms: and all d > 0, O(log a n) = O(log b n) for any constants a,b > 0 n d = O(r n ). Logarithms: For all x > 0, log n = O(n x ) every exponential log grows slower grows faster than than every every polynomial polynomial 21 22 Polynomial time Why It Matters Running time is O(n d ) for some constant d independent of the input size n. 23 24 6
Geek-speak Faux Pas du Jour Domination f(n) is o(g(n)) iff lim n →∞ f(n)/g(n)=0 that is g(n) dominates f(n) “Any comparison-based sorting algorithm requires at least O(n log n) comparisons.” If a ≤ b then n a is O(n b ) Statement doesn't "type-check." If a < b then n a is o(n b ) Use Ω for lower bounds. Note: if f(n) is Θ (g(n)) then it cannot be o(g(n)) 25 26 Working with little-o Summary n 2 = o(n 3 ) [Use algebra]: Typical initial goal for algorithm analysis is to find a reasonably tight i.e., Θ if possible n 2 1 asymptotic i.e., O or Θ lim n �� n 3 = lim n �� n = 0 bound on usually upper bound n 3 = o(e n ) [Use L’Hospital’s rule 3 times]: worst case running time as a function of problem size This is rarely the last word, but often helps separate n 3 3 n 2 6 n 6 lim n �� e n = lim n �� e n = lim n �� e n = lim n �� e n = 0 good algorithms from blatantly poor ones - so you can concentrate on the good ones! 27 28 7
Recommend
More recommend
