What makes a good algorithm? Does what its supposed to Analysis of - PDF document

What makes a good algorithm? • Does what it’s supposed to Analysis of Algorithms • Easy to understand, code, and debug • Efficient – Makes efficient use of computer’s resources • Memory • Speed Slides courtesy of Prof. Joe Geigel Complexity Suppose we’re given 2 algorithms: • Both do what they’re supposed to, • (Resource) Complexity normally refers to the rate at which the storage or time needs grow as a • Both are pretty easy to understand, function of the input to the algorithm. • � Must decide on how each algorithm – T(n) = time complexity (amount of time an algorithm handles will take based on input of size n) – S(n) = space complexity (amount of space an algorithm – Space (memory) will take based on input of size n) – Time (computation) • For the rest of this lecture, we’ll only consider T(n). But the discussion can also be applied to S(n). Units Units • What does T(n) return? • What does time in T(n) mean? – Instead we say that T(n) returns the number of – We could have T(n) return the actual amount of “basic operations” performed. time (in µ sec., for example) when run. • Basic operation • May vary based on platform (OS / machine) – Will vary based on algorithm considered • May vary based on implementation language – Independent of machine/language/compiler • May vary based on compiler used – Examples » Number of elements compared in a search • May vary depending on what else the system is » Number of database records accessed doing. » Number of messages sent through the network

Finding T(n) Asymptotic Analysis • Can be difficult to determine an exact • Based on the idea that as the input to an number algorithm gets large – The size of the data may not be enough for an – The complexity will become proportional to a exact determination. known function. – We’re really only interested in how T(n) grows – Notations: as n, the size of the input, gets larger and larger. • O(f(n)) (Big-O) – upper bounds on time • Ω (f(n)) (Big-Omega) – lower bounds on time • Θ (f(n)) (Big-Theta) – tight bounds on time Asymptotic Analysis Asymptotic Analysis • Big-O (order of) • Big-Theta – T(n) = Θ (f(n)) if and only if there exists – T(n) = O(f(n)) if and only if there are constants c 0 and n 0 such that constants c 1 , c 2 , and n 0 such that • T(n) ≤ c 0 f(n) for all n ≥ n 0 • c 1 f(n) ≤ T(n) ≤ c 2 f(n) for all n ≥ n 0 – What this really means – What this really means • Eventually, when the size of the input gets large • Eventually, when the size of the input gets large enough, the upper bounds on the runtime will be no enough, both the upper and lower bounds of the worse than proportional to the function f(n). algorithm will be proportional to f(n). Asymptotic Analysis Asymptotic Analysis • Why these constants don’t concern us • Commonly used functions (Best to worst) – O(1) – Accounts for the fact that different machines • constant growth. are faster than other • T(n) does not grow at all as the size of your input grows • Simple instructions execute in constant time. • Example: indexing an array (in memory) • This constant can vary from machine to machine – O(log n) – Really only interested on how the runtime • logarithmic growth. grows with input size • T(n) increases as input size does, but very slowly • Proportional to the log – Doesn’t double until n reaches n 2 • Example: Binary searches (like “Twenty Questions”)

Asymptotic Analysis Asymptotic Analysis • Commonly used functions (Best to worst) • Commonly used functions (Best to worst) – O(n) – O(n 2 ) • Linear growth • Quadratic growth • Runtime increases at same rate as input size • When input size doubles, runtime quadruples • When input doubles, runtime will also double • Example: scanning an array for duplicates; • Example: looping over a 1 dimensional array looping over a 2D array? – O(n log(n)) – O(n 3 ) • n log n growth • Cubic growth • When input size doubles, runtime increases slightly more than • When input size doubles, runtime increases eightfold twice. • Algorithms that split problems into smaller problems Asymptotic Analysis Algorithm Efficiencies • Commonly used functions (Best to worst) – O(n k ) • Polynomial growth • As input doubles, runtime will increase by 2 k • Quadratic and Cubic are examples of polynomial growth • Polynomial growth with k > 3 may become impractical – O (2 n ) • Exponential growth • When input size double, runtime will increase by the square • Indicative of a really “hard” problem. • Algorithm is intractable except for very small input Algorithm Efficiency Asymptotic Analysis • Examples: – 100n = O(n) – 100n = Θ (n) – n log n = O(n 2 ) – n log n ≠ Θ (n 2 ) – 0.000000000001n log n ≠ O (n)

Asymptotic Analysis Asymptotic Analysis • More examples – n = O(n) – n = Θ (n) – 1000000000n = O(n) – 1000000000n = Θ (n) – 0.0000000001n = O(n) – 0.0000000001n = Θ (n) – n = O(n 2 ) – n ≠ Θ (n 2 ) Adding Functions Adding Functions • Let’s say that your algorithm does 3 tasks, one • By the same token after the other – Polynomial functions need only consider the – For task 1, T(n) = 200 highest polynomial term – For task 2, T(n) = 20n • Example – For task 3, T(n) = 0.2n 2 – n 2 + 3544442134n + 20 = O(n 2 ) • When determining the asymptotic runtime for the – Eventually the n 2 term will dominate the other terms complete algorithm, only the “greatest” • Another example complexity term need be considered. – n + 200000log(n) = O(n) – In the long run, the runtime will be dominated by task – Eventually the n term will dominate the log n term 3. Adding functions Asymptotic Analysis and Loops • Loop within a loop – Running times are multiplicative for (int i=0; i < n; i++) Done n times for (int j = 0; j < n; j++) // do something Something is done n times in the inner loop The inner loop is done n times Something is done n 2 times

Asymptotic Analysis and Loops Best, Worst, Average Case • Loop after a loop • Usually, an algorithm’s performance will – Additive depend on the data on which it operates. – Example: for (int i=0; i < n; i++) • A Linear Search for an integer in an array, x // do something … for (int j = 0; j < n; j++) // do something else Works pretty quickly if the item searched for is here Something is done 2n times = O(n) Works not as well if the item is not in the array Best, Worst, Average Case Best, Worst, Average Case • Can categorize runtimes based on the property of • Can categorize runtimes based on the the data the algorithm is run on: property of the data the algorithm is run on: – Best case • The best that it can do – Average case • Linear search: when int searched for is at x[0] • The average performance over all possible datasets • Best case performance: 1 operation – Worst case • Usually what you are looking for • The worst that it can do • Unfortunately, difficult to calculate unless you know • Linear search: when the int searched for is not in the array the distribution of your data. • Worst case performance: n operations Best, Worst, Average Case Asymptotic Analysis • Algorithms tend to be tweaked to get good • Any questions? best case performance – This doesn’t really help us if our data is not always arranged to get us the best case – Average case is usually what we want, but difficult to calculate – Best to look at worst-case since the algorithm can only do better.

Summary Next time • Analysis of Algorithms • Searching and sorting • Complexity – What algorithms are out there – How do they compare – Space / Time • Asymptotic Analysis – Big-O / Big Theta • Best case, worst case, average case