COMP 3403 Algorithm Analysis Part 1 Chapters 1 3 Jim Diamond - PowerPoint PPT Presentation

COMP 3403 — Algorithm Analysis Part 1 — Chapters 1 – 3 Jim Diamond CAR 409 Jodrey School of Computer Science Acadia University

Chapter 0 Preliminaries Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 0 1 Course Objectives We shall cover • review of asymptotic notation • the goal of algorithm analysis • recurrence equations • order statistics, sorting and selection • general algorithm design techniques • graph theoretic algorithms • string matching • introduction to NP completeness • other topics may be covered, time permitting Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 0 2 General Information • Read course outline at http://cs.acadiau.ca/~jdiamond/comp3403/comp3403.pdf • Getting help – ask questions in class! – my office hours (CAR 409): MWF 8:30–930, other times TBA and/or by appointment – these are subject to change; I’ll let you know if they do – if you need to see me at other times, e-mail or phone me and we can set up a meeting time – the teaching assistant contact hours shall be announced RSN To pretend to know when you do not know is a disease. To know that you do not know is the best. — Lao Tzu Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 0 3 Reference Materials Introduction to the Design & Analysis of Algorithms (3 rd edition) by • Anany Levitin (Addison Wesley) Introduction to Computer Algorithms (3 rd edition) by Cormen, • Leiserson, Rivest and Stein • The Design and Analysis of Computer Algorithms by Alfred V. Aho, John E. Hopcroft and Jeffrey D. Ullman • Fundamentals of Algorithmics by Gilles Brassard and Paul Bratley • Computers and Intractability: A Guide to the Theory of NP-Completeness by Michael R. Garey and David S. Johnson Computer Algorithms: Introduction to Design & Analysis (3 rd edition) • by Sara Baase and Allen Van Gelder • There are lots of reference books in the library – reviewing other explanations of the same material may aid your understanding Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 0 4 Course Communications • I will not use ACORN for assignments (or anything else, in all likelihood) – check (and read) your e-mail frequently – look in http://cs.acadiau.ca/~jdiamond/comp3403/assignments/ Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 Introduction Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 5 Algorithm Analysis Preliminaries: 1 • What is the difference between a computer science degree and a programming course at a technical school? • Sample problem: running a dating service: match each of N guys to (one each!) of N girls – you have a compatibility function C ( m i , f j ) , 1 ≤ i, j ≤ N – business plan: find a 1 – 1, onto mapping P : { m i } → { f j } which – maximizes total happiness H : N � H = C ( i, P ( i )) i =1 <board example> – Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 6 Algorithm Analysis Preliminaries: 2 • Q: how can we compute P ( ) ? • A1: try all possible pairings (a BFI ( q.v. ) technique) • Q’: how many pairings for N pairs? • A’: N ! !!! • Q”: how big is that? A”: n ! ≈ n n e − n √ • 2 πn (Stirling’s formula) – more specifically n n e − n √ 2 πn e 1 / (12 n +1) < n ! < n n e − n √ 2 πn e 1 / 12 n • <gnuplot examples> Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 7 Problem Categorization • Roughly speaking, there are three types of problems: – problems that we can solve “efficiently” – problems that are not solvable at all – problems that we can solve, but not “efficiently” – for some, no efficient solution can possibly exist (provably!) – for others, an efficient solution might exist, but no one knows such a solution or, they’re keeping it a secret • Dating example: Where does it fit? Is n ! considered to be efficient? – • Does an inefficient algorithm for a problem mean the problem is “hard”? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 8 “Problem” vs. “Instance” • Problem: a general description of something to be solved e.g., “Given n numbers, find the smallest.” – • Instance: a specific case of a problem e.g., “Find the smallest number in { 2 , 3 , 5 , 8 , 13 } ” – • Note: problems which are unsolvable or not efficiently solvable in general may have easily solvable instances – e.g., the halting problem – e.g., find a Hamiltonian cycle (HC) in a connected graph – is there a HC here? – how about here? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 9 Algorithm Analysis • We can analyze algorithms for – space complexity: – in a conventional computer, how much memory is used by a given algorithm – time complexity: – how much CPU time is used by a given algorithm • Issue 1: for a given algorithm, solving a “bigger” instance usually takes longer than solving a “smaller” instance – how should we measure the size of an instance? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 10 Algorithm Analysis: 2 • Issue 2: implementing an algorithm & timing it on various instances isn’t necessarily a good way to analyze algorithms – (why not? infer function from plot? bad implementation? cache effects? timing inaccuracy?) – we could count the operations done to solve an instance – all operations?!? Usually not necessary. – often, the “total” work is proportional to the amount of work done by some selected operations – examples: – sorting: (people usually) just count the number of comparisons of the data items – matrix operations: just count the number of (scalar) multiplications & maybe the additions (Q (GEQ?): what is the relationship between the additions and the multiplications in a matrix multiplication operation?) Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 11 Algorithm Analysis: 3 • Issue 3: Do all operations take the same (or similar) amount of time? – Consider operations on integers: – if all operands are “small”, we can assume all operations take the same amount of time – if operands can become arbitrarily large, then we can’t assume that all operations take the same amount of time • We consider two models of arithmetic operations: Uniform model: all operations take the same amount of time – this model is good for most purposes Logarithmic model: operation times vary with the size of the operands – this model is appropriate when very large numbers are used – got any examples (of algorithms which use very large numbers)? – why is the name “logarithmic”? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 1 12 Algorithm Analysis: 4 • Issue 4: Should we consider the best case, worst case or average case analysis? – best case analysis: (usually) not of much interest – worst case analysis: usually easier to do than average case analysis – average case may be of most interest – e.g., sorting with quicksort – e.g., linear programming using the simplex method • Issue 5: Are we interested in algorithm complexity or problem complexity? – Algorithm complexity: what is the complexity of (a specific implementation of?!) a given algorithm? – worst and/or average case – Problem complexity: what is the complexity of the best algorithm that solves the problem? Best average case or best worst case. – solving this requires you to find lower bounds – usually difficult to do! How can you know you have the best? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 2 Fundamentals of the Analysis of Algorithm Efficiency Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 2 13 Goal of Algorithm Analysis • Suppose we can measure the size of a problem instance; call it N • Goal: given an algorithm A , find functions T ( n ) & S ( n ) such that the amount of time and space needed by A to solve an instance of size N are T ( N ) and S ( N ) , respectively • Note: if the time or space used varies for different instances of size N , we may want to consider the best, average and/or worst case versions of T ( n ) and S ( n ) • Note: in algorithm analysis T ( n ) and S ( n ) are ≥ 0 ∀ n ( n ≥ 0) Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 2 14 Algorithm Analysis Details • Normally we are interested in the “big picture” • E.g., suppose we have analyzed an algorithm and discovered that on an input of size n , T ( n ) = k 2 n 2 + k 1 n + k 0 • In rare cases, we might be concerned with the k 1 n and k 0 terms, but normally we are not – • The specific values of k i might vary from one computer architecture to another — this lessens their relevance • In most cases we are not even concerned with k 2 : T ( n ) will be a quadratic function on any computer • <Sample plots> Jim Diamond, Jodrey School of Computer Science, Acadia University