Evaluating Time Evaluating Time Running Time Analysis Examples - PDF document

4/3/2009 10:14 AM OUTLINE OUTLINE What is your code? What is your code? How can we evaluate it? How can we evaluate it? Why are we interested in the analysis of Why are we interested in the analysis of time complexity? time complexity? Evaluating Time Evaluating Time Running Time Analysis Examples Running Time Analysis Examples Complexity of Your Complexity of Your Best- Best -case, Worst case, Worst- -case & Average case & Average- -case case Complexity Complexity Code Code Multiple Multiple- -Variable Input Size Variable Input Size Asymptotic Behavior of Functions Asymptotic Behavior of Functions Dr. Dr. R. A. Angryk Dr. Dr. R. A. Angryk R. A. Angryk R. A. Angryk More examples More examples Questions Questions What is “the code”? What is “the code”? How can w e analyze time How can w e analyze time complexity of the code? complexity of the code? To analyze the code we build a To analyze the code we build a 1. Computational procedure (a 1. Computational procedure (a sequence sequence computational model, which makes computational model , which makes of steps) of steps) certain simplifying assumptions, e.g.: certain simplifying assumptions, e.g.: 2. Takes some value(s) (called 2. Takes some value(s) (called input input) ) Any number (integer, real), character, ... Any number (integer, real), character, ... Any number (integer real) character Any number (integer real) character 3 3. 3. Produces some value(s) (called P Produces some value(s) (called output P d d l l ( ) ( ( ) ( ll d ll d output) t t t) t ) ) uses uses one “unit” of space one “unit” of space All primitive operations take constant All primitive operations take constant time time … and that we have a … and that we have a Random Access Random Access Machine Machine Computational models Computational models What are w e interested in? What are w e interested in? Random Access Machine Random Access Machine Consumption of the computational resources Consumption of the computational resources One processor, sequential execution One processor, sequential execution CPU time (running time) CPU time (running time) Basic data types Basic data types memory usage (space) memory usage (space) Basic operations Basic operations messages sent along the network (bandwidth) messages sent along the network (bandwidth) Arithmetic operations: +, – Arithmetic operations: +, –, *, /, !, , , *, /, !, , Usually we focus on the running time! Usually we focus on the running time! Logical operations: AND, OR, NOT Logical operations: AND, OR, NOT Resource consumption differs depending on the Resource consumption differs depending on the Array indexing: A[x], where x is a variable Array indexing: A[x], where x is a variable size of the input � Specify resource size of the input Specify resource if- if -then then- -else else consumption consumption (e.g. running time) (e.g. running time) as a function of as a function of while- while -loops, for loops, for- -loops loops the inputs size the inputs size procedure calls procedure calls Recursive calls use infinite size of stack Recursive calls use infinite size of stack Resource consumption may differ greatly for Resource consumption may differ greatly for inputs of the same size, depending on their inputs of the same size, depending on their Etc. Etc. structure (highly unsorted, almost sorted, …) structure (highly unsorted, almost sorted, …) Parallel multi Parallel multi- -processor access model processor access model ( PRAM) maybe in Grad School… ☺ PRAM) – – maybe in Grad School… 1

4/3/2009 10:14 AM Is it w orth it? Is it w orth it? I w ant to show you I w ant to show you that… that… Computers are getting faster… Computers are getting faster… Memory gets cheaper… Memory gets cheaper… Programs for solving the same Programs for solving the same Is it worth it? Is it worth it? problem can differ dramatically in their problem can differ dramatically in their Computers may be very fast, but not Computers may be very fast, but not efficiency. efficiency. infinitely! infinitely! This difference might be much more This difference might be much more This difference might be much more This difference might be much more Memory may be very cheap, Memory may be very cheap, but not free! but not free! significant than the differences significant than the differences There are always competitors on the There are always competitors on the generated by new hardware or generated by new hardware or market! market! software. software. Advanced technologies involve even more Advanced technologies involve even more complex algorithms complex algorithms This is what truly separates skilled This is what truly separates skilled programmers from novices programmers from novices Comparison of tw o sorting Comparison of tw o sorting INSERTION SORT INSERTION SORT algorithms (1) algorithms (1) Sort an array of ONE MILLION NUMBERS using: Sort an array of ONE MILLION NUMBERS using: 1. Insertion sort 1. Insertion sort 2. 2. Merge sort Merge sort Ad. Ad. 1. 1. Best programmer & machine language Best programmer & machine language & one billion instructions per second computer & one billion instructions per second computer � 1GHz=10 1GHz=10 9 Hz Hz Ad. Ad. 2. 2. Bad programmer & high Bad programmer & high- -level language level language & ten million instructions per second computer & ten million instructions per second computer � 10MHz=10 10MHz=10 7 Hz Hz similar to sorting cards similar to sorting cards THE FIRST MACHINE IS 100 TIMES FASTER (10 9 /10 THE FIRST MACHINE IS 100 TIMES FASTER (10 /10 7 ) Comparison of tw o sorting Comparison of tw o sorting Example of MERGE Example of MERGE- -SORT SORT algorithms (2) algorithms (2) INSERTION SORT INSERTION SORT MERGE SORT MERGE SORT c ⋅ n 2 c ⋅ n ⋅ lg Running time formula Running time formula 2 lg 2 2 n TASK: Sort an array of 1 TASK: Sort an array of 1 10 10 6 6 10 6 10 6 mln numbers (n mln numbers ( n) ) Constant factor ( Constant factor (c c) ) great programmer great programmer bad programmer ad programmer (e g # of CPU ticks per 1 (e g # of CPU ticks per 1 (e.g. # of CPU ticks per 1 (e.g. # of CPU ticks per 1 machine language machine language machine language machine language high lev high high high-lev. lev language lev. language language language operation of comparison) operation of comparison) 2 50 50 Machine Machine 10 9 10 9 /1s =1GHz /1s =1GHz 10 10 7 7 /1s =10MHz /1s =10MHz (# ticks per second) (# ticks per second) 2 ⋅ (10 10 6 6 ) ) 2 50 50 ⋅ 10 10 6 6 ⋅ lg lg 2 2 10 10 6 Total # of ticks to sort it Total # of ticks to sort it 2 6 2 ⋅ (10 10 6 6 ) ) 2 2 /10 10 9 9 50 ⋅ 10 50 10 6 6 ⋅ lg lg 2 2 10 10 6 6 /10 10 7 7 Divided by the number of Divided by the number of ticks in one second ticks in one second Gives the RUNNING TIME: = 2 000 s ≈ 100 s Gives the RUNNING TIME: = 2 000 s 100 s To sort 10 milion of numbers 2.3 days To sort 10 milion of numbers 2.3 days 20 minutes 20 minutes 2