Asymptotic Analysis & Stable Matchings
Admin • Rubric/latex tips document posted • Will post a latex intro and some extra resources as soon as possible (hopefully tonight) • Intro form already due! • Slides, recordings will be posted this afternoon • TA hours sometime after 5PM • Full zoom setup is together! Can ask questions in chat, by raising physical hand, or “zoom hand” • Anything else?
Measuring Complexity What constitutes an efficient algorithm? • Runs quickly on large, ‘real’ instances of problems • Qualitatively better than brute force • Scales well to large instances •
Brute Force: Often Inefficient Efficient: Qualitatively better than brute force • Brute force: often exponentially large because • 2 n Might examine all subsets of a set : • n ! Might examine all orderings of a list: • 2 n But is still not efficient even • n ! though it’s qualitatively better than 2 n Example of a algorithm is “two towers” • from CS 136
Measuring Complexity : Scalability Desirable scalability property. When the input size doubles the • C algorithm should slow down by at most some constant factor Examples • f (2 n ) = 2 k n k = cn k f ( n ) = n k k , then for any fixed • g ( n ) = log n g (2 n ) = log 2 + log n ≤ c log n n ≥ 2 , then for • But not for these functions • f (2 n ) = 2 2 n = 2 n ⋅ 2 n f ( n ) = 2 n , then • g (2 n ) = (2 n !) ≥ n n ⋅ n ! g ( n ) = n ! , then • An algorithm is polynomial time if the above scaling property holds, • i.e., their running time is bounded above by a polynomial function
Growth of Functions
Worst Case Analysis But how do we measure running time? • Worst-case running time: the maximum number of steps • n needed to solve a problem instance of size Overestimates the typical runtime but gives strong guaranties • “I promise you that my algorithm is ALWAYS this fast!” • Often there’s no easy to identify “worst” case • Don’t fall into the “the worst case is when…” trap! •
Other Types Of Analysis Probabilistic. Expected running time of a randomized algorithm • e.g., the expected running time of quicksort • n Amortized. Worst-case running time for any sequence of • operations Some operations can be expensive but may make future • operations fast (doing well on average) e.g., Union-find data structure (we’ll study in a few weeks) • Average-case analysis, smoothed analysis, competitive analysis, etc. •
How to Measure Cost? • “Word RAM” model of computation • Basic idea: every operation on a primitive type in C, Java, etc. costs 1 unit of time: • Adding/multiplying/dividing/etc two ints or floats costs 1 • An if statement costs 1 • A comparison costs 1 • Dereferencing a pointer costs 1 • Array access costs 1
Model of Computation Details Word RAM model • w Each memory location and input/output cell stores a -bit • w ≥ log 2 n integer (assume ) Primitive operations: arithmetic operations, read/write memory, • array indexing, following a pointer etc. are constant time Running time: number of primitive operations • Space: number of memory cells utilized • memory input … Space is measured in “words” (ints, floats, program chars, etc) not bits . . . output …
Asymptotic Growth What matters: How functions behave “as n gets large” 15000 10 x 3 + 10 x 2 + x + 500 10000 100 x 2 - 100 x + 500 5000 2 4 6 8 10 12
Asymptotic Upper Bounds f ( n ) O ( g ( n )) c > 0 n 0 ≥ 0 Definition : is if there exists constants and 0 ≤ f ( n ) ≤ c ⋅ g ( n ) n ≥ n 0 such that for all n f ( n ) In other words, for sufficiently large , is asymptotically bounded g ( n ) above by Examples c · g ( n ) 100 n 2 = O ( n 2 ) • n log n = O ( n 2 ) f ( n ) • 5 n 3 + 2 n + 1 = O ( n 3 ) • Typical usage. Insertion sort makes O ( n 2 ) n compares to sort elements n 0 n
Class Quiz f ( n ) = 3 n 2 + 17 n log 2 n + 1000 Let . Which of the following are true? O ( n 2 ) f ( n ) A. is . choose c = 1020, n 0 = 1 O ( n 3 ) f ( n ) B. is . choose c = 1020, n 0 = 1 C. Both A and B. D. Neither A nor B.
Big Oh- Notational Abuses O ( g ( n )) is actually a set of functions, but the CS community writes • f ( n ) = O ( g ( n )) f ( n ) ∈ O ( g ( n )) instead of For example • f 1 ( n ) = O ( n log n ) = O ( n 2 ) • f 2 ( n ) = O (3 n 2 + n ) = O ( n 2 ) • c · g ( n ) f 1 ( n ) ≠ f 2 ( n ) But • f ( n ) Okay to abuse notation in this way • n 0 n
Playing with Logs: Properties log n log 2 n ln n = log e n In this class, means , • log b ( n ) = log n log b = O (log n ) Constant base doesn’t matter: • log( n m ) = m log n • log( ab ) = log a + log b • log( a / b ) = log a − log b • Exponents n a ⋅ n b = n a + b a log a n = n ( n a ) b = n ab We will use this a lot!
Comparing Running Times When comparing two functions, helpful to simplify first • n 1/log n O (1) Is = ? • n 1/log n = (2 log n ) 1/log n = 2 Simplify : True • 4 n O ( n 2 ) log Is = • 2 2 n = log 2 n = n log 2 = O ( n ) log Simplify : True • n = O (2 log 4 n )? Is • log2 n 2 log 4 n = 2 log 24 = 2 (log 2 n )/2 = 2 log 2 n = n Simplify : False •
Something Missing ≤ • Big-O notation is like • So one can accurately say “merge sort requires O (2 n ) time,” but it’s not very meaningful • Can we get terminology like big-O that lower bounds? Or that shows two functions are “equal” (up to constants and for large values of n)?
Asymptotic Lower Bounds f ( n ) Ω ( g ( n )) c > 0 n 0 ≥ 0 Definition : is if there exists constants and f ( n ) ≥ c ⋅ g ( n ) ≥ 0 n ≥ n 0 such that for all n f ( n ) In other words, for sufficiently large , is asymptotically bounded g ( n ) below by . (Same abuse of notation as big Oh) f ( n ) Examples 100 n 2 = Ω ( n 2 ) Ω ( n ) = • c · g ( n ) n log n = Ω ( n ) • 8 log n = Ω ( n 2 ) • n 0 n
Why Lower Bounds? Show that an algorithm performs at least so many steps Ω ( n ) n Searching an unordered list of items: steps in some cases • Ω ( n 2 ) Quicksort (and selection/insertion/bubble sorts) take steps • in some cases Ω ( n log n ) Mergesort takes steps in all cases • f ( n ) c · g ( n ) n 0 n
Class Quiz True or False: f ( n ) Ω ( g ( n )) g ( n ) O ( f ( n )) is if and only if is True! f ( n ) Ω ( g ( n )) c 1 > 0 n 0 ≥ 0 is if there exists constants and such that f ( n ) ≥ c 1 ⋅ g ( n ) ≥ 0 n ≥ n 0 for all g ( n ) O ( f ( n )) c 2 > 0 n 0 ≥ 0 is if there exists constants and such that 0 ≤ g ( n ) ≤ c 2 ⋅ f ( n ) n ≥ n 0 for all Set c 1 = 1/ c 2
Asymptotically Tight Bounds f ( n ) = Θ ( g ( n )) f ( n ) = O ( g ( n )) f ( n ) = Ω ( g ( n )) Definition. if and f ( n ) = O ( g ( n )) g ( n ) = O ( f ( n )) (From before, also enough if and ) c 1 > 0, c 2 > 0, n 0 ≥ 0 Equivalently, if there exist constants and such 0 ≤ c 1 ⋅ g ( n ) ≤ f ( n ) ≤ c 2 ⋅ g ( n ) n ≥ n 0 . that for all c 2 · g ( n ) f ( n ) c 1 · g ( n ) Examples 5 n 3 + 2 n + 1 = Θ ( n 3 ) • log 100 n = Θ (log 2 n ) • n 0 n
Tools for Comparing Asymptotics Logs grow slowly than any polynomial: • log a n = O ( n b ) a > 1, b > 0 for every • Exponentials grow faster than any polynomial: • n d = O ( r n ) d > 1, r > 0 for every • Taking logs • log x x > 0 • As is a strictly increasing function for , log( f ( n )) < log( g ( n )) f ( n ) < g ( n ) implies 3 log n 2 n • E.g. Compare vs log n log 3 n • Taking log of both, vs
Tools for Comparing Asymptotics • Using limits f ( x ) If lim g ( x ) = 0 , then f ( x ) = O ( g ( x )) • n →∞ f ( x ) If lim g ( x ) = c 0 < c < ∞ for some constant , then • n →∞ f ( x ) ∈ Θ ( g ( x ))
Stable Matchings
An Illustrative Example: The Stable Matching Problem Applications • Assigning first year students to advisors • Pairing job candidates with employers • Matching doctors to hospitals Fundamental Problem • Given preferences of both sides, find a matching that is resilient against opportunistic swapping
State the Problem • Two groups: hospitals and students • Students have preferences over hospitals • Hospitals have preferences over students • Each hospital has only one open slot Goal. Match hospitals to students that is “stable”, that is, no pair has an incentive to break their match! National Resident Matching Program
Matching Med-Students to Hospitals H n S n Input. A set of hospitals, a set of students and their preferences (each hospital ranks each student, each students ranks each hospital) 1st 2nd 3rd 1st 2nd 3rd MA Aamir Beth Chris Aamir NH MA OH NH Beth Aamir Chris Beth MA NH OH OH Aamir Beth Chris Chris MA NH OH
Perfect Matchings ( h , s ) M Definition. A matching is a set of ordered pairs where h ∈ H s ∈ S and such that h M • Each hospital is in at most one pair in s M • Each student is in at most one pair in M A matching is perfect if each hospital is matched to exactly one | M | = | H | = | S | student and vice versa (i.e., ) 1st 2nd 3rd 1st 2nd 3rd MA Aamir Beth Chris Aamir NH MA OH NH Beth Aamir Chris Beth MA NH OH OH Aamir Beth Chris Chris MA NH OH
Recommend
More recommend
