Introduction to Randomized Algorithms: QuickSort Lecture 2 January - PowerPoint PPT Presentation

Union bound The probability of the union of two events, is no bigger than the probability of the sum of their probabilities. Lemma For any two events E and F , we have that � � � � � � Pr E ∪ F ≤ Pr E + Pr F . Proof. Consider E and F to be a collection of elmentery events (which they are). We have � � � Pr E ∪ F = Pr[ x ] x ∈ E ∪ F � � � � � � ≤ Pr[ x ] + Pr[ x ] = Pr E + Pr F . x ∈ E x ∈ F Chandra (UIUC) CS498ABD 15 Spring 2019 15 / 51

Random Variables Definition Given a probability space (Ω , Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω → R . Chandra (UIUC) CS498ABD 16 Spring 2019 16 / 51

Random Variables Definition Given a probability space (Ω , Pr) a (real-valued) random variable X over Ω is a function that maps each elementary event to a real number. In other words X : Ω → R . Definition Expectation For a random variable X over a probability space (Ω , Pr) the expectation of X is defined as � ω ∈ Ω Pr[ ω ] X ( ω ) . In other words, the expectation is the average value of X according to the probabilities given by Pr[ · ] . Chandra (UIUC) CS498ABD 16 Spring 2019 16 / 51

Expectation Example A 6-sided unbiased die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } and Pr[ i ] = 1 / 6 for 1 ≤ i ≤ 6 . X : Ω → R where X ( i ) = i mod 2 . Then 1 E[ X ] = � 6 � 6 i =1 Pr[ i ] · X ( i ) = 1 i =1 X ( i ) = 1 / 2 . 6 Chandra (UIUC) CS498ABD 17 Spring 2019 17 / 51

Expectation Example A 6-sided unbiased die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } and Pr[ i ] = 1 / 6 for 1 ≤ i ≤ 6 . X : Ω → R where X ( i ) = i mod 2 . Then 1 E[ X ] = � 6 � 6 i =1 Pr[ i ] · X ( i ) = 1 i =1 X ( i ) = 1 / 2 . 6 Y : Ω → R where Y ( i ) = i 2 . Then 2 6 · i 2 = 91 / 6 . E[ Y ] = � 6 1 i =1 Chandra (UIUC) CS498ABD 17 Spring 2019 17 / 51

Expected number of vertices? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . Compute the expected number of vertices in H. (A) n/2. (B) n/4. (C) m/2. (D) m/4. (E) none of the above. Chandra (UIUC) CS498ABD 18 Spring 2019 18 / 51

Expected number of vertices is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . Chandra (UIUC) CS498ABD 19 Spring 2019 19 / 51

Expected number of vertices is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # vertices in H as per ω = # 1s in ω . Chandra (UIUC) CS498ABD 19 Spring 2019 19 / 51

Expected number of vertices is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # vertices in H as per ω = # 1s in ω . � E[ X ] = ω ∈ Ω Pr[ ω ] X ( ω ) = � ω ∈ Ω 1 / 2 n X ( ω ) 1 / 2 n � n � n � = k k =0 k 1 / 2 n (2 n n = 2 ) = n / 2 Chandra (UIUC) CS498ABD 19 Spring 2019 19 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is (A) n/2. (B) n/4. (C) m/2. (D) m/4. (E) none of the above. Chandra (UIUC) CS498ABD 20 Spring 2019 20 / 51

Expected number of edges is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . Chandra (UIUC) CS498ABD 21 Spring 2019 21 / 51

Expected number of edges is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # edges present in H as per ω = ?? Chandra (UIUC) CS498ABD 21 Spring 2019 21 / 51

Expected number of edges is: Probability Space Ω = { 0 , 1 } n . For ω ∈ { 0 , 1 } n , ω v = 1 if vertex v is present in H, else is zero. 1 For each ω ∈ Ω , Pr[ ω ] = 2 n . X ( ω ) = # edges present in H as per ω = ?? How to compute E[ X ] ? Chandra (UIUC) CS498ABD 21 Spring 2019 21 / 51

Indicator Random Variables Definition A binary random variable is one that takes on values in { 0 , 1 } . Chandra (UIUC) CS498ABD 22 Spring 2019 22 / 51

Indicator Random Variables Definition A binary random variable is one that takes on values in { 0 , 1 } . Special type of random variables that are quite useful. Definition Given a probability space (Ω , Pr) and an event A ⊆ Ω the indicator random variable X A is a binary random variable where X A ( ω ) = 1 if ω ∈ A and X A ( ω ) = 0 if ω �∈ A . Chandra (UIUC) CS498ABD 22 Spring 2019 22 / 51

Indicator Random Variables Definition A binary random variable is one that takes on values in { 0 , 1 } . Special type of random variables that are quite useful. Definition Given a probability space (Ω , Pr) and an event A ⊆ Ω the indicator random variable X A is a binary random variable where X A ( ω ) = 1 if ω ∈ A and X A ( ω ) = 0 if ω �∈ A . Example A 6-sided unbiased die. Ω = { 1 , 2 , 3 , 4 , 5 , 6 } and Pr[ i ] = 1 / 6 for 1 ≤ i ≤ 6 . Let A be the even that i is divisible by 3 . Then X A ( i ) = 1 if i = 3 , 6 and 0 otherwise. Chandra (UIUC) CS498ABD 22 Spring 2019 22 / 51

Expectation of indicator random variable Proposition For an indicator variable X A , E[ X A ] = Pr[ A ] . Proof. � E[ X A ] = X A ( ω ) Pr[ ω ] ω ∈ Ω � � = 1 · Pr[ ω ] + 0 · Pr[ ω ] ω ∈ A ω ∈ Ω \ A � = Pr[ ω ] ω ∈ A = Pr[ A ] . Chandra (UIUC) CS498ABD 23 Spring 2019 23 / 51

Linearity of Expectation Lemma Let X , Y be two random variables (not necessarily independent) over a probability space (Ω , Pr) . Then E[ X + Y ] = E[ X ] + E[ Y ] . Proof. � E[ X + Y ] = Pr[ ω ] ( X ( ω ) + Y ( ω )) ω ∈ Ω � � = Pr[ ω ] X ( ω ) + Pr[ ω ] Y ( ω ) = E[ X ] + E[ Y ] . ω ∈ Ω ω ∈ Ω Chandra (UIUC) CS498ABD 24 Spring 2019 24 / 51

Linearity of Expectation Lemma Let X , Y be two random variables (not necessarily independent) over a probability space (Ω , Pr) . Then E[ X + Y ] = E[ X ] + E[ Y ] . Proof. � E[ X + Y ] = Pr[ ω ] ( X ( ω ) + Y ( ω )) ω ∈ Ω � � = Pr[ ω ] X ( ω ) + Pr[ ω ] Y ( ω ) = E[ X ] + E[ Y ] . ω ∈ Ω ω ∈ Ω Corollary E[ a 1 X 1 + a 2 X 2 + . . . + a n X n ] = � n i =1 a i E[ X i ] . Chandra (UIUC) CS498ABD 24 Spring 2019 24 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Let X = � e ∈ E X A e (Number of edges in H) Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Let X = � e ∈ E X A e (Number of edges in H) �� � Pr[ A e ] = m � � E[ X ] = E = E[ X A e ] = X A e 4 e ∈ E e ∈ E e ∈ E Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

Expected number of edges? Let G = ( V , E ) be a graph with n vertices and m edges. Let H be the graph resulting from independently deleting every vertex of G with probability 1 / 2 . The expected number of edges in H is Event A e = edge e ∈ E is present in H. � � Pr = Pr[ u and v both are present ] = A e =( u , v ) Pr[ u is present ] · Pr[ v is present ] = 1 2 · 1 2 = 1 4 . X A e indicator random variables, then E[ X A e ] = Pr[ A e ] . Let X = � e ∈ E X A e (Number of edges in H) �� � Pr[ A e ] = m � � E[ X ] = E = E[ X A e ] = X A e 4 e ∈ E e ∈ E e ∈ E It is important to setup random variables carefully. Chandra (UIUC) CS498ABD 25 Spring 2019 25 / 51

Expected number of triangles? Let G = ( V , E ) be a graph with n vertices and m edges. Assume G has t triangles (i.e., a triangle is a simple cycle with three vertices). Let H be the graph resulting from deleting independently each vertex of G with probability 1 / 2 . The expected number of triangles in H is (A) t/2. (B) t/4. (C) t/8. (D) t/16. (E) none of the above. Chandra (UIUC) CS498ABD 26 Spring 2019 26 / 51

Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Independent. Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Independent. X = # H , Y = # T . Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

Independent Random Variables Definition Random variables X , Y are said to be independent if ∀ x , y ∈ R , Pr[ X = x ∧ Y = y ] = Pr[ X = x ] Pr[ Y = y ] . Examples Two independent un-biased coin flips: Ω = { HH , HT , TH , TT } . X = 1 if first coin is H else 0 . Y = 1 if second coin is H else 0 . Independent. X = # H , Y = # T . Dependent. Why? Chandra (UIUC) CS498ABD 27 Spring 2019 27 / 51

Independent Randomized Variables Lemma If X and Y are independent then E[ XY ] = E[ X ] · E[ Y ] Proof. � E[ X · Y ] = Pr[ ω ] ( X ( ω ) · Y ( ω )) ω ∈ Ω � = Pr[ X = x ∧ Y = y ] ( x · y ) x , y ∈ R � = Pr[ X = x ] · Pr[ Y = y ] · x · y x , y ∈ R � � = ( Pr[ X = x ] x )( Pr[ Y = y ] y ) = E[ X ] E[ Y ] x ∈ R y ∈ R Chandra (UIUC) CS498ABD 28 Spring 2019 28 / 51

Types of Randomized Algorithms Typically one encounters the following types: Las Vegas randomized algorithms: for a given input x 1 output of algorithm is always correct but the running time is a random variable . In this case we are interested in analyzing the expected running time. Chandra (UIUC) CS498ABD 29 Spring 2019 29 / 51

Types of Randomized Algorithms Typically one encounters the following types: Las Vegas randomized algorithms: for a given input x 1 output of algorithm is always correct but the running time is a random variable . In this case we are interested in analyzing the expected running time. Monte Carlo randomized algorithms: for a given input x the 2 running time is deterministic but the output is random ; correct with some probability. In this case we are interested in analyzing the probability of the correct output (and also the running time). Algorithms whose running time and output may both be random. 3 Chandra (UIUC) CS498ABD 29 Spring 2019 29 / 51

Analyzing Las Vegas Algorithms Deterministic algorithm Q for a problem Π : Let Q ( x ) be the time for Q to run on input x of length | x | . 1 Worst-case analysis: run time on worst input for a given size n . 2 T wc ( n ) = max x : | x | = n Q ( x ) . Chandra (UIUC) CS498ABD 30 Spring 2019 30 / 51

Analyzing Las Vegas Algorithms Deterministic algorithm Q for a problem Π : Let Q ( x ) be the time for Q to run on input x of length | x | . 1 Worst-case analysis: run time on worst input for a given size n . 2 T wc ( n ) = max x : | x | = n Q ( x ) . Randomized algorithm R for a problem Π : Let R ( x ) be the time for Q to run on input x of length | x | . 1 R ( x ) is a random variable: depends on random bits used by R . 2 E[ R ( x )] is the expected running time for R on x 3 Worst-case analysis: expected time on worst input of size n 4 T rand − wc ( n ) = max x : | x | = n E[ R ( x )] . Chandra (UIUC) CS498ABD 30 Spring 2019 30 / 51

Analyzing Monte Carlo Algorithms Randomized algorithm M for a problem Π : Let M ( x ) be the time for M to run on input x of length | x | . 1 For Monte Carlo, assumption is that run time is deterministic. Let Pr[ x ] be the probability that M is correct on x . 2 Pr[ x ] is a random variable: depends on random bits used by M . 3 Worst-case analysis: success probability on worst input 4 P rand − wc ( n ) = min x : | x | = n Pr[ x ] . Chandra (UIUC) CS498ABD 31 Spring 2019 31 / 51

Part III Randomized Quick Sort Chandra (UIUC) CS498ABD 32 Spring 2019 32 / 51

Randomized QuickSort Randomized QuickSort Pick a pivot element uniformly at random from the array. 1 Split array into 3 subarrays: those smaller than pivot, those 2 larger than pivot, and the pivot itself. Recursively sort the subarrays, and concatenate them. 3 Chandra (UIUC) CS498ABD 33 Spring 2019 33 / 51

Analysis What events to count? Number of Comparisions. Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

Analysis What events to count? Number of Comparisions. What is the probability space? All the coin tosses at all levels and parts of recursion. Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

Analysis What events to count? Number of Comparisions. What is the probability space? All the coin tosses at all levels and parts of recursion. Too Big!! Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

Analysis What events to count? Number of Comparisions. What is the probability space? All the coin tosses at all levels and parts of recursion. Too Big!! What random variables to define? What are the events of the algorithm? Chandra (UIUC) CS498ABD 34 Spring 2019 34 / 51

Analysis via Recurrence Given array A of size n , let Q ( A ) be number of comparisons of 1 randomized QuickSort on A . Note that Q ( A ) is a random variable. 2 Let A i left and A i right be the left and right arrays obtained if: 3 Let X i be indicator random variable, which is set to 1 if pivot is of rank i in A , else zero. n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i · right ) . i =1 Chandra (UIUC) CS498ABD 35 Spring 2019 35 / 51

Analysis via Recurrence Given array A of size n , let Q ( A ) be number of comparisons of 1 randomized QuickSort on A . Note that Q ( A ) is a random variable. 2 Let A i left and A i right be the left and right arrays obtained if: 3 Let X i be indicator random variable, which is set to 1 if pivot is of rank i in A , else zero. n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i · right ) . i =1 Since each element of A has probability exactly of 1 / n of being chosen: E[ X i ] = Pr[ pivot has rank i ] = 1 / n . Chandra (UIUC) CS498ABD 35 Spring 2019 35 / 51

Independence of Random Variables Lemma Random variables X i is independent of random variables Q ( A i left ) as well as Q ( A i right ) , i.e. � X i · Q ( A i � � Q ( A i � left ) = E[ X i ] E left ) E � � � � X i · Q ( A i Q ( A i right ) = E[ X i ] E right ) E Proof. This is because the algorithm, while recursing on Q ( A i left ) and Q ( A i right ) uses new random coin tosses that are independent of the coin tosses used to decide the first pivot. Only the latter decides value of X i . Chandra (UIUC) CS498ABD 36 Spring 2019 36 / 51

Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We have, for any A : n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i right ) i =1 Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We have, for any A : n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i right ) i =1 By linearity of expectation, and independence random variables: n � � � � � � �� � Q ( A i Q ( A i Q ( A ) = n + E[ X i ] left ) + E right ) . E E i =1 Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We have, for any A : n � � � Q ( A i left ) + Q ( A i Q ( A ) = n + X i right ) i =1 By linearity of expectation, and independence random variables: n � � � � � � �� � Q ( A i Q ( A i Q ( A ) = n + E[ X i ] left ) + E right ) . E E i =1 n 1 � � � ⇒ Q ( A ) ≤ n + n ( T ( i − 1) + T ( n − i )) . E i =1 Chandra (UIUC) CS498ABD 37 Spring 2019 37 / 51

Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . Chandra (UIUC) CS498ABD 38 Spring 2019 38 / 51

Analysis via Recurrence Let T ( n ) = max A : | A | = n E[ Q ( A )] be the worst-case expected running time of randomized QuickSort on arrays of size n . We derived: n 1 � � � Q ( A ) ≤ n + n ( T ( i − 1) + T ( n − i )) . E i =1 Note that above holds for any A of size n . Therefore n 1 � A : | A | = n E[ Q ( A )] = T ( n ) ≤ n + max n ( T ( i − 1) + T ( n − i )) . i =1 Chandra (UIUC) CS498ABD 38 Spring 2019 38 / 51

Solving the Recurrence n 1 � T ( n ) ≤ n + n ( T ( i − 1) + T ( n − i )) i =1 with base case T (1) = 0 . Chandra (UIUC) CS498ABD 39 Spring 2019 39 / 51

Solving the Recurrence n 1 � T ( n ) ≤ n + n ( T ( i − 1) + T ( n − i )) i =1 with base case T (1) = 0 . Lemma T ( n ) = O ( n log n ) . Chandra (UIUC) CS498ABD 39 Spring 2019 39 / 51

Solving the Recurrence n 1 � T ( n ) ≤ n + n ( T ( i − 1) + T ( n − i )) i =1 with base case T (1) = 0 . Lemma T ( n ) = O ( n log n ) . Proof. (Guess and) Verify by induction. Chandra (UIUC) CS498ABD 39 Spring 2019 39 / 51

Part IV Slick analysis of QuickSort Chandra (UIUC) CS498ABD 40 Spring 2019 40 / 51

A Slick Analysis of QuickSort Let Q ( A ) be number of comparisons done on input array A : For 1 ≤ i < j < n let R ij be the event that rank i element is 1 compared with rank j element. X ij is the indicator random variable for R ij . That is, X ij = 1 if 2 rank i is compared with rank j element, otherwise 0 . Chandra (UIUC) CS498ABD 41 Spring 2019 41 / 51

A Slick Analysis of QuickSort Let Q ( A ) be number of comparisons done on input array A : For 1 ≤ i < j < n let R ij be the event that rank i element is 1 compared with rank j element. X ij is the indicator random variable for R ij . That is, X ij = 1 if 2 rank i is compared with rank j element, otherwise 0 . � Q ( A ) = X ij 1 ≤ i < j ≤ n and hence by linearity of expectation, � � � � � � � � Q ( A ) = X ij = Pr R ij . E E 1 ≤ i < j ≤ n 1 ≤ i < j ≤ n Chandra (UIUC) CS498ABD 41 Spring 2019 41 / 51

A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 As such, probability of comparing 5 to 8 is Pr[ R 4 , 7 ] . Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 If pivot too small (say 3 [rank 2]). Partition and call recursively: 1 7 5 9 1 3 4 8 6 1 3 7 5 9 4 8 6 = ⇒ Decision if to compare 5 to 8 is moved to subproblem. Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

A Slick Analysis of QuickSort R ij = rank i element is compared with rank j element. Question: What is Pr[ R ij ] ? 7 5 9 1 3 4 8 6 With ranks: 6 4 8 1 2 3 7 5 If pivot too small (say 3 [rank 2]). Partition and call recursively: 1 7 5 9 1 3 4 8 6 1 3 7 5 9 4 8 6 = ⇒ Decision if to compare 5 to 8 is moved to subproblem. If pivot too large (say 9 [rank 8]): 2 7 5 9 1 3 4 8 6 7 5 9 1 3 4 8 6 7 5 1 3 4 8 6 9 = ⇒ Decision if to compare 5 to 8 moved to subproblem. Chandra (UIUC) CS498ABD 42 Spring 2019 42 / 51

A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? As such, probability of com- 7 5 9 1 3 4 8 6 paring 5 to 8 is Pr[ R 4 , 7 ] . 6 4 8 1 2 3 7 5 If pivot is 5 (rank 4). Bingo! 1 7 5 9 1 3 4 8 6 1 3 4 5 7 9 8 6 = ⇒ Chandra (UIUC) CS498ABD 43 Spring 2019 43 / 51

A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? As such, probability of com- 7 5 9 1 3 4 8 6 paring 5 to 8 is Pr[ R 4 , 7 ] . 6 4 8 1 2 3 7 5 If pivot is 5 (rank 4). Bingo! 1 7 5 9 1 3 4 8 6 1 3 4 5 7 9 8 6 = ⇒ If pivot is 8 (rank 7). Bingo! 2 7 5 9 1 3 4 8 6 7 5 1 3 4 6 8 9 = ⇒ Chandra (UIUC) CS498ABD 43 Spring 2019 43 / 51

A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? As such, probability of com- 7 5 9 1 3 4 8 6 paring 5 to 8 is Pr[ R 4 , 7 ] . 6 4 8 1 2 3 7 5 If pivot is 5 (rank 4). Bingo! 1 7 5 9 1 3 4 8 6 1 3 4 5 7 9 8 6 = ⇒ If pivot is 8 (rank 7). Bingo! 2 7 5 9 1 3 4 8 6 7 5 1 3 4 6 8 9 = ⇒ If pivot in between the two numbers (say 6 [rank 5]): 3 7 5 9 1 3 4 8 6 5 1 3 4 6 7 8 9 = ⇒ 5 and 8 will never be compared to each other. Chandra (UIUC) CS498ABD 43 Spring 2019 43 / 51

A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? Conclusion: R i , j happens if and only if: i th or j th ranked element is the first pivot out of i th to j th ranked elements. How to analyze this? Thinking acrobatics! Assign every element in the array a random priority (say in 1 [0 , 1] ). Choose pivot to be the element with lowest priority in 2 subproblem. Equivalent to picking pivot uniformly at random 3 (as QuickSort do). Chandra (UIUC) CS498ABD 44 Spring 2019 44 / 51

A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? How to analyze this? Thinking acrobatics! Assign every element in the array a random priority (say in 1 [0 , 1] ). Choose pivot to be the element with lowest priority in 2 subproblem. = ⇒ R i , j happens if either i or j have lowest priority out of elements rank i to j , Chandra (UIUC) CS498ABD 45 Spring 2019 45 / 51

A Slick Analysis of QuickSort Question: What is Pr[R i , j ] ? How to analyze this? Thinking acrobatics! Assign every element in the array a random priority (say in 1 [0 , 1] ). Choose pivot to be the element with lowest priority in 2 subproblem. = ⇒ R i , j happens if either i or j have lowest priority out of elements rank i to j , There are k = j − i + 1 relevant elements. = 2 2 � � Pr R i , j k = j − i + 1 . Chandra (UIUC) CS498ABD 45 Spring 2019 45 / 51

A Slick Analysis of QuickSort Question: What is Pr[ R ij ] ? Chandra (UIUC) CS498ABD 46 Spring 2019 46 / 51

A Slick Analysis of QuickSort Question: What is Pr[ R ij ] ? Lemma � � 2 Pr R ij = j − i +1 . Chandra (UIUC) CS498ABD 46 Spring 2019 46 / 51

A Slick Analysis of QuickSort Question: What is Pr[ R ij ] ? Lemma � � 2 Pr R ij = j − i +1 . Proof. Let a 1 , . . . , a i , . . . , a j , . . . , a n be elements of A in sorted order. Let S = { a i , a i +1 , . . . , a j } Observation: If pivot is chosen outside S then all of S either in left array or right array. Observation: a i and a j separated when a pivot is chosen from S for the first time. Once separated no comparison. Observation: a i is compared with a j if and only if either a i or a j is chosen as a pivot from S at separation... Chandra (UIUC) CS498ABD 46 Spring 2019 46 / 51

A Slick Analysis of QuickSort Continued... Lemma � � 2 Pr = j − i +1 . R ij Proof. Let a 1 , . . . , a i , . . . , a j , . . . , a n be sort of A . Let S = { a i , a i +1 , . . . , a j } Observation: a i is compared with a j if and only if either a i or a j is chosen as a pivot from S at separation. Observation: Given that pivot is chosen from S the probability that it is a i or a j is exactly 2 / | S | = 2 / ( j − i + 1) since the pivot is chosen uniformly at random from the array. Chandra (UIUC) CS498ABD 47 Spring 2019 47 / 51