Selection in expected linear time Tirgul 4 What happens if we are - PDF document

Selection in expected linear time Tirgul 4 • What happens if we are not looking for the smallest or largest • Order Statistics element, but for the i th order statistics? – minimum/maximum • One optional solution: sort ( Ө ( n lg n )) and index, can we do better? – Selection • We can still get an expected asymptotic running time of Θ ( n ) • Heaps using a modification of a randomized quicksort. (average case – Overview analysis) – Heapify – Build-Heap Order statistics Randomized Select • The i th order statistics of a set of n elements is the i th smallest RandomizedSelect ( A,p,r,i ) element. 1. if p == r • For example the minimum is the first order statistics of the set and 2. then return A [ p ] the maximum is the n th . 3. q ← RandomizedPartition ( A,p,r ) • A median is the central element in the set. 4. if i < q then return RandomizedSelect ( A,p,q -1 ,i ) • The median is a very important characteristic of a set and many 5. else if i > q then times we will prefer using the median then using the average. return RandomizedSelect ( A, q +1 , r, i – q ) (why?) 6. else return A [ q ] Minimum & Maximum Randomized Select • We use the same RandomizedPartition like in the • How many comparisons are necessary to determine the minimum/maximum of a set of n elements? randomized quicksort. • An upper bound of n -1 is easy to obtain, but can we do better? • This time, instead of recursively sorting both sides of the pivot, we only deal with one. • It is easy to show that the answer is no. • Are we guaranteed to do better than sort+select? • How about finding both minimum and maximum, can we do better than 2*( n -1) ? • No, like quicksort, we have a worst case of O ( n 2 ) (why?) • yes • But let’s look at the average case: 1

Randomized Select Heaps • A heap is a complete binary tree, in which each node is larger than • We are using the same technique used to analyze the both its sons. randomized quicksort .  −  1 n 1 ( ) ∑ ( ) ≤  −  + T n max( T ( k ), T ( n k )) dn • The largest element of each sub tree is in the root of the sub tree. n   k = 1 • Note: this does not mean that the 2 n − 1 ( ) dn ∑ ≤ + T ( k ) 16 root’s 2 sons are the next largest. n = k n / 2 −  −  2 n 1 2 c n 1 n / 2 ∑ ∑ ∑ ≤ + =  −  + ck dn k k dn 13 9 • Assuming T( k ) ≤ ck we get: n n   k = n / 2 k = 1 k = 1 2 c  n ( n − 1 ) n / 2 ( n / 2 − 1 )  =  −  + dn 12 3 7 4 n  2 2      c n 3 1 ( ) ≤ − −  −  + =  −  + c n 1 1 dn c n dn 5 2 1 2  2   4 2  • We can pick c large enough such that: − + ≤ 3 / 4 cn 1 / 2 c dn cn Order Statistics Heaps • A heap can be represented by an array. • So we can find the i th order statistics either in Ө ( n lg n ) time, or in • Levels are stored one after the other. an average Ө ( n ) time, but with a worst case of O ( n 2 ). • The root is stored in A[1]. • Can we do better? 16 • Yes we can, a modified version of quick-select has a linear worst • The sons of A[i] are A[2i] case time (but with a larger constant). 13 9 and A[2i+1] . • We won’t get into details (see Cormen, 10.3 – selection in worst- 12 3 7 4 case linear time). 16 13 9 12 3 7 4 5 2 1 5 2 1 Select in worst case linear time Heapify select algorithm idea: • • Assumes that both subtrees of the root are heaps, but the root may be 1. Devide the input into n / c groups of c elements (for example, c = 5) smaller than one of its children. 2. Find the median of each group. • The idea is to let the value at the 3. Find the median of these medians. 1 root to “float down” to the 4. Partition the input around the median of medians and call select recursively. right position. • Proof idea: 13 9 • What can we say about – Asymptotically, at least ¼ of the elements complexity? 12 3 7 4 are larger than the pivot and at least ¼ are smaller than the pivot. • Worst case complexity – In the worst case, the number of elements 5 2 of lg n (the tree is complete). in the recursive call is 3 n /4. – You’ve seen in class that quicksort achieves n lg n time even when the recurrence is called for 9 n /10 of the elements. 2

Heapify Priority Queue Heapify(Node x) • Each inserted element has a priority. largest = max {left(x), right(x)} • Extraction order is according to priority. if ( largest > x ) • Supported operation are Insert, Maximum, Extract-Max. exchange (largest, x) heapify (x) • Easily implemented with heaps. 1 13 13 9 12 9 12 3 7 4 5 3 7 4 5 2 1 2 Heap-Extract-Max Priority Queue • Save the root as max . • Priority Queues using heaps: • Remove the last node and place it in the root. – Maximum operation takes O (1) • Do Heapify . – Extract-Max operation takes O (log n ) • Return max . – Insert operation takes O (log n ) • Priority Queues using sorted list 16 13 – Maximum operation takes O (1) 13 9 12 9 – Extract-Max operation takes O (1) – Insert operation takes O ( n ) 12 3 7 4 5 3 7 4 5 2 1 1 2 Heap-Insert Build-Heap Build-Heap(A) • Insert new value at the end of the heap. for i =  length[A]/2  downto 1 • Let it “float up” to the right position. do Heapify[A,i] • We still have an O (lg n) complexity. 16 16 3 5 7 2 1 9 4 12 16 13 3 5 9 16 13 7 4 12 2 1 13 9 15 9 3 5 7 2 13 9 4 12 16 1 3 16 9 12 13 7 4 5 2 1 12 3 7 4 12 13 7 4 5 2 1 15 new 3 5 7 16 13 9 4 12 2 1 16 13 9 12 3 7 4 5 2 1 5 2 1 3 value 3

Build-Heap vs. Heap-Insert • We want to create a new heap, containing n items, what should we do? Build a heap or insert the n items one by one? • Build-Heap runs in O ( n ) (why?). • Inserting n items takes O ( n log n ). • Sometimes Build-Heap and Heap-Insert create different heaps from the same input. – For example: the input sequence 1, 2, 3, 4 4 4 Build-Heap: Heap-Insert: 2 3 3 2 1 1 Heapsort Heapsort(A) Build-Heap(A) for i=length[A] downto 2 do exchange A[1] with A[i] heap-size[A]=heap-size[A]-1 Heapify(A, 1) 16 13 9 12 3 7 4 5 2 1 9 5 7 2 3 1 4 12 13 16 13 12 9 5 3 7 4 1 2 16 7 5 4 2 3 1 9 12 13 16 12 5 9 2 3 7 4 1 13 16 5 3 4 2 1 7 9 12 13 16 Questions • How to implement a stack/queue using a priority queue? • How to implement an Increase-Key operation which increases the value of some node? • How to delete a given node from the heap in O (log n )? • How to search for a key in a heap? 4