The Divide and Conquer Paradigm CSE 421: Algorithms and Computational Complexity Outline: General Idea Summer 2007 Review of Merge Sort Why does it work? Larry Ruzzo Importance of balance Importance of super-linear growth Divide and Conquer Algorithms Two interesting applications Polynomial Multiplication Matrix Multiplication Finding & Solving Recurrences 2 1 Algorithm Design Techniques Mergesort (review) Divide & Conquer Mergesort: (recursively) sort 2 half-lists, then merge results. Reduce problem to one or more sub-problems of the same type Typically, each sub-problem is at most a constant T(n)=2T(n/2)+cn, n ≥ 2 fraction of the size of the original problem T(1)=0 e.g. Mergesort, Binary Search, Strassen’s Algorithm, Log n levels O(n) Quicksort (kind of) Solution: O(n log n) work per (details later) level 3 4 1
Why Balanced Subdivision? Another D&C Approach Suppose we've already invented DumbSort, Alternative "divide & conquer" algorithm: taking time n 2 Sort n-1 Try Just One Level of divide & conquer: Sort last 1 DumbSort(first n/2 elements) Merge them DumbSort(last n/2 elements) Merge results T(n)=T(n-1)+T(1)+3n for n ≥ 2 D&C in a Time: 2 (n/2) 2 + n = n 2 /2 + n << n 2 T(1)=0 nutshell Almost twice as fast! Solution: 3n + 3(n-1) + 3(n-2) … = Θ (n 2 ) 9 10 Another D&C Approach, cont. Another D&C Approach, cont. Moral 1: “two halves are better than a whole” Moral 3: unbalanced division less good: Two problems of half size are better than one full-size (.1n) 2 + (.9n) 2 + n = .82n 2 + n problem, even given the O(n) overhead of recombining, since The 18% savings compounds significantly if you carry recursion to the base algorithm has super-linear complexity. more levels, actually giving O(nlogn), but with a bigger constant. So worth doing if you can’t get 50-50 split, but balanced is better if you Moral 2: “If a little's good, then more's better” can. This is intuitively why Quicksort with random splitter is good – two levels of D&C would be almost 4 times faster, 3 levels badly unbalanced splits are rare, and not instantly fatal. almost 8, etc., even though overhead is growing. Best is (1) 2 + (n-1) 2 + n = n 2 - 2n + 2 + n usually full recursion down to some small constant size (balancing "work" vs "overhead"). Little improvement here. 11 12 2
Closest Pair of Points 5.4 Closest Pair of Points Closest pair. Given n points in the plane, find a pair with smallest Euclidean distance between them. Fundamental geometric primitive. Graphics, computer vision, geographic information systems, molecular modeling, air traffic control. Special case of nearest neighbor, Euclidean MST, Voronoi. fast closest pair inspired fast algorithms for these problems Brute force. Check all pairs of points p and q with Θ (n 2 ) comparisons. 1-D version. O(n log n) easy if points are on a line. Assumption. No two points have same x coordinate. to make presentation cleaner 14 Closest Pair of Points: First Attempt Closest Pair of Points: First Attempt Divide. Sub-divide region into 4 quadrants. Divide. Sub-divide region into 4 quadrants. Obstacle. Impossible to ensure n/4 points in each piece. L L 15 16 3
Closest Pair of Points Closest Pair of Points Algorithm. Algorithm. Divide: draw vertical line L so that roughly ½ n points on each side. Divide: draw vertical line L so that roughly ½ n points on each side. Conquer: find closest pair in each side recursively. L L 21 12 17 18 Closest Pair of Points Closest Pair of Points Algorithm. Find closest pair with one point in each side, assuming that distance < δ . Divide: draw vertical line L so that roughly ½ n points on each side. Conquer: find closest pair in each side recursively. Combine: find closest pair with one point in each side. seems like Θ (n 2 ) Return best of 3 solutions. L L 8 21 21 δ = min(12, 21) 12 12 19 20 4
Closest Pair of Points Closest Pair of Points Find closest pair with one point in each side, assuming that distance < δ . Find closest pair with one point in each side, assuming that distance < δ . Observation: only need to consider points within δ of line L. Observation: only need to consider points within δ of line L. Sort points in 2 δ -strip by their y coordinate. L L 7 6 5 21 21 4 δ = min(12, 21) δ = min(12, 21) 12 12 3 2 1 δ δ 21 22 Closest Pair of Points Closest Pair of Points Find closest pair with one point in each side, assuming that distance < δ . Def. Let s i be the point in the 2 δ -strip, with Observation: only need to consider points within δ of line L. the i th smallest y-coordinate. Sort points in 2 δ -strip by their y coordinate. Claim. If |i – j| ≥ 8, then the distance between Only check distances of those within 8 positions in sorted list! j 39 s i and s j is at least δ . Pf. No two points lie in same ½ δ -by- ½ δ box. L only 8 boxes 31 7 30 ½ δ 6 29 5 21 ½ δ 4 28 i 27 δ = min(12, 21) 26 12 3 25 2 1 δ δ δ 23 24 5
Closest Pair Algorithm Going From Code to Recurrence Closest-Pair(p 1 , …, p n ) { if(n <= ??) return ?? Compute separation line L such that half the points are on one side and half on the other side. Carefully define what you’re counting, and write it down! “Let C(n) be the number of comparisons between sort keys used by δ 1 = Closest-Pair(left half) δ 2 = Closest-Pair(right half) MergeSort when sorting a list of length n ≥ 1” δ = min( δ 1 , δ 2 ) In code, clearly separate base case from recursive case , Delete all points further than δ from separation line L highlight recursive calls, and operations being counted. Sort remaining points p[1]…p[m] by y-coordinate. Write Recurrence(s) for i = 1..m k = 1 while i+k <= m && p[i+k].y < p[i].y + δ δ = min( δ , distance between p[i] and p[i+k]); k++; return δ . } 26 25 Closest Pair Algorithm Closest Pair of Points: Analysis Basic operations: Base Case distance calcs Closest-Pair(p 1 , …, p n ) { Running time. Recursive calls (2) if(n <= 1) return ∞ 0 Compute separation line L such that half the points � 0 n = 1 � are on one side and half on the other side. T( n ) � � � � � T( n ) = O ( n log n ) ( ) + 7 n 2 T n /2 n > 1 � δ 1 = Closest-Pair(left half) 2T(n / 2) δ 2 = Closest-Pair(right half) δ = min( δ 1 , δ 2 ) BUT - that’s only the number of distance calculations Delete all points further than δ from separation line L Basic operations at Sort remaining points p[1]…p[m] by y-coordinate. this recursive level for i = 1..m k = 1 while i+k <= m && p[i+k].y < p[i].y + δ O(n) δ = min( δ , distance between p[i] and p[i+k]); k++; return δ . } 27 28 6
Closest Pair Algorithm Closest Pair of Points: Analysis Basic operations: Base Case comparisons Closest-Pair(p 1 , …, p n ) { Running time. Recursive calls (2) if(n <= 1) return ∞ 0 Compute separation line L such that half the points O(n log n) � 0 n = 1 � are on one side and half on the other side. � � T( n ) = O ( n log 2 n ) T( n ) � � � 2 T n /2 ( ) + O ( n log n ) n > 1 � δ 1 = Closest-Pair(left half) 2T(n / 2) δ 2 = Closest-Pair(right half) δ = min( δ 1 , δ 2 ) 1 Q. Can we achieve O(n log n)? Delete all points further than δ from separation line L O(n) A. Yes. Don't sort points from scratch each time. Basic operations at Sort remaining points p[1]…p[m] by y-coordinate. O(n log n) Sort by x at top level only. this recursive level Each recursive call returns δ and list of all points sorted by y for i = 1..m k = 1 Sort by merging two pre-sorted lists. while i+k <= m && p[i+k].y < p[i].y + δ O(n) δ = min( δ , distance between p[i] and p[i+k]); k++; T ( n ) � 2 T n /2 ) + O ( n ) T( n ) = O ( n log n ) ( � return δ . } 29 30 Integer Arithmetic 5.5 Integer Multiplication Add. Given two n-digit integers a and b, compute a + b. O(n) bit operations. Multiply. Given two n-digit integers a and b, compute a × b. Brute force solution: Θ (n 2 ) bit operations. 1 1 0 1 0 1 0 1 * 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 Multiply 1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 0 1 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 1 1 0 1 0 1 0 1 1 1 0 1 0 1 0 1 0 + 0 1 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 1 0 1 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 Add 0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1 32 7
Recommend
More recommend
