COMP 3403 Algorithm Analysis Part 3 Chapters 6 7 Jim Diamond - PowerPoint PPT Presentation

COMP 3403 — Algorithm Analysis Part 3 — Chapters 6 – 7 Jim Diamond CAR 409 Jodrey School of Computer Science Acadia University

Chapter 6 Transform-and-Conquer Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 95 Transform and Conquer • Idea: somehow transform the given instance, or any instance of a problem, to something simpler or something already solved • Three major variations: – instance simplification : transform a problem instance to a simpler or more convenient instance of the same problem – representation change : transform a problem instance to a different representation of the same instance – problem reduction : transform a problem instance to an instance of a different problem , for which a solution technique is already known • The concept of problem reduction is well-known in various areas of mathematics, and figures heavily in the study of NP-completeness – Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 96 Presorting • Many problems involving lists are easier when the list is sorted – – computing the median (selection problem) – checking if all elements are distinct (element uniqueness) • Also: – topological sorting helps to solve some problems on dags – presorting is used in many geometric algorithms • Note: if sorting is more expensive than another solution to the original problem, it makes little or no sense to do this transformation Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 97 Presorting Example: Checking For Uniqueness • Suppose you have a list of numbers and want to confirm that all numbers are unique – this is Θ( n 2 ) – GEQ: What is EXACT answer? • Instead, consider presorting the list – • Time complexity? T unique ( n ) = T sort ( n ) + T scan ( n ) ∈ Θ( n lg n ) + Θ( n ) = Θ( n lg n ) • Other possibilities? – – linear time? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 98 Searching • Searching is a ubiquitous problem in computer science • Example: use grep to find a matching string in a text file or text stream • Example: use a web search engine to find 345,678 matches to a query • Example: in a database use a command like select <stuff> from <some table> where <some condition>; to ask the database to search for some data • Q: how much time does grep take to search through an input of n bytes? – – even if the search stops upon the first match, you still might need to process the entire file • While linear time is generally a good time complexity for a problem, in the case of searching that may not be acceptable – Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 99 Binary Trees • A binary tree is a tree in which each node has at most two children (the left child and the right child) Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 100 Binary Search Trees (also called “ordered binary trees”) • A binary search tree is a binary tree in which – all children in the left subtree of a given node N have values less – than N ’s value all children in the right subtree of a given node N have values – greater than N ’s value 6 3 8 1 5 9 • A binary search tree is said to be balanced if, for every pair of leaves, the lengths of the paths from the root to these leaves are different by at most 1 – • Q: how can we efficiently build a balanced search tree? Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 101 Binary Search Trees: 2 • If we use a straightforward technique for building a binary search tree, some input sequences create “deep” trees – – 2 would be at the root, 3 its only child, 5 the only child of 3, and so on with n items in this sequence, the maximum depth would be n − 1 – • Q: how many “probes” to find an item which is in the tree? A: on average, about n/ 2 • Q: how many “probes” to determine an item is not in the tree? A: on average, about n/ 2 • Not much better than an unsorted list! – • Solution: balance the tree Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 102 Balanced Search Trees • There are various ways of balancing a search tree – AVL trees: binary trees where the subtree heights differ by at most 1 – if an addition or deletion unbalances the tree, some rotations are done to re-balance the tree – Red-black trees: binary trees where, at any node, the height of one subtree can be at most twice the height of the other • 2–3 trees: all leaves are at the same depth, but each internal node can have either 2 or 3 children Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 103 2–3 Tree Node Insertion Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 104 2–3 Tree: Analysis • Q: how good is a 2–3 tree? (That is, how quickly can we search?) • A: it depends on the exact shape of the tree • Consider the case of an n -key tree of height h where all internal nodes have degree 2; we must have n = 1 + 2 + 4 + · · · + 2 h = 2 h +1 − 1 Therefore h = log 2 ( n + 1) − 1 • Now consider the case of an n -key tree of height h where all internal nodes have degree 3; we must have n = 2 + 6 + 18 + · · · + 2 · 3 h = 2(1 + 3 + 9 + · · · + 3 h ) = 3 h +1 − 1 Therefore h = log 3 ( n + 1) − 1 • These provide upper and lower bounds for any 2–3 tree with n keys, so that log 3 ( n + 1) − 1 ≤ h ≤ log 2 ( n + 1) − 1 • Therefore insertion, deletion and searching are all Θ(log n ) in both the average case and the worst case Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 105 Heaps • A priority queue is an abstract data type which supports the following operations: – • A heap is a data structure which implements a priority queue – it is an implicit data structure — no pointers/links/etc are used • A heap is a balanced binary tree in which the deepest leaves are all to the left of the other leaves • Note that the children of the node in array index n are in indices 2 n and 2 n + 1 Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 106 Heaps: 2 • In a (max-)heap, the values stored in the children of a node N have values less than that stored in N – • This is similar to a binary tree – but in the case of a heap there is no specified relationship between the values in the two child nodes • Note that a left child can have a value greater or less than that of its sibling Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 107 Heap Properties (Max Heap) • For any given n , there is only 1 binary tree which has the right shape to be a heap • The largest element in a heap is always at the root • For any node in a heap, that node with its left and right subtrees is also a heap • To represent a heap in an array, just write down the elements top to bottom, left to right Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 108 Constructing a Heap (“Heapify”) • Insert the elements in the array (or the binary tree) in the order they are received • Then starting at the bottom level of non-leaves, if the parent’s value is not larger than both children, swap the parent with the largest child – Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 109 Heapify Pseudo-Code HeapBottomUp(H[1..n]) // Construct a max-heap from the elements of the given array // using the bottom-up algorithm. // Input: an array H[1..n] of orderable items // Output: a max-heap H[1..n] for i = n/2 downto 1 do // i is root of current heap k = i // index of sub-tree being checked v = H[k] // save this node ’ s value heap = false while not heap and 2 * k < n do j = 2 * k if j < n // if there are two children if H[j] < H[j + 1] // pick the larger value j++ if v >= H[j] // is this node ’ s value bigger? heap = true // if so we are done else H[k] = H[j] // else "bubble" child up k = j H[k] = v // put the saved value in the empty slot Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 110 Efficiency of Heapify Assume the tree is full; i.e., n = 2 k − 1 for some k • define h = ⌊ lg n ⌋ to be the height of the tree – in this case, h = k − 1 – • In the worst case, every time we examine an internal node it will need to be moved to a leaf – • Thus a value at level i in the tree will be compared to 2( h − i ) other values (note: the root is at level 0) • The total number of comparisons is therefore h − 1 h − 1 2( h − i )2 i = 2 � � � � � C worst ( n ) = 2( h − i ) = n − lg( n + 1) i =0 level i =0 i keys – Jim Diamond, Jodrey School of Computer Science, Acadia University

Chapter 6 111 Heapsort Using A Max-Heap • We can sort an array A of n items as follows: – for i = n to 2 do – swap A [1] with A [ i ] – – swap new root with largest child until A [1 .. i − 1] is a valid heap – • A [1 .. n ] is now sorted in increasing order Jim Diamond, Jodrey School of Computer Science, Acadia University