Algorithms 1. Existence 2. Efficiency Time Space Worst case - PowerPoint PPT Presentation

Algorithms 1. Existence 2. Efficiency • Time • Space Worst case behavior analysis as a function of input size Asymptotic growth: O W Q o w Donald Knuth

Upper Bounds Definition: f(n) = O(g(n))  $ c,k > 0 ' 0  f(n)  c  g(n) " n>k  Lim f(n) / g(n) exists n  O(g(n))={f | $ c,k>0 ' 0  f(n)  c  g(n) " n>k} “ f(n) is big-O of g(n) ” g(n) n = O(n 2 ) Ex: 33n+17 = O(n) f(n) n 8 +n 7 = O(n 12 ) n 100 = O(2 n ) n k 213 = O(1)

Lower Bounds Definition: f(n) = W (g(n))  g(n)=O(f(n))  Lim g(n) / f(n) exists n  W (g(n))={f | $ c,k>0 ' 0  g(n)  c  f(n) " n>k} “ f(n) is Omega of g(n) ” f(n) 100n = W (n) Ex: 33n+17 = W (log n) g(n) n 8 -n 7 = W (n 8 ) 213 = W (1/n) n k 10 100 = W (1)

Tight Bounds Definition: f(n) = Q (g(n))  f(n)=O(g(n)) and g(n)=O(f(n))  f(n)=O(g(n)) and f(n)= W (g(n))  Lim g(n)/f(n) and Lim f(n)/g(n) exist n  n  “ f(n) is Theta of g(n) ” c 1  g(n) 99n = Q (n) f(n) Ex: n + log n = Q (n) c 2  g(n) n 8 -n 7 = Q (n 8 ) n 2 + cos(n) = Q (n 2 ) n 213 = Q (1) k

Loose Bounds Definition: f(n) = o (g 1 (n))  f(n)=O(g 1 (n)) and f(n) ≠ W (g 1 (n)) “ f(n) is little-o of g 1 (n) ” Definition: f(n) = w (g 2 (n))  f(n)= W (g 2 (n)) and f(n) ≠O( g 2 (n)) “ f(n) is little-omega of g 2 (n) ” g 1 (n) 8n = o (n log log n) Ex: f(n) n log n = w (n) n 6 = o (n 6.01 ) g 2 (n) n 2 + sin(n) = w (n) 213 = o (log n) n k

Growth Laws Let f 1 (n)=O(g 1 (n)) and f 2 (n)=O(g 2 (n)) Thm: f 1 (n) + f 2 (n) = O(max(g 1 (n) , g 2 (n)) • Sequential code Thm: f 1 (n) • f 2 (n) = O(g 1 (n) • g 2 (n)) • Nested loops & subroutine calls Thm: n k = O(c n ) " c,k>0 Ex: n 1000 = O(1.001 n )

Solving Recurrences T(n) = a • T(n/b) + f(n) a≥1, b>1, and let c = log b a Thm: f(n)= O (n c- e ) for some e> 0  T(n)= Q (n c ) f(n)= Q (n c )  T(n)= Q (n c log n) f(n)= W (n c+ e ) some e> 0 and a • f(n/b ) ≤ d • f(n) for some d<1 " n>n 0  T(n)= Q (f(n)) Ex: T(n) = 2T(n/2)+n  T(n)= Q (n log n) T(n) = 9T(n/3)+n  T(n)= Q (n 2 ) T(n) = T(2n/3)+1  T(n)= Q (log n)

Stirling’s Formula        Factorial: n! 1 2 3 . . . (n - 2) (n - 1) n n       n 1           πn Θ Theorem: n! 2 1       e  n  where e is Euler’s constant = 2.71828… n   n   Theorem:   πn n! 2   e     n     πn n n log 2   Corollary:           πn log (n! ) log 2 n log O(n log n)       e e 2   log(n!) = O(n log n) • Useful in analyses and bounds

Problem: Given any five points in/on the unit 1 square, is there always a pair with distance ≤ ? 2 1 1 • What approaches fail? • What techniques work and why? • Lessons and generalizations

Problem: Given any five points in/on the unit equilateral triangle, is there always a pair with distance ≤ ½ ? 1 1 1 • What approaches fail? • What techniques work and why? • Lessons and generalizations

Problem: Given any ten points in/on the unit square, what is the maximum pairwise distance? 1 1 • What approaches fail? • What techniques work and why? • Lessons and generalizations

Data Structures

Data Structures • Techniques for organizing information effectively • Allowed operations: • Initialize • Insert • Delete • Search • Min/max • Successor • Predecessor • Merge • Split • Revert

Data Structures Primitive types: Composite types: Lists: 1. Boolean 23. Array 46. Doubly linked list 2. Character 24. Record 47. Array list 3. Floating-point 25. Union 48. Linked list 4. Double 26. Tagged union 49. Self-organizing list 5. Integer 50. Skip list Arrays: 6. Enumerated type 51. Unrolled linked list 27. Bit array 52. VList Abstract data types: 28. Bit field 53. Xor linked list 7. Array 29. Bitboard 54. Zipper 8. Container 30. Bitmap 55. Doubly connected edge list 9. Map 31. Circular buffer 56. Difference list 10. Associative array 32. Control table 57. Free list 11. Dictionary 33. Image 12. Multimap 34. Dynamic array Binary trees: 13. List 35. Gap buffer 58. AA tree 14. Set 36. Hashed array tree 59. AVL tree 15. Multiset / Bag 37. Heightmap 60. Binary search tree 16. Priority queue 38. Lookup table 61. Binary tree 17. Queue 39. Matrix 62. Cartesian tree 18. Double-ended queue 40. Parallel array 63. Order statistic tree 19. Stack 41. Sorted array 64. Pagoda 20. String 42. Sparse array 65. Randomized binary search tree 21. Tree 43. Sparse matrix 66. Red-black tree 22. Graph 44. Iliffe vector 67. Rope 45. Variable-length array

Data Structures Binary trees (continued): B-trees: 113. Beap 68. Scapegoat tree 90. B-tree 114. Skew heap 69. Self-balancing search tree 91. B+ tree 115. Ternary heap 70. Splay tree 92. B*-tree 116. D-ary heap 71. T-tree 93. B sharp tree 117. Brodal queue 72. Tango tree 94. Dancing tree Multiway trees: 73. Threaded binary tree 95. 2-3 tree 118. Ternary tree 74. Top tree 96. 2-3-4 tree 119. K-ary tree 75. Treap 97. Queap 120. And – or tree 76. Weight-balanced tree 98. Fusion tree 121. (a,b)-tree 77. Binary data structure 99. Bx-tree 122. Link/cut tree 100. AList 123. SPQR-tree Trees: Heaps: 124. Spaghetti stack 78. Trie 101. Heap 125. Disjoint-set data structure 79. Radix tree 102. Binary heap 126. Fusion tree 80. Suffix tree 103. Weak heap 127. Enfilade 81. Suffix array 104. Binomial heap 128. Exponential tree 82. Compressed suffix array 105. Fibonacci heap 129. Fenwick tree 83. FM-index 106. AF-heap 130. Van Emde Boas tree 84. Generalised suffix tree 107. Leonardo Heap 131. Rose tree 85. B-trie 108. 2-3 heap 86. Judy array Space-partitioning trees: 109. Soft heap 87. X-fast trie 132. Segment tree 110. Pairing heap 88. Y-fast trie 133. Interval tree 111. Leftist heap 89. Ctrie 134. Range tree 112. Treap

Data Structures Space-partitioning trees (cont): Application-specific trees: 180. MinHash 135. Bin 158. Abstract syntax tree 181. Quotient filter 136. Kd-tree 159. Parse tree 182. Ctrie 137. Implicit kd-tree 160. Decision tree Graphs: 138. Min/max kd-tree 161. Alternating decision tree 183. Graph 139. Adaptive k-d tree 162. Minimax tree 184. Adjacency list 140. Quadtree 163. Expectiminimax tree 185. Adjacency matrix 141. Octree 164. Finger tree 186. Graph-structured stack 142. Linear octree 165. Expression tree 187. Scene graph 143. Z-order 166. Log-structured merge-tree 188. Binary decision diagram 144. UB-tree 189. 0-suppressed decision diagram Hashes: 145. R-tree 167. Bloom filter 190. And-inverter graph 146. R+ tree 168. Count-Min sketch 191. Directed graph 147. R* tree 169. Distributed hash table 192. Directed acyclic graph 148. Hilbert R-tree 170. Double Hashing 193. Propositional dir. acyclic graph 149. X-tree 171. Dynamic perfect hash table 194. Multigraph 150. Metric tree 172. Hash array mapped trie 195. Hypergraph 151. Cover tree 173. Hash list 152. M-tree Other: 174. Hash table 153. VP-tree 196. Lightmap 175. Hash tree 154. BK-tree 197. Winged edge 176. Hash trie 155. Bounding interval hierarchy 198. Doubly connected edge list 177. Koorde 156. BSP tree 199. Quad-edge 178. Prefix hash tree 157. Rapidly exploring random tree 200. Routing table 179. Rolling hash 201. Symbol table

Arrays • Sequence of " indexible" locations 7 . . . 1 2 3 4 5 6 • Unordered: • O(1) to add • O(n) to search / delete • O(n) for min / max • Ordered: • O(n) to add / delete • O(log n) to (binary) search • O(1) for min / max

Stacks • LIFO (Last-In First-Out) in out • Operations: push / pop O(1) each • Can not access “middle” • Analogy: trays/plates at cafeteria • Applications: • Recursion • Compiling / parsing • Dynamic binding • Web surfing

Queues • FIFO (First-In First-Out) in out • Operations: push / pop O(1) each • Can not access “middle” • Analogy: line at the store • Applications: • Simulations • Scheduling • Networks • Operating systems

Linked Lists • Successor / predecessor pointers • Types: • Single linked • Double linked • Circular • Operations: • Add: O(1) time • Search: O(n) time • Delete: O(1) time (given pointer)

Extra Credit Problem: Given a pointer to a read-only (unmodefiable) linked list containing an unknown number of nodes n, devise an O(n)-time and O(1) space algorithm that determines whether that list contains a cycle. or • What approaches fail? • What techniques work and why? • Lessons and generalizations

Trees • Parent/children pointers c • Binary / N -ary b e • Ordered / unordered a d f • Height -balanced: • B -trees • AVL trees • Red -black trees • 2 -3 trees • add / delete / search in O(log n) time

B-Trees • Multi-rotations occur infrequently • Rotations don’t propagate far • Larger tree  fewer rotations • Same for other height -balanced trees • Non -balanced search trees average O(log n) height

Height-Balanced AVL Trees