Divide and Conquer Chapter 4 1 Integer Multiplication 2 Integer - PowerPoint PPT Presentation

Divide and Conquer Chapter 4 1

Integer Multiplication 2

Integer Multiplication 𝑨 = 𝑦 × 𝑧 Each with n bits x= 1 0 1 1 y= 1 1 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 1 1 Z= 1 0 0 0 0 1 0 0 3

Integer Multiplication Naïve_IM(x, y) 1. z = 0 2. while y > 0 3. if (y is odd) then z += x 4. n bit operations x *= 2 5. n iterations y /= 2 6. end 7. return z 8. 𝑈 𝑜 = Θ 𝑜 2 4

D&C Integer Multiplication 𝑦 𝑦 1 𝑦 2 𝑧 𝑧 1 𝑧 2 𝑜 𝑜 2 𝑐𝑗𝑢𝑡 2 𝑐𝑗𝑢𝑡 5

D&C Integer Multiplication 𝑦 𝑦 1 𝑦 2 𝑧 𝑧 1 𝑧 2 𝑦 1 × 𝑧 2 𝑦 2 × 𝑧 2 𝑦 1 × 𝑧 1 𝑦 2 × 𝑧 1 𝑜 𝑜 2 + 𝑦 2 2 + 𝑧 2 𝑨 = 𝑦 1 2 𝑧 1 2 𝑜 z = 𝑦 1 𝑧 1 2 𝑜 + 𝑦 1 𝑧 2 + 𝑦 2 𝑧 1 2 2 + 𝑦 2 𝑧 2 𝑈 𝑜 = 4𝑈 𝑜 𝑈 𝑜 = Θ 𝑜 2 2 + 𝑜 6

Karatsuba’s Algorithm 𝑦 𝑦 1 𝑦 2 𝑧 𝑧 1 𝑧 2 𝑨 = 𝑦𝑧 𝑜 z = 𝑦 1 𝑧 1 2 𝑜 + 𝑦 1 𝑧 2 + 𝑦 2 𝑧 1 2 2 + 𝑦 2 𝑧 2 𝑜 𝑨 = 𝑦 1 𝑧 1 2 𝑜 + 2 + 𝑦 2 𝑧 2 𝑦 1 − 𝑦 2 𝑧 2 − 𝑧 1 + 𝑦 1 𝑧 1 + 𝑦 2 𝑧 2 2 𝑈 𝑜 = 3𝑈 𝑜 𝑈 𝑜 = Θ 𝑜 lg 3 2 + 𝑜 7

Example 𝑦 = 34 𝑦 1 = 3 𝑦 2 = 4 𝑧 = 53 𝑧 1 = 5 𝑧 2 = 3 𝑜 = 2 𝑨 = 𝑦𝑧 z = 𝑦 1 𝑧 1 10 2 + ൫ 𝑦 1 − 𝑦 2 𝑧 2 − 𝑧 1 + 2 2 + 𝑦 2 𝑧 2 𝑦 1 𝑧 1 + 𝑦 2 𝑧 2 10 ൯ 𝑦 = 5 ⋅ 3 100 + −1 ⋅ −2 + 3 ⋅ 5 + 4 ⋅ 3 10 + 4 ⋅ 3 = 1500 + 2 + 15 + 12 ⋅ 10 + 12 = 1802 8

Matrix Multiplication Section 4.2 9

Matrix Multiplication = . 𝐷 = 𝐵. 𝐶 𝑙=𝑜 𝑏 𝑗𝑙 . 𝑐 𝑙𝑘 , assuming A, B, and C are 𝑑 𝑗𝑘 = σ 𝑙=1 square 𝑜 × 𝑜 matrices 10

Simple Matrix Multiplication n n n 𝑈 𝑜 = Θ 𝑜 3 11

D&C Matrix Multiplication 𝐵 = 𝐵 11 𝐵 12 𝐵 22 , 𝐶 = 𝐶 11 𝐶 12 𝐶 22 , 𝐵 21 𝐶 21 𝐷 = 𝐷 11 𝐷 12 𝐷 21 𝐷 22 𝐷 11 𝐷 22 = 𝐵 11 𝐷 12 𝐵 22 . 𝐶 11 𝐵 12 𝐶 12 𝐷 21 𝐵 21 𝐶 21 𝐶 22 𝐷 11 = 𝐵 11 . 𝐶 11 + 𝐵 12 . 𝐶 21 𝐷 12 = 𝐵 11 . 𝐶 12 + 𝐵 12 . 𝐶 22 𝐷 21 = 𝐵 21 . 𝐶 11 + 𝐵 22 . 𝐶 21 𝐷 22 = 𝐵 21 . 𝐶 12 + 𝐵 22 . 𝐶 22 12

D&C Matrix Multiplication 13

Analysis of the D&C Algorithm 𝑜 2 + Θ 𝑜 2 Τ 𝑈 𝑜 = 8𝑈 By applying the Master Theorem 𝑏 = 8 , 𝑐 = 2 , 𝑔 𝑜 = Θ 𝑜 2 𝑜 log 𝑐 𝑏 = 𝑜 3 𝑔 𝑜 = Θ 𝑜 2 = 𝑃 𝑜 3−1 Case 1 applies 𝑈 𝑜 = Θ 𝑜 log 𝑐 𝑏 = Θ 𝑜 3 14

Strassen’s Algorithm 𝐵 = 𝐵 11 𝐵 12 𝐵 22 , 𝐶 = 𝐶 11 𝐶 12 𝐶 22 , 𝐵 21 𝐶 21 𝐷 = 𝐷 11 𝐷 12 𝐷 21 𝐷 22 𝑇 1 = 𝐶 12 − 𝐶 22 𝑇 6 = 𝐶 11 + 𝐶 22 𝑇 2 = 𝐵 11 + 𝐵 12 𝑇 7 = 𝐵 12 − 𝐵 22 𝑇 3 = 𝐵 21 + 𝐵 22 𝑇 8 = 𝐶 21 + 𝐶 22 𝑇 4 = 𝐶 21 − 𝐶 11 𝑇 9 = 𝐵 11 − 𝐵 21 𝑇 5 = 𝐵 11 + 𝐵 22 𝑇 10 = 𝐶 11 + 𝐶 12 15

Strassen’s Algorithm 𝑄 1 = 𝐵 11 . 𝑇 1 𝑄 2 = 𝑇 2 . 𝐶 22 𝑄 3 = 𝑇 3 . 𝐶 11 𝑄 4 = 𝐵 22 . 𝑇 4 𝑄 5 = 𝑇 5 . 𝑇 6 𝑄 6 = 𝑇 7 . 𝑇 8 𝑄 7 = 𝑇 9 . 𝑇 10 𝐷 11 = 𝑄 5 + 𝑄 4 − 𝑄 2 + 𝑄6 𝐷 12 = 𝑄 1 + 𝑄 2 𝐷 21 = 𝑄 3 + 𝑄 4 𝐷 22 = 𝑄 5 + 𝑄 1 − 𝑄 3 − 𝑄 7 16

Strassen’s Algo Sketch 1. Strassen(A, B, n) 2. Split A, B into quadrants 3. Compute S1, …, S10 (Matrix add/sub) 4. Compute P1, …, P7 recursively 5. Compute C1, …, C4 (Matrix add/sub) 6. Return C 𝑜 2 + Θ 𝑜 2 Τ 𝑈 𝑜 = 7𝑈 17

Analysis of Strassen’s Algorithm 𝑜 2 + Θ 𝑜 2 Τ 𝑈 𝑜 = 7𝑈 By applying the Master Theorem 𝑏 = 7 , 𝑐 = 2 , 𝑔 𝑜 = Θ 𝑜 2 𝑜 log 𝑐 𝑏 = 𝑜 log 2 7 = 𝑜 2.80735… ≅ 𝑜 2.8 𝑔 𝑜 = Θ 𝑜 2 = 𝑃 𝑜 lg 7−0.8 Case 1 applies 𝑈 𝑜 = Θ 𝑜 log 𝑐 𝑏 = 𝑃 𝑜 2.81 18

Linear-time Selection Section 9.3 19

Linear-time Selection Given an array 𝐵 of 𝑜 elements and an integer 1 ≤ 𝑙 ≤ 𝑜 , find the 𝑙 𝑢ℎ smallest element in 𝐵 Naïve algorithm, sort 𝐵 and pick the 𝑙 𝑢ℎ element in the sorted array ➔ Θ 𝑜𝑚𝑕 𝑜 Select and remove the smallest element k times ➔ Θ 𝑜𝑙 Another quick-sort-like algorithm Pick the first element (pivot) Place it in its position in the array Recursively process one subarray 20

Quick-sort-like Algorithm 𝑙 𝐵 p 1 21

Quick-sort-like Algorithm 𝑙 𝐵 p 1 p 2 22

Quick-sort-like Algorithm 𝑙 𝐵 p 1 p 2 p 3 23

Quick-sort-like Algorithm 𝑙 𝐵 p 1 p 2 p 3 p 4 24

Quick-sort-like Algorithm 𝑙 𝐵 p 1 p 2 p 3 𝑈 𝑜 = Θ 𝑜 2 p 4 ✓ How to choose a good pivot? 25

Median of Fives A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45} 26

Median of Fives A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45} 1. Partition into groups of 5 94 61 28 32 50 93 90 82 11 31 27 91 22 48 88 13 64 38 69 43 42 12 70 6 35 57 30 65 23 37 19 21 24 67 45 27

Median of Fives A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45} 2. Sort each sublist 12 11 6 21 24 22 42 23 13 19 27 50 30 45 82 37 28 32 57 43 48 88 61 31 35 69 67 65 94 70 64 38 91 93 90 28

Median of Fives A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45} 3. Find the median of each sublist 12 11 6 21 24 22 42 23 13 19 27 50 30 45 82 37 28 32 57 43 48 𝑁 88 61 31 35 69 67 65 94 70 64 38 91 93 90 29

Median of Fives A = {94, 82, 88, 12, 23, 61, 11, 13, 70, 37, 28, 31, 64, 6, 19, 32, 27, 38, 35, 21, 50, 91, 69, 57, 24, 93, 22, 43, 30, 67, 90, 48, 42, 65, 45} 4. Recursively find the median of the medians 𝑁 = 82,37,28,32,57,43,48 Median of medians (m*) = 43 Partition A around m* and recursively process one side 30

Algorithm Pseudo-code S ELECT (𝐵, 𝑜, 𝑙) 1. if (𝑜 ≤ 5) then sort 𝐵 and return 𝑙 𝑢ℎ element 2. Partition 𝐵 into groups of 5 3. 𝑁  Find the median of each group 4. 𝑜 𝑜 𝑛 ∗ = S ELECT (𝑁, 5 , 10 ) 5. Partition 𝐵 around 𝑛 ∗ , let it be at position 𝑗 6. if (𝑗 = 𝑙) then return 𝑛 ∗ 7. if (𝑗 > 𝑙) then return S ELECT (𝐵[1, 𝑗 − 1], 𝑗 − 1, 𝑙) 8. if (𝑗 < 𝑙) then return S ELECT (𝐵[𝑗 + 1, 𝑜], 𝑜 − 𝑗, 𝑙 – 𝑗) 9. 31

Size of Sublist 𝑁 𝑛 ∗ medians of fives 𝑜 5 group 32

Size of Sublist 𝑁 < < 𝑛 ∗ medians of fives 𝑜 5 group 33

Size of Sublist < < < < < < < 𝑁 < < 𝑛 ∗ medians of fives < < < < < < < 𝑜 5 group 34

Size of Sublist ? 𝑛 ∗ ? 𝑛 ∗ A 35

Size of Sublist 𝑜 5 groups each of size 5 𝑛 ∗ is larger than half of them 𝑛 ∗ is larger than 𝑜 10 groups 𝑛 ∗ is larger than at least 3 elements in each group Similarly, 𝑛 ∗ is less than at least 3 elements in each group Size of each of the two sublists is [ 3 10 𝑜, 7 10 𝑜] Worst-case scenario, we prune 3n/10 elements and recursively process 7n/10 36

Recurrence Relation 𝑏 ; 𝑜 ≤ 5 𝑈 𝑜 = ቐ 𝑜 7𝑜 𝑈 5 + 𝑈 10 + 𝑏𝑜 ; 𝑜 > 5 Can we apply the Master theorem? Let’s try recursive tree expansion 37

Recurrence Tree 𝑈 𝑜 38

Recurrence Tree 𝑏𝑜 𝑈 𝑜 𝑈 7𝑜 5 10 39

Recurrence Tree 𝑏𝑜 𝑏𝑜 7𝑏𝑜 5 10 𝑜 𝑈 7𝑜 𝑈 7𝑜 𝑈 49𝑜 𝑈 25 50 50 100 40

Recurrence Tree 𝑏𝑜 𝑏𝑜 𝑏𝑜 7𝑏𝑜 9𝑜 5 10 10 𝑜 𝑈 7𝑜 𝑈 7𝑜 𝑈 49𝑜 81𝑜 𝑈 25 50 50 100 100 41

Running Time 𝑗 𝑗 9 9 𝑗=𝑒 𝑗=𝑒 𝑈 𝑜 = σ 𝑗=0 𝑜 = 𝑜 σ 𝑗=0 10 10 We do not know the depth 𝑒 of the tree 𝑗 9 1 𝑗=∞ 𝑈 𝑜 ≤ 𝑜 σ 𝑗=0 ≤ 𝑜 ≤ 10𝑜 1− 9 10 10 𝑈 𝑜 = Θ 𝑜 42

Proof by Induction We want to prove that 𝑈 𝑜 = 𝑃 𝑜 𝑈 𝑜 ≤ 𝑑𝑜 , for 𝑑 > 0 and 𝑜 ≥ 𝑜 0 Base case: 𝑈 𝑜 = 𝑏 for 𝑜 ≤ 5 Setting 𝑑 ≥ 𝑏 satisfies the base case Assume that 𝑈 𝑜 ≤ 𝑑𝑜 is true for all 𝑜 ≤ 𝑛 We want to prove that it is true for 𝑜 = 𝑛 + 1 𝑛+1 7 𝑛+1 𝑈 𝑛 + 1 = 𝑈 + 𝑈 + 𝑏 𝑛 + 1 5 10 𝑛+1 7 𝑛+1 𝑈 𝑛 + 1 ≤ 𝑑 + 𝑑 + 𝑏 𝑛 + 1 5 10 43

Proof by Induction We want to prove that 𝑑 𝑛 + 1 + 𝑑 7 𝑛 + 1 + 𝑏 𝑛 + 1 ≤ 𝑑 𝑛 + 1 5 10 5 + 7𝑑 𝑑 10 + 𝑏 ≤ 𝑑 𝑑 10 ≥ 𝑏 𝑑 ≥ 10𝑏 By setting 𝑑 ≥ 10𝑏 , the inequality 𝑈 𝑛 + 1 ≤ 𝑑 𝑛 + 1 will be true 𝑈 𝑜 = 𝑃 𝑜 44