CS 473: Algorithms Ruta Mehta University of Illinois, Urbana-Champaign Spring 2018 Ruta (UIUC) CS473 1 Spring 2018 1 / 61
CS 473: Algorithms, Spring 2018 Review session Lecture 99 Feb 22, 2018 Some of the slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 2 Spring 2018 2 / 61
What we saw so far... Fast Fourier Transform (FFT). Dynamic Programming String algorithms. Graph algorithms: shortest path, independent set, dominating set, etc. Randomozed Algorithms Quick sort, High probability analysis: Markov, Chebyshev, and Chernoff inequalities Ruta (UIUC) CS473 3 Spring 2018 3 / 61
What we saw so far... Fast Fourier Transform (FFT). Dynamic Programming String algorithms. Graph algorithms: shortest path, independent set, dominating set, etc. Randomozed Algorithms Quick sort, High probability analysis: Markov, Chebyshev, and Chernoff inequalities Hashing, Fingerprinting Ruta (UIUC) CS473 3 Spring 2018 3 / 61
Part I FFT Ruta (UIUC) CS473 4 Spring 2018 4 / 61
What is Fast Fourier Transform Definition Given a polynomial a = ( a 0 , a 1 , . . . , a n − 1 ) in coefficient representation the Discrete Fourier Transform (DFT) of a is the vector a ′ = ( a ′ 0 , a ′ 1 , . . . , a ′ n − 1 ) where a ′ j = a ( ω j n ) for 0 ≤ j < n . a ′ is a sample representation of polynomial with coefficient reprentation a at n ’th roots of unity. We have shown that a ′ can be computed from a in O ( n log n ) time. This divide and conquer algorithm is called the Fast Fourier Transform (FFT). Ruta (UIUC) CS473 5 Spring 2018 5 / 61
Why FFT? Convolution and Polynomial Multiplication Convolution Convolution of vectors a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) is a vector c = ( c 0 , c 1 , . . . , c 2 n − 2 ) , where � a i · b j c k = i , j : i + j = k Ruta (UIUC) CS473 6 Spring 2018 6 / 61
Why FFT? Convolution and Polynomial Multiplication Convolution Convolution of vectors a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) is a vector c = ( c 0 , c 1 , . . . , c 2 n − 2 ) , where � a i · b j c k = i , j : i + j = k Polynomial Multiplication If vectors a and b are coefficients of two n − 1 degree polynomials, i =0 b i x i then c (abusing notation) a ( x ) = � n − 1 b ( x ) = � n − 1 i =0 a i x i , is the coefficient vector of the product polynomial a ( x ) ∗ b ( x ) . Ruta (UIUC) CS473 6 Spring 2018 6 / 61
Why FFT? Convolution and Polynomial Multiplication Convolution Given vectors a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) find its convolution vector c = ( c 0 , c 1 , . . . , c 2 n − 2 ) . Evaluate polynomials a and b at the 2 n th roots of unity, to get 1 their sample representation a ′ and b ′ . Compute sample representation c ′ = ( a ′ 0 b ′ 0 , . . . , a ′ 2 n − 2 b ′ 2 n − 2 ) 2 of product c = a · b Compute c from c ′ using inverse Fourier transform. 3 Step 1 takes O ( n log n ) using two FFTs Step 2 takes O ( n ) time Step 3 takes O ( n log n ) using one FFT Ruta (UIUC) CS473 7 Spring 2018 7 / 61
Problem Let ¯ a = a 0 , a 1 , . . . , a n − 1 be a sequence of n numbers representing value of a function at different points, we would like to “smooth” it using vector ¯ b = ( b 0 , b 1 , . . . , b k − 1 ) for k ≤ n as follows: a ′ = a ′ 0 , a ′ 1 , . . . , a ′ n − 1 where a ′ ¯ i = a i b 0 + ( a i +1 b 1 + . . . + a i + k − 1 b k − 1 ) + ( a i − 1 b 1 + a i − 2 b 2 + . . . + a i − k +1 b k − 1 ) . If an index goes out of bounds we assume that the corresponding value is 0 . a ′ can be computed in O ( n 2 ) time. a and ¯ Given ¯ b describe how ¯ Ruta (UIUC) CS473 8 Spring 2018 8 / 61
Application of FFT Let ¯ a = a 0 , a 1 , . . . , a n − 1 be a sequence of n numbers representing value of a function at different points, we would like to “smooth” it using vector ¯ b = ( b 0 , b 1 , . . . , b k − 1 ) for k ≤ n as follows: a ′ = a ′ 0 , a ′ 1 , . . . , a ′ n − 1 where a ′ ¯ i = a i b 0 + ( a i +1 b 1 + . . . + a i + k − 1 b k − 1 ) + ( a i − 1 b 1 + a i − 2 b 2 + . . . + a i − k +1 b k − 1 ) . If an index goes out of bounds we assume that the corresponding value is 0 . a ′ can be computed in O ( n log n ) time. a and ¯ Given ¯ b describe how ¯ Ruta (UIUC) CS473 9 Spring 2018 9 / 61
Ruta (UIUC) CS473 10 Spring 2018 10 / 61
Part II Dynamic Programming Ruta (UIUC) CS473 11 Spring 2018 11 / 61
Recursion Reduction: Reduce one problem to another Recursion A special case of reduction reduce problem to a smaller instance of itself 1 self-reduction 2 Problem instance of size n is reduced to one or more instances 1 of size n − 1 or less. For termination, problem instances of small size are solved by 2 some other method as base cases . Ruta (UIUC) CS473 12 Spring 2018 12 / 61
What is Dynamic Programming? Every recursion can be memoized. Automatic memoization does not help us understand whether the resulting algorithm is efficient or not. Ruta (UIUC) CS473 13 Spring 2018 13 / 61
What is Dynamic Programming? Every recursion can be memoized. Automatic memoization does not help us understand whether the resulting algorithm is efficient or not. Dynamic Programming: A recursion that when memoized leads to an efficient algorithm. Ruta (UIUC) CS473 13 Spring 2018 13 / 61
Edit Distance Definition Edit distance between two words X and Y is the number of letter insertions, letter deletions and letter substitutions required to obtain Y from X . Example The edit distance between FOOD and MONEY is at most 4 : FOOD → MOOD → MONOD → MONED → MONEY Ruta (UIUC) CS473 14 Spring 2018 14 / 61
Edit Distance: Alternate View Alignment Place words one on top of the other, with gaps in the first word indicating insertions, and gaps in the second word indicating deletions. F O O D M O N E Y Ruta (UIUC) CS473 15 Spring 2018 15 / 61
Edit Distance: Alternate View Alignment Place words one on top of the other, with gaps in the first word indicating insertions, and gaps in the second word indicating deletions. F O O D M O N E Y Formally, an alignment is a set M of pairs ( i , j ) such that each index appears exactly once, and there is no “crossing”: if ( i , j ) , ..., ( i ′ , j ′ ) then i < i ′ and j < j ′ . One of i or j could be empty, in which case no comparision. In the above example, this is M = { (1 , 1) , (2 , 2) , (3 , 3) , ( , 4) , (4 , 5) } . Ruta (UIUC) CS473 15 Spring 2018 15 / 61
Edit Distance: Alternate View Alignment Place words one on top of the other, with gaps in the first word indicating insertions, and gaps in the second word indicating deletions. F O O D M O N E Y Formally, an alignment is a set M of pairs ( i , j ) such that each index appears exactly once, and there is no “crossing”: if ( i , j ) , ..., ( i ′ , j ′ ) then i < i ′ and j < j ′ . One of i or j could be empty, in which case no comparision. In the above example, this is M = { (1 , 1) , (2 , 2) , (3 , 3) , ( , 4) , (4 , 5) } . Cost of an alignment: the number of mismatched columns. Ruta (UIUC) CS473 15 Spring 2018 15 / 61
Edit Distance Problem Problem Given two words, find the edit distance between them, i.e., an alignment of smallest cost. Ruta (UIUC) CS473 16 Spring 2018 16 / 61
Edit Distance Basic observation Let A = α x and B = β y α, β : strings. x and y single characters. Possible alignments between A and B α x α x α x or or β y β y β y Observation Prefixes must have optimal alignment! Ruta (UIUC) CS473 17 Spring 2018 17 / 61
Edit Distance Basic observation Let A = α x and B = β y α, β : strings. x and y single characters. Possible alignments between A and B α x α x α x or or β y β y β y Observation Prefixes must have optimal alignment! EDIST ( α, β ) + [ x � = y ] EDIST ( A , B ) = min 1 + EDIST ( α, B ) 1 + EDIST ( A , β ) Ruta (UIUC) CS473 17 Spring 2018 17 / 61
Recursive Algorithm Assume strings are given as arrays A [1 .. m ] and B [1 .. n ] EDIST ( A [1 .. i ] , B [1 .. j ]) If ( i = 0 ) return j If ( j = 0 ) return i m 1 = 1 + EDIST ( A [1 .. ( i − 1)] , B [1 .. j ]) m 2 = 1 + EDIST ( A [1 .. i ] , B [1 .. ( j − 1)])) If ( A [ m ] = B [ n ]) then m 3 = EDIST ( A [1 .. ( i − 1)] , B [1 .. ( j − 1)]) Else m 3 = 1 + EDIST ( A [1 .. ( i − 1)] , B [1 .. ( j − 1)]) return min( m 1 , m 2 , m 3 ) Call EDIST ( A [1 .. m ] , B [1 .. n ]) Ruta (UIUC) CS473 18 Spring 2018 18 / 61
Memoizing the Recursive Algorithm int M [0 .. m ][0 .. n ] Initialize all entries of M [ i ][ j ] to ∞ return EDIST ( A [1 .. m ] , B [1 .. n ]) EDIST ( A [1 .. i ] , B [1 .. j ]) If ( M [ i ][ j ] < ∞ ) return M [ i ][ j ] (* return stored value *) If ( i = 0 ) M [ i ][ j ] = j ElseIf ( j = 0 ) M [ i ][ j ] = i Else m 1 = 1 + EDIST ( A [1 .. ( i − 1)] , B [1 .. j ]) m 2 = 1 + EDIST ( A [1 .. i ] , B [1 .. ( j − 1)])) If ( A [ i ] = B [ j ]) m 3 = EDIST ( A [1 .. ( i − 1)] , B [1 .. ( j − 1)]) Else m 3 = 1 + EDIST ( A [1 .. ( i − 1)] , B [1 .. ( j − 1)]) M [ i ][ j ] = min( m 1 , m 2 , m 3 ) return M [ i ][ j ] Ruta (UIUC) CS473 19 Spring 2018 19 / 61
Matrix and DAG of Computation 0, 0 α x i x j δ i, j δ . . . . . . ... ... . . . . . . . . . . . . m, n Ruta (UIUC) CS473 20 Spring 2018 20 / 61
Recommend
More recommend
