Sparse Fourier Transforms Eric Price UT Austin Eric Price Sparse - PowerPoint PPT Presentation

Generic Algorithm Outline 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Algorithm for k = 1 (exact or approximate) 1 Method to reduce to k = 1 case 2 ◮ Split � x into O ( k ) “random” parts Eric Price Sparse Fourier Transforms 8 / 36

Generic Algorithm Outline 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Algorithm for k = 1 (exact or approximate) 1 Method to reduce to k = 1 case 2 ◮ Split � x into O ( k ) “random” parts ◮ Can sample time domain of the parts. Eric Price Sparse Fourier Transforms 8 / 36

Generic Algorithm Outline 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Algorithm for k = 1 (exact or approximate) 1 Method to reduce to k = 1 case 2 ◮ Split � x into O ( k ) “random” parts ◮ Can sample time domain of the parts. ⋆ O ( k log k ) time to get one sample from each of the k parts. Eric Price Sparse Fourier Transforms 8 / 36

Generic Algorithm Outline 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Algorithm for k = 1 (exact or approximate) 1 Method to reduce to k = 1 case 2 ◮ Split � x into O ( k ) “random” parts ◮ Can sample time domain of the parts. ⋆ O ( k log k ) time to get one sample from each of the k parts. Finds “most” of signal; repeat on residual 3 Eric Price Sparse Fourier Transforms 8 / 36

Talk Outline Algorithm for k = 1 1 Eric Price Sparse Fourier Transforms 9 / 36

Talk Outline Algorithm for k = 1 1 Reducing k to 1 2 Eric Price Sparse Fourier Transforms 9 / 36

Talk Outline Algorithm for k = 1 1 Reducing k to 1 2 3 Putting it together Eric Price Sparse Fourier Transforms 9 / 36

Talk Outline Algorithm for k = 1 1 Reducing k to 1 2 3 Putting it together Continuous setting 4 Eric Price Sparse Fourier Transforms 9 / 36

Talk Outline Algorithm for k = 1 1 Reducing k to 1 2 3 Putting it together Continuous setting 4 Eric Price Sparse Fourier Transforms 10 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Then x = ( a , a ω t , a ω 2 t , a ω 3 t , . . . , a ω ( n − 1 ) t ) . Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Then x = ( a , a ω t , a ω 2 t , a ω 3 t , . . . , a ω ( n − 1 ) t ) . x 0 = a Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Then x = ( a , a ω t , a ω 2 t , a ω 3 t , . . . , a ω ( n − 1 ) t ) . x 1 = a ω t x 0 = a Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Then x = ( a , a ω t , a ω 2 t , a ω 3 t , . . . , a ω ( n − 1 ) t ) . x 1 = a ω t x 0 = a x 1 / x 0 = ω t = ⇒ t . Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Then x = ( a , a ω t , a ω 2 t , a ω 3 t , . . . , a ω ( n − 1 ) t ) . x 1 = a ω t x 0 = a x 1 / x 0 = ω t = ⇒ t . � Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, exact case Lemma a We can compute a 1 -sparse � x in O ( 1 ) time. � x : � a if i = t � x i = t 0 otherwise Then x = ( a , a ω t , a ω 2 t , a ω 3 t , . . . , a ω ( n − 1 ) t ) . x 1 = a ω t x 0 = a x 1 / x 0 = ω t = ⇒ t . � (Related to OFDM, Prony’s method, matrix pencil.) Eric Price Sparse Fourier Transforms 11 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. Eric Price Sparse Fourier Transforms 12 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. x 1 / x 0 = ω t With exact sparsity: log n bits in a single measurement. Eric Price Sparse Fourier Transforms 12 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. x 1 / x 0 = ω t + noise With exact sparsity: log n bits in a single measurement. With noise: only constant number of useful bits. Eric Price Sparse Fourier Transforms 12 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. x 1 / x 0 = ω t + noise With exact sparsity: log n bits in a single measurement. With noise: only constant number of useful bits. Choose Θ ( log n ) time shifts c to recover i . Eric Price Sparse Fourier Transforms 12 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. x c 2 / x 0 = ω c 2 t + noise With exact sparsity: log n bits in a single measurement. With noise: only constant number of useful bits. Choose Θ ( log n ) time shifts c to recover i . Eric Price Sparse Fourier Transforms 12 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. x c 3 / x 0 = ω c 3 t + noise With exact sparsity: log n bits in a single measurement. With noise: only constant number of useful bits. Choose Θ ( log n ) time shifts c to recover i . Eric Price Sparse Fourier Transforms 12 / 36

Algorithm for k = 1: one dimension, approximate case Lemma Suppose � x is approximately 1 -sparse: | � x t | / � � x � 2 � 90 %. Then we can recover it with O ( log n ) samples and O ( log 2 n ) time. x c 3 / x 0 = ω c 3 t + noise With exact sparsity: log n bits in a single measurement. With noise: only constant number of useful bits. Choose Θ ( log n ) time shifts c to recover i . Error correcting code with efficient recovery = ⇒ lemma. � Eric Price Sparse Fourier Transforms 12 / 36

Talk Outline Algorithm for k = 1 1 Reducing k to 1 2 3 Putting it together Continuous setting 4 Eric Price Sparse Fourier Transforms 13 / 36

Algorithm for general k Reduce general k to k = 1. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. Random permutation 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. Random permutation 1-sparse recovery 1-sparse recovery x x ′ O ( k ) � Permute Filters 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. Random permutation 1-sparse recovery 1-sparse recovery x x ′ � Permute Filters O ( k ) 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 14 / 36

Algorithm for general k Reduce general k to k = 1. “Filters”: partition frequencies into O ( k ) buckets. ◮ Sample from time domain of each bucket with O ( log n ) overhead. ◮ Recovered by k = 1 algorithm Most frequencies alone in bucket. Random permutation 1-sparse recovery 1-sparse recovery x x ′ � Permute Filters O ( k ) 1-sparse recovery Recovers most of � x : 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Eric Price Sparse Fourier Transforms 14 / 36

Going from finding most coordinates to finding all � x Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x x ′ � O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x x ′ � O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x x ′ � O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Repeat, k → k / 2 → k / 4 → · · · Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x x ′ � O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Repeat, k → k / 2 → k / 4 → · · · Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x x ′ � O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Repeat, k → k / 2 → k / 4 → · · · Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x x ′ � O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Repeat, k → k / 2 → k / 4 → · · · Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x � x ′ O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Repeat, k → k / 2 → k / 4 → · · · Theorem We can compute � x in O ( k log n ) expected time. Eric Price Sparse Fourier Transforms 15 / 36

Going from finding most coordinates to finding all x ′ x − � � Partial k -sparse recovery 1-sparse recovery 1-sparse recovery x � x ′ O ( k ) Permute Filters 1-sparse recovery 1-sparse recovery Lemma (Partial sparse recovery) x ′ such that In O ( k log n ) expected time, we can compute an estimate � x ′ is k / 2 -sparse. x − � � Repeat, k → k / 2 → k / 4 → · · · Theorem We can compute � x in O ( k log n ) expected time. Eric Price Sparse Fourier Transforms 15 / 36

Talk Outline Algorithm for k = 1 1 Reducing k to 1 2 3 Putting it together Continuous setting 4 Eric Price Sparse Fourier Transforms 16 / 36

How can you hope for sublinear time? Time Frequency n -dimensional DFT: O ( n log n ) x → � x × ∗ = = Eric Price Sparse Fourier Transforms 17 / 36

How can you hope for sublinear time? Time Frequency n -dimensional DFT: O ( n log n ) x → � x × ∗ = = Eric Price Sparse Fourier Transforms 17 / 36

How can you hope for sublinear time? Time Frequency n -dimensional DFT: O ( n log n ) x → � x × ∗ n -dimensional DFT of first k terms: O ( n log n ) x · rect → � x ∗ sinc. = = Eric Price Sparse Fourier Transforms 17 / 36

How can you hope for sublinear time? Time Frequency n -dimensional DFT: O ( n log n ) x → � x × ∗ n -dimensional DFT of first k terms: O ( n log n ) x · rect → � x ∗ sinc. = = Eric Price Sparse Fourier Transforms 17 / 36

How can you hope for sublinear time? Time Frequency n -dimensional DFT: O ( n log n ) x → � x × ∗ n -dimensional DFT of first k terms: O ( n log n ) x · rect → � x ∗ sinc. = = k -dimensional DFT of first k terms: O ( B log B ) alias ( x · rect ) → subsample ( � x ∗ sinc ) . Eric Price Sparse Fourier Transforms 17 / 36

How can you hope for sublinear time? Time Frequency n -dimensional DFT: O ( n log n ) x → � x × ∗ n -dimensional DFT of first k terms: O ( n log n ) x · rect → � x ∗ sinc. = = k -dimensional DFT of first k terms: O ( B log B ) alias ( x · rect ) → subsample ( � x ∗ sinc ) . Eric Price Sparse Fourier Transforms 17 / 36

Use a better filter GMS05, HIKP12, IKP14, IK14 Filter (time): Gaussian · sinc Filter (frequency): Gaussian * rectangle Filter: sparse F for which � F is large on an interval . Eric Price Sparse Fourier Transforms 18 / 36

Use a better filter GMS05, HIKP12, IKP14, IK14 Filter (time): Gaussian · sinc Filter (frequency): Gaussian * rectangle Filter: sparse F for which � F is large on an interval . We can permute the frequencies: x ′ ⇒ � x i = � i = x σ i = x σ − 1 i Eric Price Sparse Fourier Transforms 18 / 36

Use a better filter GMS05, HIKP12, IKP14, IK14 Filter (time): Gaussian · sinc Filter (frequency): Gaussian * rectangle Filter: sparse F for which � F is large on an interval . We can permute the frequencies: x ′ ⇒ � x i = � i = x σ i = x σ − 1 i Allows us to convert worst case to random case. Eric Price Sparse Fourier Transforms 18 / 36

Algorithm for exactly sparse signals Original signal x Goal ˆ x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals Computed F · x Filtered signal c F ∗ c x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals F · x aliased to k terms Filtered signal c F ∗ c x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals F · x aliased to k terms Computed samples of c F ∗ c x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals F · x aliased to k terms Computed samples of c F ∗ c x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals F · x aliased to k terms Knowledge about ˆ x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals F · x aliased to k terms Knowledge about ˆ x Eric Price Sparse Fourier Transforms 19 / 36

Algorithm for exactly sparse signals F · x aliased to k terms Knowledge about ˆ x Lemma If t is isolated in its bucket and in the “super-pass” region, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Eric Price Sparse Fourier Transforms 19 / 36

Algorithm Lemma For most t, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Eric Price Sparse Fourier Transforms 20 / 36

Algorithm Lemma For most t, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Time-shift x by one and repeat: b ′ = � x t ω t . Divide to get b ′ / b = ω t Eric Price Sparse Fourier Transforms 20 / 36

Algorithm Lemma For most t, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Time-shift x by one and repeat: b ′ = � x t ω t . Divide to get b ′ / b = ω t = ⇒ can compute t . Eric Price Sparse Fourier Transforms 20 / 36

Algorithm Lemma For most t, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Time-shift x by one and repeat: b ′ = � x t ω t . Divide to get b ′ / b = ω t = ⇒ can compute t . ◮ Just like our 1-sparse recovery algorithm, x 1 / x 0 = ω t . Eric Price Sparse Fourier Transforms 20 / 36

Algorithm Lemma For most t, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Time-shift x by one and repeat: b ′ = � x t ω t . Divide to get b ′ / b = ω t = ⇒ can compute t . ◮ Just like our 1-sparse recovery algorithm, x 1 / x 0 = ω t . x ′ such that � x ′ is k / 2-sparse. Gives partial sparse recovery: � x − � 1-sparse recovery 1-sparse recovery x ′ x � Permute Filters O ( k ) 1-sparse recovery 1-sparse recovery Eric Price Sparse Fourier Transforms 20 / 36

Algorithm Lemma For most t, the value b we compute for its bucket satisfies b = � x t . Computing the b for all O ( k ) buckets takes O ( k log n ) time. Time-shift x by one and repeat: b ′ = � x t ω t . Divide to get b ′ / b = ω t = ⇒ can compute t . ◮ Just like our 1-sparse recovery algorithm, x 1 / x 0 = ω t . x ′ such that � x ′ is k / 2-sparse. Gives partial sparse recovery: � x − � 1-sparse recovery 1-sparse recovery x ′ x � Permute Filters O ( k ) 1-sparse recovery 1-sparse recovery Repeat k → k / 2 → k / 4 → · · · Eric Price Sparse Fourier Transforms 20 / 36