Lecture 1: Applied Harmonic Analysis and Compressed Sensing Gitta Kutyniok (Technische Universit¨ at Berlin) Winter School on “Compressed Sensing”, TU Berlin December 3–5, 2015 Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 1 / 50
Outline The Geometric Separation Problem 1 Inspiring Empirical Results Goal for Today Separation via Compressed Sensing 2 Sparsity and Underdetermined Systems Avalanche of Recent Work Separation of Points and Curves 3 Wavelet and Shearlet Systems Asymptotic Separation Result General Separation Estimate Microlocal Analysis Heuristics Conclusions 4 Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 2 / 50
General Challenge in Data Analysis Modern Data in general is often composed of two or more morphologically distinct constituents, and we face the task of separating those components given the composed data. Examples include... Audio data: Sinusoids and peaks. Imaging data: Cartoon and texture. High-dimensional data: Lower-dimensional structures of different dimensions. Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 3 / 50
Separating Artifacts in Images, I + + (Source: J. L. Starck, M. Elad, D. L. Donoho; 2005 (Artificial Data)) Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 4 / 50
Separating Artifacts in Images, II + + + (Source: J. L. Starck, M. Elad, D. L. Donoho; 2005) Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 5 / 50
Problem from Neurobiology Alzheimer Research: Detection of characteristics of Alzheimer. Separation of spines and dendrites. (Confocal-Laser Scanning-Microscopy) Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 6 / 50
Numerical Result + (Source: Brandt, K, Lim, S¨ undermann; 2010) Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 7 / 50
Goal for Today Neurobiological Data: Observed signal x = x 1 + x 2 . x 1 = Point structures. x 2 = Curvilinear structures. Challenges for Today: Mathematical methodology to derive the empirical results! Fundamental mathematical concept behind the empirical results! Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 8 / 50
Applied Harmonic Analysis Approach to Imaging Science Exploit a carefully designed representation system ( ψ λ ) λ ∈ Λ ⊆ L 2 ( R 2 ): L 2 ( R 2 ) ∋ f − � → ( � f , ψ λ � ) λ ∈ Λ − → � f , ψ λ � ψ λ = f . λ ∈ Λ Desiderata: Special features encoded in the “large” coefficients | � f , ψ λ � | . Efficient representations: � f ≈ � f , ψ λ � ψ λ , #(Λ N ) small λ ∈ Λ N Methodology: Modification of the coefficients according to the task. Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 9 / 50
How does Compressed Sensing help with Component Separation? Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 10 / 50
‘Mathematical Model’ Model for 2 Components: Observe a signal x composed of two subsignals x 1 and x 2 : x = x 1 + x 2 . Extract the two subsignals x 1 and x 2 from x , if only x is known. Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 11 / 50
‘Mathematical Model’ Model for 2 Components: Observe a signal x composed of two subsignals x 1 and x 2 : x = x 1 + x 2 . Extract the two subsignals x 1 and x 2 from x , if only x is known. Isn’t this impossible? There are two unknowns for every datum. Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 11 / 50
‘Mathematical Model’ Model for 2 Components: Observe a signal x composed of two subsignals x 1 and x 2 : x = x 1 + x 2 . Extract the two subsignals x 1 and x 2 from x , if only x is known. Isn’t this impossible? There are two unknowns for every datum. But we have additional Information: The two components are geometrically different. Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 11 / 50
Birth of ℓ 1 -Component Separation (2001) Composition of Sinusoids and Spikes sampled at n points: � c 0 � x = x 0 1 + x 0 2 = Φ 1 c 0 1 + Φ 2 c 0 1 2 = [ Φ 1 | Φ 2 ] , c 0 2 where x , c 0 1 , and c 0 2 are n × 1. Φ 1 is the n × n -Fourier matrix ((Φ 1 ) t , k = e 2 π itk / n ). Φ 2 is the n × n -Identity matrix. 1 0.5 0 -0.5 -1 0 50 100 150 200 250 Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 12 / 50
Sparsity and ℓ 1 Assumption: Letting A be an n × N -matrix, n << N , the seeked solution c 0 of x = Ac 0 satisfies: i � = 0 } is ‘small’, i.e., c 0 is sparse. � c 0 � 0 = # { i : c 0 Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 13 / 50
Sparsity and ℓ 1 Assumption: Letting A be an n × N -matrix, n << N , the seeked solution c 0 of x = Ac 0 satisfies: i � = 0 } is ‘small’, i.e., c 0 is sparse. � c 0 � 0 = # { i : c 0 Ideal: Solve... ( P 0 ) min c � c � 0 subject to x = Ac Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 13 / 50
Sparsity and ℓ 1 Assumption: Letting A be an n × N -matrix, n << N , the seeked solution c 0 of x = Ac 0 satisfies: i � = 0 } is ‘small’, i.e., c 0 is sparse. � c 0 � 0 = # { i : c 0 Ideal: Solve... ( P 0 ) min c � c � 0 subject to x = Ac Basis Pursuit (Chen, Donoho, Saunders; 1998) ( P 1 ) min c � c � 1 subject to x = Ac Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 13 / 50
Sparsity and ℓ 1 Assumption: Letting A be an n × N -matrix, n << N , the seeked solution c 0 of x = Ac 0 satisfies: i � = 0 } is ‘small’, i.e., c 0 is sparse. � c 0 � 0 = # { i : c 0 Ideal: Solve... ( P 0 ) min c � c � 0 subject to x = Ac Basis Pursuit (Chen, Donoho, Saunders; 1998) ( P 1 ) min c � c � 1 subject to x = Ac Meta-Result: If the solution c 0 is sufficiently sparse, and A is sufficiently incoherent, then c 0 can be recovered from x via ( P 1 ). Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 13 / 50
First Results of Compressed Sensing Composition of Sinusoids and Spikes sampled at n points: � c 0 � x = x 0 1 + x 0 2 = Φ 1 c 0 1 + Φ 2 c 0 1 2 = [ Φ 1 | Φ 2 ] . c 0 2 Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 14 / 50
First Results of Compressed Sensing Composition of Sinusoids and Spikes sampled at n points: � c 0 � x = x 0 1 + x 0 2 = Φ 1 c 0 1 + Φ 2 c 0 1 2 = [ Φ 1 | Φ 2 ] . c 0 2 Theorem (Donoho, Huo; 2001) 2 ) � 0 < (1 + √ n ) / 2, then If #(Sinusoids) + #(Spikes) = � ( c 0 1 ) � 0 + � ( c 0 ( c 0 1 , c 0 2 ) = argmin( � c 1 � 1 + � c 2 � 1 ) subject to x = Φ 1 c 1 + Φ 2 c 2 . Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 14 / 50
First Results of Compressed Sensing Composition of Sinusoids and Spikes sampled at n points: � c 0 � x = x 0 1 + x 0 2 = Φ 1 c 0 1 + Φ 2 c 0 1 2 = [ Φ 1 | Φ 2 ] . c 0 2 Theorem (Donoho, Huo; 2001) 2 ) � 0 < (1 + √ n ) / 2, then If #(Sinusoids) + #(Spikes) = � ( c 0 1 ) � 0 + � ( c 0 ( c 0 1 , c 0 2 ) = argmin( � c 1 � 1 + � c 2 � 1 ) subject to x = Φ 1 c 1 + Φ 2 c 2 . Theorem (Bruckstein, Elad; 2002)(Donoho, Elad; 2003) Let A = ( a i ) N i =1 be an n × N -matrix with normalized columns, n << N , and let c 0 satisfy � c 0 � 0 < 1 1 + µ ( A ) − 1 � � , 2 with coherence µ ( A ) = max i � = j |� a i , a j �| . Then c 0 = argmin � c � 1 subject to x = Ac . Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 14 / 50
Component Separation using Compressed Sensing Let x be a signal composed of two subsignals x 0 1 and x 0 2 : x = x 0 1 + x 0 2 . Desiderata for two orthonormal bases Φ 1 and Φ 2 : x 0 i = Φ i c 0 i with � c 0 i � 0 small, i = 1 , 2 � Sparsity! µ ([Φ 1 | Φ 2 ]) small � Morphological Difference! Solve ( c ∗ 1 , c ∗ 2 ) = argmin( � c 1 � 1 + � c 2 � 1 ) subject to x = Φ 1 c 1 + Φ 2 c 2 and derive the approximate components x 0 i ≈ x ∗ i = Φ i c ∗ i , i = 1 , 2 . Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 15 / 50
Two Paths Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 16 / 50
Avalanche of Recent Work Problem: Solve x = Ac 0 with A an n × N -matrix ( n < N ). Deterministic World: Mutual coherence of A = ( a k ) k . Bound � c 0 � 0 dependent on µ ( A ). Efficiently solve the problem x = Ac 0 . Contributors: Bruckstein, Cohen, Dahmen, DeVore, Donoho, Elad, Fuchs, Gribonval, Huo, K, Rauhut, Temlyakov, Tropp, ... Random World: Restricted isometry constants of a random A = ( a k ) k . Bound � c 0 � 0 by n / (2 log( N / n ))(1 + o (1)). Efficiently solve the problem x = Ac 0 with high probability. Contributors: Cand` es, Donoho, Fornasier, K, Krahmer, Rauhut, Romberg, Tanner, Tao, Tropp, Vershynin, Ward, ... Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 17 / 50
Novel Direction for Sparsity Geometric Clustering: x = Ac 0 with A an n × N -matrix ( n < N ). Nonzeros of c 0 often ◮ arise not in arbitrary patterns, ◮ but are rather highly structured. Interactions between columns of A in ill-posed problems ◮ is not arbitrary, ◮ but rather geometrically driven. Gitta Kutyniok (TU Berlin) Lecture 1 Winter School 2015 18 / 50
Recommend
More recommend
