Sparsity-inducing dual frames and applications to compressed sensing - PowerPoint PPT Presentation

Sparsity-inducing dual frames and applications to compressed sensing with frames Shidong Li San Francisco State University Jointly with Tiebin Mi , Remin University of China CIMPA 2013 New Trends in Applied Harmonic Analysis Sparse Representations, Compressed Sensing and Multifractal Analysis August 5-16, 2013 Mar del Plata, ARGENTINA

Outline The notion of sparsity-inducing dual frames 1 Sparse duals applied to compressed sensing with frames 2 A performance analysis of the ℓ 1 -synthesis method Focus: The sparse-dual-based ℓ 1-analysis approach The “decoupling” effect of sparse-dual-based-analysis approach 3

Outline The notion of sparsity-inducing dual frames 1 Sparse duals applied to compressed sensing with frames 2 A performance analysis of the ℓ 1 -synthesis method Focus: The sparse-dual-based ℓ 1-analysis approach The “decoupling” effect of sparse-dual-based-analysis approach 3

The notion of sparsity-inducing dual frames Question : Suppose f has a sparse frame representation f = Dx , where x is sparse, is there a dual frame D such that D ∗ f = x ? Proposition/Definition (The existence of the sparse dual) Suppose D is a non-exact frame. Suppose f has a sparse representation with respect to D , namely, f = Dx . Then there exits a sparsity-inducing dual frame D of D such that D D ∗ = I and D ∗ f = x .

All dual frame formula Theorem (The dual frame formula) [Li’95] For every given finite non-exact frame D , the set of all dual frames ˜ D of D is given by � � D = ( DD ∗ ) − 1 D + W ˜ I − D ∗ ( DD ∗ ) − 1 D , (1) where W can be an arbitrary matrix (of the right dimensions). Conclusion : sparsity-inducing dual frames always exist!

Basic Properties of Sparse Duals There are many choices of such sparse duals. Sparse duals are signal dependent, but there are sparse duals for classes signals. Sparse duals are stable! Theorem (Stability of D ) Let D ∈ R d × n . Let T f ⊂ { 1 , 2 , · · · , n } be the sparsity index set of f such that | T f | = s and �D ∗ f − ( D ∗ f ) T f � 1 ≤ ε . Suppose � f − g � 2 ≤ δ , and let T g be the sparsity index set for g with | T g | = s . Then √ √ �D ∗ g − ( D ∗ g ) T g � 1 ≤ 2 ε + n − s ( B + s ) δ. Here B is the upper frame bound of the sparse dual frame D . Motivation of the notion of sparse duals : in the study of compressed sensing when f has a coherent sparse frame representation.

Outline The notion of sparsity-inducing dual frames 1 Sparse duals applied to compressed sensing with frames 2 A performance analysis of the ℓ 1 -synthesis method Focus: The sparse-dual-based ℓ 1-analysis approach The “decoupling” effect of sparse-dual-based-analysis approach 3

Compressed sensing: when f has a sparse representation CS observation model: The linear measurement of f by A ∈ R m × d : y = Af + z , f = Dx . given The task of such CS problems: The task is to recover f from underdetermined measurement y . Example: Pulse Radar , where f is sparse w.r.t a frame D Received and de-modulated signal, up to some discrete approximation, is given by K y ( t ) = x k · g ( t − τ k ) · exp ( j ω k t ) . � k = 1 Here, the observation y is a frame expansion, and is naturally sparse w.r.t. the Gabor frame { g ( t − τ j ) · exp ( j ω k t ) } jk .

Two classes of signal recovery methods Major approach 1: The ℓ 1 -synthesis approach Let x be sparse. Determine x via x = argmin x ∈ R d � x � 1 s.t. � y − ADx � 2 ≤ ε, ˜ (2) Then find f via synthesis operation, i.e., ˜ f = D ˜ x . Major approach 2: The ℓ 1 -analysis approach The analysis approach is to recover f directly: f = argmin f ∈ R n � D ∗ f � 1 s . t . � y − Af � 2 ≤ ε. ˜ (3) Here D ∗ is the “analysis” operator.

Issues with the ℓ 1 -synthesis approach Largely for the lack of good understanding of its performance. Available analysis are (largely) based on the accurate recovery of the coeff. x by, e.g., the RIP condition ∀ x ∈ { x : � x � 0 ≤ s } , ( 1 − γ s ) � x � 2 2 ≤ � Ax � 2 2 ≤ ( 1 + γ s ) � x � 2 2 . When D is coherent, AD hardly satisfies the RIP [Rauhut et al.,2008] , ⇒ recovering x is not guaranteed. However, the synthesis approach often recovers f fine, even though coeff x may not be recovered exactly. Hence RIP-based analysis would not be able to explain it. Still have limitations in recovering signals that are NOT TOO sparse.

Example where ℓ 1 -synthesis fails D is a Gabor frame as seen in Pulse-Dopplar Radar applications. L1 synthesis 3 4 Original signal L1 synthesis 2.5 3 2 Coefficients by L1−synthesis 2 1.5 1 1 0 0.5 −1 0 −2 −0.5 −1 −3 −1 −0.5 0 0.5 1 1.5 2 2.5 3 20 40 60 80 100 120 Real coefficients (a) ℓ 1 synthesis (b) ℓ 1 synthesis We’ll come back to address the performance understanding issue + the recoverability issue.

Major approach 2: The ℓ 1 -analysis approach The ℓ 1 -analysis approach The analysis approach is to recover f directly: ˜ f = argmin f ∈ R n � D ∗ f � 1 s . t . � y − Af � 2 ≤ ε. (4) Here D is assumed a Parseval frame, and D ∗ is the “analysis” operator. The idea is to focus on recovering f , and not to care for the exact coefficient x (since coefficients are not unique any way).

An inspiring error bound for ℓ 1 -analysis Theorem [Candès etc.’2011] Let f be the original signal, and f # be the solution to the ℓ 1 -analysis approach (4). Then � f # − f � 2 ≤ C 0 · ε + C 1 · � D ∗ f − ( D ∗ f ) s � 1 √ s , (5) provided that A obeys a D -RIP condition with δ 2 s < 0 . 08. The D -RIP condition [Candès etc. 2011] The D-RIP constant δ s of A is the smallest δ s > 0 satisfying ( 1 − δ s ) � f � 2 2 ≤ � Af � 2 2 ≤ ( 1 + δ s ) � f � 2 (6) 2 for all f ∈ Σ s , the union of all subspaces spanned by all subsets of s columns of D .

An unforgiven reality of the conventional ℓ 1 -analysis approach The error bound is heavily depend on the decay property of � D ∗ f − ( D ∗ f ) s � 1 . Recall � f # − f � 2 ≤ C 0 · ε + C 1 · � D ∗ f − ( D ∗ f ) s � 1 √ s . (7) In particular, if D is a Parseval frame, = ⇒ D ∗ f often has the worst sparsity property! ⇒ The choice of the “analysis” operator D ∗ is problematic in = general!

Example where ℓ 1 -analysis fails D is a Gabor frame as seen in radar applications, same example as seen in the ℓ 1 -synthesis. L1 analysis 3 4 Original signal L1 analysis 2.5 3 Signal reconstructed by L1−analysis 2 2 1.5 1 1 0 0.5 −1 0 −2 −0.5 −1 −3 −1 0 1 2 3 4 20 40 60 80 100 120 Real signal (c) ℓ 1 analysis (d) ℓ 1 analysis

What are good choices of analysis operators? Exploring Optimality: Alternative dual-based analysis method Use non-canonical dual frame ˜ D as the analysis operator � ˜ D ∗ f � 1 � Af − y � 2 ≤ ε min s.t. (8) f One good property: For any dual frame ˜ D , the error bound similar to the conventional ℓ 1 analysis holds

Theorem [Liu, Li, Mi 2011] Let D be a frame of R n with frame bounds 0 < A ≤ B < ∞ . Let ˜ D be an alternative dual frame of D with frame bounds 0 < ˜ A ≤ ˜ B < ∞ . Suppose � 2 � � � ρ B ˜ B · δ s + a + ρ B ˜ B · δ b < 1 − 2 ρ B ˜ B 1 − (9) holds for some positive integers a and b satisfying 0 < b − a ≤ 3 a . Then the solution ¯ f to (8) satisfies D ∗ f − (˜ D ∗ f ) s � 1 f − f � 2 ≤ C 0 · ε + C 1 · � ˜ � ¯ √ s , (10) where ρ ≡ s / b .

An optimal-dual-based analysis operators? An optimal dual-based analysis model Consider a “blind” optimization D ∗ = I , f ∈ R n � ˜ D ∗ f � 1 s.t. � y − A f � 2 ≤ ε, min (11) D ˜ D of D where the optimization is over both the class of all dual frames ˜ and the feasible signal set f . This has the potential to give rise to better results, because the D o is chosen to minimize the � ˜ D ∗ f � 1 among all dual optimal dual ˜ frames ˜ D . It is not hard to see that this optimal-dual-based analysis problem is equivalent to the synthesis procedure: x ∈ R d � x � 1 s.t. � y − ADx � 2 ≤ ε, min (12) D ∗ = I , f ∈ R n � ˜ D ∗ f � 1 s.t. � y − Af � 2 ≤ ε, min (13) D ˜

A deeper understanding of the synthesis approach Consequently, there is a better performance analysis of the synthesis procedure! Theorem [Li, Liu, Mi 2012] Let ˜ D o be the optimal dual of D in the sense of (13). Then under some D-RIP condition of A , the solution ¯ f to the ℓ 1 -synthesis approach satisfies D ∗ o f − (˜ D ∗ o f ) s � 1 f − f � 2 ≤ C 0 · ε + C 1 · � ˜ � ¯ √ s . (14) This provides a deeper understanding of the synthesis method: fine approximations of f . But x is off by a large margin. Here the error bound does not depend on the accuracy of the recovered coefficients x , but the decaying property of | ˜ D ∗ o f | . The old RIP analysis results will NOT be able to explain these fine approximations, b/c RIP places the focus on the exact recovery of coeff. x .