Fast algorithms for nonconvex compressive sensing Rick Chartrand Los Alamos National Laboratory New Mexico Consortium September 2, 2009 Slide 1 of 23 Operated by Los Alamos National Security, LLC for NNSA
Outline Motivating Example Nonconvex compressive sensing Examples Fast algorithm Summary Slide 2 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example Motivating example Suppose we want to reconstruct an Consider radial sampling, image from samples of its Fourier such as in MRI or (roughly) transform. How many samples do CT. we need? Shepp-Logan phantom, x Ω Slide 3 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example Nonconvexity is better Fewer measurements are needed with nonconvex minimization: � Du � p min p , subject to ( F u ) | Ω = ( F x ) | Ω . u Slide 4 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example Nonconvexity is better Fewer measurements are needed with nonconvex minimization: � Du � p min p , subject to ( F u ) | Ω = ( F x ) | Ω . u With p = 1 , solution is u = x with 18 lines ( | Ω | | x | = 6 . 9% ). p = 1 , 18 lines backprojection, 18 lines Slide 4 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example Nonconvexity is better Fewer measurements are needed with nonconvex minimization: � Du � p min p , subject to ( F u ) | Ω = ( F x ) | Ω . u With p = 1 , solution is u = x with 18 lines ( | Ω | | x | = 6 . 9% ). With p = 1 / 2 , 10 lines suffice ( | Ω | | x | = 3 . 8% ). (More than 10 4500 local minima.) p = 1 p = 1 , 18 lines 2 , 10 lines p = 1 , 10 lines backprojection, 18 lines Slide 4 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example New results These are old results (Mar. 2006); what’s new? Slide 5 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example New results These are old results (Mar. 2006); what’s new? ◮ Reconstruction (to 50 dB) in 13 seconds (in Matlab; versus literature-best 1–3 minutes). 10 lines fastest 10-line re- covery Slide 5 of 23 Operated by Los Alamos National Security, LLC for NNSA
Motivating Example New results These are old results (Mar. 2006); what’s new? ◮ Reconstruction (to 50 dB) in 13 seconds (in Matlab; versus literature-best 1–3 minutes). ◮ Exact reconstruction from 9 lines (3.5% of Fourier transform). 10 lines fastest 10-line re- 9 lines recovery from covery fewest samples Slide 5 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing Outline Motivating Example Nonconvex compressive sensing Examples Fast algorithm Summary Slide 6 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing What is compressive sensing? = = , . A b Ψ x ′ x x ◮ Compressive sensing is the reconstruction of sparse signals x from surprisingly few incoherent measurements b = Ax . Slide 7 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing What is compressive sensing? = = , . A b Ψ x ′ x x ◮ Compressive sensing is the reconstruction of sparse signals x from surprisingly few incoherent measurements b = Ax . ◮ We suppose the existence of an operator or dictionary Ψ such that most of the components of Ψ x are (nearly) zero. Slide 7 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing What is compressive sensing? = = , . A b Ψ x ′ x x ◮ An undersampled measurement Ax is tantamount to a compressed version of x . If x is sufficiently sparse, it can be recovered perfectly. Slide 7 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing What is compressive sensing? = = , . A b Ψ x ′ x x ◮ An undersampled measurement Ax is tantamount to a compressed version of x . If x is sufficiently sparse, it can be recovered perfectly. ◮ We exploit the fact that sparsity is mathematically special, yet a general property of natural or human signals. Slide 7 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing Optimization for sparse recovery ◮ Let x ∈ R N be sparse: � Ψ x � 0 = K , K ≪ N . Slide 8 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing Optimization for sparse recovery ◮ Let x ∈ R N be sparse: � Ψ x � 0 = K , K ≪ N . ◮ Suppose A is an M × N matrix, M ≪ N , with A and Ψ incoherent . For example, A = ( a ij ) , i.i.d. a ij ∼ N (0 , σ 2 ) . Let b = Ax . Slide 8 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing Optimization for sparse recovery ◮ Let x ∈ R N be sparse: � Ψ x � 0 = K , K ≪ N . ◮ Suppose A is an M × N matrix, M ≪ N , with A and Ψ incoherent . For example, A = ( a ij ) , i.i.d. a ij ∼ N (0 , σ 2 ) . Let b = Ax . Unique solution is u = x with opti- mally small M , but is NP-hard. min � Ψ u � 0 , s.t. Au = b. u M ≥ 2 K suffices with probability 1. Slide 8 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing Optimization for sparse recovery ◮ Let x ∈ R N be sparse: � Ψ x � 0 = K , K ≪ N . ◮ Suppose A is an M × N matrix, M ≪ N , with A and Ψ incoherent . For example, A = ( a ij ) , i.i.d. a ij ∼ N (0 , σ 2 ) . Let b = Ax . Unique solution is u = x with opti- mally small M , but is NP-hard. min � Ψ u � 0 , s.t. Au = b. u M ≥ 2 K suffices with probability 1. Can be solved efficiently; requires more measurements for reconstruc- min � Ψ u � 1 , s.t. Au = b. u tion. M ≥ CK log( N/K ) Slide 8 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing Optimization for sparse recovery ◮ Let x ∈ R N be sparse: � Ψ x � 0 = K , K ≪ N . ◮ Suppose A is an M × N matrix, M ≪ N , with A and Ψ incoherent . For example, A = ( a ij ) , i.i.d. a ij ∼ N (0 , σ 2 ) . Let b = Ax . Unique solution is u = x with opti- mally small M , but is NP-hard. min � Ψ u � 0 , s.t. Au = b. u M ≥ 2 K suffices with probability 1. Can be solved efficiently; requires more measurements for reconstruc- min � Ψ u � 1 , s.t. Au = b. u tion. M ≥ CK log( N/K ) where 0 < p < 1 . Solvable in prac- tice; requires fewer measurements � Ψ u � p min p , s.t. Au = b, u than ℓ 1 . M ≥ C 1 ( p ) K + pC 2 ( p ) K log( N/K ) (with V. Staneva) Slide 8 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 2 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 1 p Au = b Slide 9 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 2 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 2 p Au = b Slide 9 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 2 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 3 p Au = b Slide 9 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 2 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 4 p Au = b Slide 9 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 2 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 5 p Au = b Slide 9 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 1 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 2 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 3 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 4 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 5 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 6 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Nonconvex compressive sensing The geometry of ℓ p min u � u � p p , subject to Au = b p = 1 : x | u 1 | p + | u 2 | p + | u 3 | p = 0 . 7 p Au = b Slide 10 of 23 Operated by Los Alamos National Security, LLC for NNSA
Recommend
More recommend
