Quantized Corrupted Sensing with Random Dithering Zhongxing Sun Beijing Institute of Technology Joint work with Wei Cui and Yulong Liu ISIT 2020 Zhongxing Sun Beijing Institute of Technology ISIT 2020 1 / 22
Overview Introduction: From Corrupted Sensing to Quantized Corrupted Sensing Recovery Procedure Performance Guarantees Concrete Examples Numerical Simulations Summary Zhongxing Sun Beijing Institute of Technology ISIT 2020 2 / 22
Corrupted Sensing: recover a structured signal from corrupted measurements Corrupted sensing 1 : y = Φ x ⋆ + √ m v ⋆ + n , (1) where: Φ ∈ R m × n : the sensing matrix with m << n ; x ⋆ ∈ R n : the unknown structured signal; v ⋆ ∈ R m : the unknown structured corruption; n ∈ R m : the random unstructured noise. 1 The factor √ m in (1) makes the columns of Φ and √ m I m have the same scale, which helps the theoretical results to be more interpretable. Zhongxing Sun Beijing Institute of Technology ISIT 2020 3 / 22
Corrupted Sensing: recover a structured signal from corrupted measurements Corrupted sensing 1 : y = Φ x ⋆ + √ m v ⋆ + n , (1) where: Φ ∈ R m × n : the sensing matrix with m << n ; x ⋆ ∈ R n : the unknown structured signal; v ⋆ ∈ R m : the unknown structured corruption; n ∈ R m : the random unstructured noise. Goal To estimate x ⋆ and v ⋆ given y and Φ . (See e.g. [FM14],[MT14],[CL19]) 1 The factor √ m in (1) makes the columns of Φ and √ m I m have the same scale, which helps the theoretical results to be more interpretable. Zhongxing Sun Beijing Institute of Technology ISIT 2020 3 / 22
Examples of Corrupted Sensing Signal model: y = Φ x ⋆ + √ m v ⋆ + n . Corrupted sensing model has seen a lot of interest in modern data-intensive science. Zhongxing Sun Beijing Institute of Technology ISIT 2020 4 / 22
Examples of Corrupted Sensing Signal model: y = Φ x ⋆ + √ m v ⋆ + n . Corrupted sensing model has seen a lot of interest in modern data-intensive science. Examples of applications include: face recognition [WYGSM09]; subspace clustering [EV09]; video background subtraction [CLMW11]. ... Zhongxing Sun Beijing Institute of Technology ISIT 2020 4 / 22
Examples of Corrupted Sensing Signal model: y = Φ x ⋆ + √ m v ⋆ + n . Corrupted sensing model has seen a lot of interest in modern data-intensive science. Examples of applications include: face recognition [WYGSM09]; subspace clustering [EV09]; video background subtraction [CLMW11]. ... Figure: Examples from [WYGSM09]. Zhongxing Sun Beijing Institute of Technology ISIT 2020 4 / 22
Recovery Procedures and Performance Guarantees Signal model: y = Φ x ⋆ + √ m v ⋆ + n . Zhongxing Sun Beijing Institute of Technology ISIT 2020 5 / 22
Recovery Procedures and Performance Guarantees Signal model: y = Φ x ⋆ + √ m v ⋆ + n . Table: Theoretical Analyses of Corrupted Sensing Measurement Matrix Recovery Proce- Noise Paper dure Random Orthogonal Constrained δ = 0 [MT14] Constrained Gaussian Bounded [FM14] Partially Penalized Constrained Bounded Sub-Gaussian Partially Penalized [CL19] Sub-Gaussian Fully Penalized Zhongxing Sun Beijing Institute of Technology ISIT 2020 5 / 22
Quantized Corrupted Sensing In many practical applications, one would like to quantize or digitize the measurements into bitstreams: y = Q ( Φ x ⋆ + √ m v ⋆ + n ) , where Q ( · ) stands for some quantization scheme. Zhongxing Sun Beijing Institute of Technology ISIT 2020 6 / 22
Quantized Corrupted Sensing In many practical applications, one would like to quantize or digitize the measurements into bitstreams: y = Q ( Φ x ⋆ + √ m v ⋆ + n ) , where Q ( · ) stands for some quantization scheme. Question: Can we recover a structured signal from the quantized corrupted measurements y = Q ( Φ x ⋆ + √ m v ⋆ + n ) ? Furthermore, what role does quantization play in the reconstruction error? Zhongxing Sun Beijing Institute of Technology ISIT 2020 6 / 22
Quantized Corrupted Sensing Signal model: y = Q ( Φ x ⋆ + √ m v ⋆ + n ) . (2) Good news: If v ⋆ = 0 , model (2) reduces to Quantized Compressed Sensing (QCS). Some theoretical results exist: 1-bit measurements [BB08]; uniform quantization [XJ19], [TR20]; general non-linear function [PV16], [TAH15]. Zhongxing Sun Beijing Institute of Technology ISIT 2020 7 / 22
Quantized Corrupted Sensing Signal model: y = Q ( Φ x ⋆ + √ m v ⋆ + n ) . (2) Good news: If v ⋆ = 0 , model (2) reduces to Quantized Compressed Sensing (QCS). Some theoretical results exist: 1-bit measurements [BB08]; uniform quantization [XJ19], [TR20]; general non-linear function [PV16], [TAH15]. Bad news: If v ⋆ � = 0 , the existing theoretical analyses for QCS cannot be directly generalized to corrupted sensing model. Zhongxing Sun Beijing Institute of Technology ISIT 2020 7 / 22
A Failed Example 1.1 hyperbolic tangent hyperbolic tangent uniform quantizer uniform quantizer sign sign theoretical scaling theoretical scaling Recover error (log) Recover error (log) 1.05 10 -1 1 0.95 0.9 100 150 200 250 300 350 400 450500 100 200 300 400 500 Number of measurements (log) Number of measurements (log) (a) Corruption v ⋆ = 0 (b) Corruption v ⋆ � = 0 Figure: Log-log error curves for different nonlinear quantization schemes. Zhongxing Sun Beijing Institute of Technology ISIT 2020 8 / 22
A Failed Example 1.1 hyperbolic tangent hyperbolic tangent uniform quantizer uniform quantizer sign sign theoretical scaling theoretical scaling Recover error (log) Recover error (log) 1.05 10 -1 1 0.95 0.9 100 150 200 250 300 350 400 450500 100 200 300 400 500 Number of measurements (log) Number of measurements (log) (a) Corruption v ⋆ = 0 (b) Corruption v ⋆ � = 0 Figure: Log-log error curves for different nonlinear quantization schemes. Solution: to consider some specific but more tractable nonlinear quantization schemes. Zhongxing Sun Beijing Institute of Technology ISIT 2020 8 / 22
Our Strategy: Benefit from Noise In this paper, we consider introducing a random noise before quantization (also known as dithering): y = Φ x ⋆ + √ m v ⋆ + n , ¯ Φ x ⋆ + √ m v ⋆ + n + τ (3) � � y = Q U (¯ y + τ ) = Q U , where Q U ( x ) = ∆( ⌊ x ∆ ⌋ + 1 2 ) is the uniform scalar quantizer with resolution ∆ > 0, and τ i ∼ Unif( − ∆ 2 , ∆ 2 ] is the random uniform dithering. Zhongxing Sun Beijing Institute of Technology ISIT 2020 9 / 22
Our Strategy: Benefit from Noise In this paper, we consider introducing a random noise before quantization (also known as dithering): y = Φ x ⋆ + √ m v ⋆ + n , ¯ Φ x ⋆ + √ m v ⋆ + n + τ (3) � � y = Q U (¯ y + τ ) = Q U , where Q U ( x ) = ∆( ⌊ x ∆ ⌋ + 1 2 ) is the uniform scalar quantizer with resolution ∆ > 0, and τ i ∼ Unif( − ∆ 2 , ∆ 2 ] is the random uniform dithering. Goal To disentangle signal x ⋆ and corruption v ⋆ given Φ and the quantized samples { y i } m i =1 . Zhongxing Sun Beijing Institute of Technology ISIT 2020 9 / 22
Benefit from Noise: Linearization with Independent Noise Key Observation [Sch65],[GS93] Uniform dither can substantially result in independent distributed quantization error: z := Q U (¯ y + τ ) − (¯ y + τ ) is independent from ¯ y and τ . Zhongxing Sun Beijing Institute of Technology ISIT 2020 10 / 22
Benefit from Noise: Linearization with Independent Noise Key Observation [Sch65],[GS93] Uniform dither can substantially result in independent distributed quantization error: z := Q U (¯ y + τ ) − (¯ y + τ ) is independent from ¯ y and τ . Then, the problem can be reformulated as y + τ ) = Φ x ⋆ + √ m v ⋆ + τ + z + n . y = Q U (¯ Zhongxing Sun Beijing Institute of Technology ISIT 2020 10 / 22
Benefit from Noise: Linearization with Independent Noise Key Observation [Sch65],[GS93] Uniform dither can substantially result in independent distributed quantization error: z := Q U (¯ y + τ ) − (¯ y + τ ) is independent from ¯ y and τ . Then, the problem can be reformulated as y + τ ) = Φ x ⋆ + √ m v ⋆ + τ + z + n . y = Q U (¯ It’s natural to use the generalized Lasso: x , v � y − Φ x − √ m v � 2 , s.t. � x � sig ≤ � x ⋆ � sig min (4) � v � cor ≤ � v ⋆ � cor . e.g., � · � sig : sparse signals → ℓ 1 -norm; low-rank matrices → nuclear norm. Zhongxing Sun Beijing Institute of Technology ISIT 2020 10 / 22
Combing Existing Results from Corrupted Sensing Under the linearized model: Φ , √ m I � � x ⋆ � � y = + τ + z + n . v ⋆ Zhongxing Sun Beijing Institute of Technology ISIT 2020 11 / 22
Combing Existing Results from Corrupted Sensing Under the linearized model: Φ , √ m I � � x ⋆ � � y = + τ + z + n . v ⋆ Extended matrix deviation inequality [CL19] implies a tight lower bound for the restricted singular value of the extended sensing matrix [ Φ , √ m I m ]: ( a , b ) ∈T ∩ S n + m − 1 � Φ a + √ m b � 2 ≥ √ m − C · γ ( T ∩ S n + m − 1 ) inf holds with probability at least 1 − exp {− γ ( T ∩ S n + m − 1 ) 2 } . Zhongxing Sun Beijing Institute of Technology ISIT 2020 11 / 22
Recommend
More recommend
