KSVD - Gradient Descent Method For Compressive Sensing Optimization - PowerPoint PPT Presentation

KSVD - Gradient Descent Method For Compressive Sensing Optimization Endra Department of Computer Engineering Faculty of Engineering Bina Nusantara University

INTRODUCTION INTRODUCTION

INTRODUCTION INTRODUCTION

WHAT IS COMPRESSIVE SENSING ? Candes, E.J., and Wakin, M.B., March. 2008, An Introduction to Compressive Sampling, IEEE Signal Processing Magazine., pp. 21-30.

WHAT IS COMPRESSIVE SENSING ? When Sensing Meet Compression Automatically translates analog data into already compressed digital form.

Applications and Opportunities Of Compressive Sensing New Analog-to-Digital Converters (Analog to Information)

COMPRESSIVE SENSING COMPRESSIVE SENSING CS Theory Requires Three Aspects : 1.The desired signals/images are sparse/compressible. Need a suitable basis or sparse dictionary (Fourier, Wavelet, Overcomplete Dictionary)  Dictionary Learning (K-SVD (Singular Value Decomposition). 2. CS Matrix  Requires a small mutual coherence with dictionary. 3. Reconstruction algorithms  Matching / Basis Pursuit. In this paper, We used OMP (Orthogonal Matching Pursuit) & Iteratively Reweighted Least Squares (IRLS) – Ell-p – minimization.

COMPRESSIVE SENSING FRAMEWORK COMPRESSIVE SENSING FRAMEWORK  M  N   M   K 1 M 1 N K K 1 K  M 1 θ θ D S Sparse Equivalent Measurement Matrix Sparse Dictionary Coefficent M  Basis/Dictionary N   & Small Mutual Coherence Between K   N If      y x Complete x (Basis)      K   N If y D Over-Complete (Dictionary)

PREVIOUS WORKS PREVIOUS WORKS 1. 2006, David L. Donoho, Emmanuel J. Candès, Justin Romberg, and Terence Tao  First Papers in CS, Using Random Matrix for CS Measurement. 2. 2006, M. Aharon, M. Elad, and A. Bruckstein  Using KSVD for Designing Overcomplete Dictionaries for Sparse Representation. 3. 2007 , M. Elad  Optimized CS Measurement by Reducing t-Averaged Mutual Coherence. 4. 2010 , Vahid Abolghasemi, Saideh Ferdowsi, Bahador Makkiabadi and Saeid Sanei  Optimized CS Measurement by Using Gradient-Descent Method, Better than Elad’s Method. In This Paper We Combined KSVD & Gradient-Descent Methods to Perform the Joint optimization of Dictionary and CS Measurement Matrix.

OPTIMIZED MEASUREMENT MATRIX OPTIMIZED MEASUREMENT MATRIX Random Gaussian Matrix that fulfill the required property of CS measurement (Incoherency & RIP) usually to be used to encode the signal. can be optimized by reducing the mutual coherence :       T Equivalent Dictionary, D , D : max d d    i j , 1 i , j K i close to orthonormal Gram-Matrix of Equivalent Dictionary : 2 2   t  G  min G I min D D I I D D F F

Optimized Measurement Matrix Optimized Measurement Matrix Gradient Gradient-Descent Method Descent Method       2 T      T T T   E D D I Tr D D I D D I  F    E     T E 4 D D D I  d ij        D D k E  i 1 i       T           D D D D D I  i i 1 i i i     1 D OPTIMIZED CS MEASUREMENT MATRIX

Joint Optimization of Dictionary and CS Measurement Matrix   Y X X Is Training Patches      2    2      min X Y s.t. i , S    , , i  F F 0   Get by Using KSVD Optimize by Using Gradient Descent Method 2       X I          min s.t. i , S         , , i      Y 0 F   W Z    eq eq D : W d ... d eq 1 K 2      min Z D s.t. i , S    , , eq i  F 0 Joint KSVD - Gradient Descent Method

SIMULATION METHOD From Each of 30 Training-Images (481 x 321) Was Taken Randomly 200 - 8 x 8 patches  6000 patches. These 6000 patches Were Used as Training Patches for Joint KSVD - Gradient Descent Method.

SIMULATION METHOD Test Image (481 x 321)

RESULTS The PSNR comparison of reconstructed image from compressive sensing by using OMP for : KSVD - Random, Uncoupled KSVD- Gradient Descent and Joint KSVD-Gradient Descent.

RESULTS The PSNR comparison of reconstructed image from compressive sensing by using (IRLS) – Ell-p - Minimization for : KSVD - Random, Uncoupled KSVD-Gradient Descent and Joint KSVD- Gradient Descent. .

RESULTS KSVD- KSVD- Random Random IRLS-ell-p OMP Uncoupled KSVD- Uncoupled Gradient KSVD- Descent Gradient Descent

RESULTS Joint KSVD- Gradient Descent OMP IRLS-ell-p The comparison of reconstructed image for m = 15 by using OMP (left column) and IRLS – ell-p - minimization (right column) where : (a) & (d) KSVD-Random, (b) & (e) Uncoupled KSVD- Gradient Descent , (c) & (f) Joint KSVD- Gradient Descent.

CONCLUSION From the results, it showed that by optimizing measurement matrix and dictionary learning simultaneously provided the improvement of the image reconstruction from compressive sensing. Further improvement can be attempted in future work by optimizing measurement matrix and dictionary learning simultaneously based on block-sparse representations.

REFERENCES [32] Petros Boufounos, Justin Romberg and Richard Baraniuk, “Compressive Sensing : Theory and Applications,” IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Las Vegas, Nevada, Apr. 2008 [Online]. Available: http://www.ece.rice.edu/~richb/talks/cs-tutorial- ICASSP-mar08.pdf. [1] – [31] [33] Jianwei Ma., “Data Recovery from Compressed Measurement”, School of Aerospace, Tsinghua University, Beijing. [34] E. Candès, Electrical Engineering Colloquium, University of Washington, December 2010. [35] Michael Elad, Optimized Projection Directions for Compressed Sensing, The IV Workshop on SIP & IT Holon Institute of Technology June 20th, 2007. [36] Michael Elad, Sparse & Redundant Representation Modeling of Images, Summer School on Sparsity in Image and Signal Analysis, Holar, Iceland, August 15 – 20 , 2010.