adaptive sparse recovery
play

Adaptive Sparse Recovery Eric Price MIT 2012-04-26 Joint work - PowerPoint PPT Presentation

Apr 02, 2024 •367 likes •1.92k views

Adaptive Sparse Recovery Eric Price MIT 2012-04-26 Joint work with Piotr Indyk and David Woodruff, 2011-2012 Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 1 / 29 Outline Motivating Example 1 Eric Price (MIT) Adaptive Sparse

  1. General Compressive Sensing Want to observe n -dimensional vector x ◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network. Normally takes n observations to find. But we know some structure on the input: ◮ Genetics: most people don’t have the mutation. ◮ Images: mostly smooth with some edges. ◮ Traffic: Zipf distribution. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

  2. General Compressive Sensing Want to observe n -dimensional vector x ◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network. Normally takes n observations to find. But we know some structure on the input: ◮ Genetics: most people don’t have the mutation. ◮ Images: mostly smooth with some edges. ◮ Traffic: Zipf distribution. We use this structure to compress space (e.g. JPEG). Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

  3. General Compressive Sensing Want to observe n -dimensional vector x ◮ Which of n people have a genetic mutation. ◮ Image ◮ Traffic pattern of packets on a network. Normally takes n observations to find. But we know some structure on the input: ◮ Genetics: most people don’t have the mutation. ◮ Images: mostly smooth with some edges. ◮ Traffic: Zipf distribution. We use this structure to compress space (e.g. JPEG). Can we use structure to save on observations ? Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 8 / 29

  4. Cameras 5 megapixel camera takes 15 million byte-size observations. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

  5. Cameras 5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

  6. Cameras 5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

  7. Cameras 5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Cheap in visible light (silicon), very expensive in infrared. ◮ $30,000 for 256x256 IR sensor. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

  8. Cameras 5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Cheap in visible light (silicon), very expensive in infrared. ◮ $30,000 for 256x256 IR sensor. Use structure to take only a few million observations. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

  9. Cameras 5 megapixel camera takes 15 million byte-size observations. Compresses it (JPEG) down to a million bytes. Why do we need to bother with so many observations? [Donoho,Cand` es-Tao] Cheap in visible light (silicon), very expensive in infrared. ◮ $30,000 for 256x256 IR sensor. Use structure to take only a few million observations. ◮ What structure? Sparsity. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 9 / 29

  10. Sparsity A vector is k -sparse if k components are non-zero. Images are almost sparse in the wavelet basis: Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 10 / 29

  11. Sparsity A vector is k -sparse if k components are non-zero. Images are almost sparse in the wavelet basis: Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 10 / 29

  12. Sparsity A vector is k -sparse if k components are non-zero. Images are almost sparse in the wavelet basis: Same kind of structure as in genetic testing! Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 10 / 29

  13. Linear sketching/Compressive sensing Suppose an n -dimensional vector x is k -sparse in known basis. Given Ax , a set of m << n linear products. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

  14. Linear sketching/Compressive sensing Suppose an n -dimensional vector x is k -sparse in known basis. Given Ax , a set of m << n linear products. Why linear? Many applications: ◮ Genetic testing: mixing blood samples. ◮ Streaming updates: A ( x + ∆ ) = Ax + A ∆ . ◮ Camera optics: filter in front of lens. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

  15. Linear sketching/Compressive sensing Suppose an n -dimensional vector x is k -sparse in known basis. Given Ax , a set of m << n linear products. Why linear? Many applications: ◮ Genetic testing: mixing blood samples. ◮ Streaming updates: A ( x + ∆ ) = Ax + A ∆ . ◮ Camera optics: filter in front of lens. Then it is possible to recover x from Ax . ◮ Quickly ◮ Robustly: get close to x if x is close to k -sparse. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

  16. Linear sketching/Compressive sensing Suppose an n -dimensional vector x is k -sparse in known basis. Given Ax , a set of m << n linear products. Why linear? Many applications: ◮ Genetic testing: mixing blood samples. ◮ Streaming updates: A ( x + ∆ ) = Ax + A ∆ . ◮ Camera optics: filter in front of lens. Then it is possible to recover x from Ax . ◮ Quickly ◮ Robustly: get close to x if x is close to k -sparse. Note: impossible without using sparsity ( A is underdetermined). Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 11 / 29

  17. Standard Sparse Recovery Framework Specify distribution on m × n matrices A (independent of x ). Given linear sketch Ax , recover ˆ x . Satisfying the recovery guarantee: � ˆ x − x � 2 � ( 1 + ǫ ) min � x − x k � 2 k -sparse x k with probability 2 / 3. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

  18. Standard Sparse Recovery Framework Specify distribution on m × n matrices A (independent of x ). Given linear sketch Ax , recover ˆ x . Satisfying the recovery guarantee: � ˆ x − x � 2 � ( 1 + ǫ ) min � x − x k � 2 k -sparse x k with probability 2 / 3. Solvable with O ( 1 ǫ k log n k ) measurements [Cand` es-Romberg-Tao ’06, Gilbert-Li-Porat-Strauss ’10] Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

  19. Standard Sparse Recovery Framework Specify distribution on m × n matrices A (independent of x ). Given linear sketch Ax , recover ˆ x . Satisfying the recovery guarantee: � ˆ x − x � 2 � ( 1 + ǫ ) min � x − x k � 2 k -sparse x k with probability 2 / 3. Solvable with O ( 1 ǫ k log n k ) measurements [Cand` es-Romberg-Tao ’06, Gilbert-Li-Porat-Strauss ’10] Matching lower bound. [Do Ba-Indyk-P-Woodruff ’10, P-Woodruff ’11] Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

  20. Adaptive Sparse Recovery Framework For i = 1 . . . r : ◮ Choose matrix A i based on previous observations (possibly randomized). ◮ Observe A i x . ◮ Number of measurements m is total number of rows in all A i . ◮ Number of rounds is r . Given linear sketch Ax , recover ˆ x . Satisfying the recovery guarantee: � ˆ x − x � 2 � ( 1 + ǫ ) min � x − x k � 2 k -sparse x k with probability 2 / 3. Solvable with O ( 1 ǫ k log n k ) measurements [Cand` es-Romberg-Tao ’06, Gilbert-Li-Porat-Strauss ’10] Matching lower bound. [Do Ba-Indyk-P-Woodruff ’10, P-Woodruff ’11] Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 12 / 29

  21. Adaptive result in comparison to previous work Nonadaptive: Θ ( 1 ǫ k log n k ) . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

  22. Adaptive result in comparison to previous work Nonadaptive: Θ ( 1 ǫ k log n k ) . Adaptive: O ( k log n k ) with ǫ = o ( 1 ) ([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

  23. Adaptive result in comparison to previous work Nonadaptive: Θ ( 1 ǫ k log n k ) . Adaptive: O ( k log n k ) with ǫ = o ( 1 ) ([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O ( 1 ǫ k log log n k ) . [Indyk-P-Woodruff ’11] Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

  24. Adaptive result in comparison to previous work Nonadaptive: Θ ( 1 ǫ k log n k ) . Adaptive: O ( k log n k ) with ǫ = o ( 1 ) ([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O ( 1 ǫ k log log n k ) . [Indyk-P-Woodruff ’11] ◮ Using r = O ( log log n log ∗ k ) rounds. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

  25. Adaptive result in comparison to previous work Nonadaptive: Θ ( 1 ǫ k log n k ) . Adaptive: O ( k log n k ) with ǫ = o ( 1 ) ([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O ( 1 ǫ k log log n k ) . [Indyk-P-Woodruff ’11] ◮ Using r = O ( log log n log ∗ k ) rounds. Even when r = 2, can get O ( k log n + 1 ǫ k log ( k /ǫ )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

  26. Adaptive result in comparison to previous work Nonadaptive: Θ ( 1 ǫ k log n k ) . Adaptive: O ( k log n k ) with ǫ = o ( 1 ) ([Haupt-Baraniuk-Castro-Nowak ’09], in a slightly different setting) This talk: O ( 1 ǫ k log log n k ) . [Indyk-P-Woodruff ’11] ◮ Using r = O ( log log n log ∗ k ) rounds. Even when r = 2, can get O ( k log n + 1 ǫ k log ( k /ǫ )) ◮ Separating dependence on n and ǫ . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 13 / 29

  27. Applications of Adaptivity When does adaptivity make sense? Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  28. Applications of Adaptivity When does adaptivity make sense? Genetic testing: Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  29. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  30. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  31. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: ◮ Programmable pixels (mirrors or LCD display): Yes . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  32. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: ◮ Programmable pixels (mirrors or LCD display): Yes . ◮ Hardwired lens: No . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  33. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: ◮ Programmable pixels (mirrors or LCD display): Yes . ◮ Hardwired lens: No . Streaming algorithms: Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  34. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: ◮ Programmable pixels (mirrors or LCD display): Yes . ◮ Hardwired lens: No . Streaming algorithms: ◮ Adaptivity corresponds to multiple passes. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  35. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: ◮ Programmable pixels (mirrors or LCD display): Yes . ◮ Hardwired lens: No . Streaming algorithms: ◮ Adaptivity corresponds to multiple passes. ◮ Router finding most common connections: No . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  36. Applications of Adaptivity When does adaptivity make sense? Genetic testing: ◮ Yes , but multiple rounds can be costly. Cameras: ◮ Programmable pixels (mirrors or LCD display): Yes . ◮ Hardwired lens: No . Streaming algorithms: ◮ Adaptivity corresponds to multiple passes. ◮ Router finding most common connections: No . ◮ Mapreduce finding most frequent URLs: Yes . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 14 / 29

  37. Outline Motivating Example 1 Formal Introduction to Sparse Recovery/Compressive Sensing 2 Algorithm 3 Conclusion 4 Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 15 / 29

  38. Outline of Algorithm Theorem Adaptive 1 + ǫ -approximate k-sparse recovery is possible with O ( 1 ǫ k log log ( n / k )) measurements. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 16 / 29

  39. Outline of Algorithm Theorem Adaptive 1 + ǫ -approximate k-sparse recovery is possible with O ( 1 ǫ k log log ( n / k )) measurements. Lemma Adaptive O ( 1 ) -approximate 1 -sparse recovery is possible with O ( log log n ) measurements. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 16 / 29

  40. Outline of Algorithm Theorem Adaptive 1 + ǫ -approximate k-sparse recovery is possible with O ( 1 ǫ k log log ( n / k )) measurements. Lemma Adaptive O ( 1 ) -approximate 1 -sparse recovery is possible with O ( log log n ) measurements. Lemma implies theorem using standard tricks ([GLPS10]). Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 16 / 29

  41. 1-sparse recovery: non-adaptive lower bound Lemma Adaptive C-approximate 1 -sparse recovery is possible with O ( log log n ) measurements for some C = O ( 1 ) . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

  42. 1-sparse recovery: non-adaptive lower bound Lemma Adaptive C-approximate 1 -sparse recovery is possible with O ( log log n ) measurements for some C = O ( 1 ) . Non-adaptive lower bound: why is this hard? Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

  43. 1-sparse recovery: non-adaptive lower bound Lemma Adaptive C-approximate 1 -sparse recovery is possible with O ( log log n ) measurements for some C = O ( 1 ) . Non-adaptive lower bound: why is this hard? Hard case: x is random e i plus Gaussian noise w . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

  44. 1-sparse recovery: non-adaptive lower bound Lemma Adaptive C-approximate 1 -sparse recovery is possible with O ( log log n ) measurements for some C = O ( 1 ) . Non-adaptive lower bound: why is this hard? Hard case: x is random e i plus Gaussian noise w . Noise � w � 2 2 = Θ ( 1 ) so C -approximate recovery requires finding i . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

  45. 1-sparse recovery: non-adaptive lower bound Lemma Adaptive C-approximate 1 -sparse recovery is possible with O ( log log n ) measurements for some C = O ( 1 ) . Non-adaptive lower bound: why is this hard? Hard case: x is random e i plus Gaussian noise w . Noise � w � 2 2 = Θ ( 1 ) so C -approximate recovery requires finding i . Observations � v , x � = v i + � v , w � = v i + � v � 2 √ n z , for z ∼ N ( 0 , Θ ( 1 )) . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 17 / 29

  46. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  47. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  48. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  49. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  50. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  51. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  52. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Information capacity I ( i , � v , x � ) � 1 2 log ( 1 + SNR ) where SNR denotes the “signal-to-noise ratio,” SNR = E [ signal 2 ] E [ v 2 i ] � 2 / n = 1 E [ noise 2 ] � v � 2 Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  53. 1-sparse recovery: non-adaptive lower bound Observe � v , x � = v i + � v � 2 √ n z , where z ∼ N ( 0 , Θ ( 1 )) Information capacity I ( i , � v , x � ) � 1 2 log ( 1 + SNR ) where SNR denotes the “signal-to-noise ratio,” SNR = E [ signal 2 ] E [ v 2 i ] � 2 / n = 1 E [ noise 2 ] � v � 2 Finding i needs Ω ( log n ) non-adaptive measurements. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 18 / 29

  54. 1-sparse recovery: changes in adaptive setting Information capacity I ( i , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” � � E [ v 2 i ] SNR = Θ . � v � 2 2 / n Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

  55. 1-sparse recovery: changes in adaptive setting Information capacity I ( i , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” � � E [ v 2 i ] SNR = Θ . � v � 2 2 / n If i is independent of v , this is O ( 1 ) . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

  56. 1-sparse recovery: changes in adaptive setting Information capacity I ( i , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” � � E [ v 2 i ] SNR = Θ . � v � 2 2 / n If i is independent of v , this is O ( 1 ) . As we learn about i , we can increase E [ v 2 i ] for constant � v � 2 . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

  57. 1-sparse recovery: changes in adaptive setting Information capacity I ( i , � v , x � ) � 1 2 log ( 1 + SNR ) . where SNR denotes the “signal-to-noise ratio,” � � E [ v 2 i ] SNR = Θ . � v � 2 2 / n If i is independent of v , this is O ( 1 ) . As we learn about i , we can increase E [ v 2 i ] for constant � v � 2 . ◮ Equivalently, for constant E [ v 2 i ] we can decrease � v � 2 . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 19 / 29

  58. 1-sparse recovery: idea x = e i + w Candidate set Signal 0 bits v SNR = 2 I ( i , � v , x � ) � log SNR = 1 � v , x � = v i + � v , w � Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

  59. 1-sparse recovery: idea x = e i + w Candidate set Signal 0 bits 1 bit v SNR = 2 2 I ( i , � v , x � ) � log SNR = 2 � v , x � = v i + � v , w � Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

  60. 1-sparse recovery: idea x = e i + w Candidate set Signal 0 bits 1 bit 2 bits v SNR = 2 4 I ( i , � v , x � ) � log SNR = 4 � v , x � = v i + � v , w � Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

  61. 1-sparse recovery: idea x = e i + w Candidate set Signal 0 bits 1 bit 2 bits 4 bits v SNR = 2 8 I ( i , � v , x � ) � log SNR = 8 � v , x � = v i + � v , w � Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

  62. 1-sparse recovery: idea x = e i + w Candidate set Signal 0 bits 1 bit 2 bits 4 bits 8 bits v SNR = 2 16 I ( i , � v , x � ) � log SNR = 16 � v , x � = v i + � v , w � Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 20 / 29

  63. Goal Shown intuition for specific distribution on x Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

  64. Goal Shown intuition for specific distribution on x Match previous convergence for arbitrary x = α e i + w ? Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

  65. Goal Shown intuition for specific distribution on x Match previous convergence for arbitrary x = α e i + w ? ◮ α may not be 1. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

  66. Goal Shown intuition for specific distribution on x Match previous convergence for arbitrary x = α e i + w ? ◮ α may not be 1. ◮ Work for a specific x with 3 / 4 probability. Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

  67. Goal Shown intuition for specific distribution on x Match previous convergence for arbitrary x = α e i + w ? ◮ α may not be 1. ◮ Work for a specific x with 3 / 4 probability. ◮ Distribution over A , for fixed w . Eric Price (MIT) Adaptive Sparse Recovery 2012-04-26 21 / 29

Recommend

Adaptive Sparse Recovery with Limited Adaptivity Akshay Kamath Eric Price UT Austin 2018-11-27

Adaptive Sparse Recovery with Limited Adaptivity Akshay Kamath Eric Price UT Austin 2018-11-27

Adaptive Sparse Recovery with Limited Adaptivity Akshay Kamath Eric Price UT Austin 2018-11-27 Akshay Kamath, Eric Price (UT Austin) Adaptive Sparse Recovery with Limited Adaptivity 1 / 33 Outline Introduction 1 Analysis for k = 1 2

2k views • 162 slides
Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart

Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart

Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart University of Georgia Semester Program on High-Dimensional Approximation ICERM 7 October 2014 Part I: Nonnegative Sparse Recovery (joint work with

957 views • 69 slides
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup: Data/signal in n-dimensional

418 views • 25 slides
Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems

Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems

Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems Evelyn Herrholz Gerd Teschke November 9, 2010 Abstract In this paper we shall be concerned with compressive sampling strategies and sparse

417 views • 30 slides
Bounds on Sparse Recovery with Additional Structures Abbas Kazemipour University of Maryland.

Bounds on Sparse Recovery with Additional Structures Abbas Kazemipour University of Maryland.

Bounds on Sparse Recovery with Additional Structures Abbas Kazemipour University of Maryland. College Park kaazemi@umd.edu March 23, 2015 Abbas Kazemipour (UMD) Sparse Recovery with Side Info. March 23, 2015 1 / 19 Overview 1 Restricted

599 views • 42 slides
Efficient PDE Optimization under Uncertainty using Adaptive Model Reduction and Sparse Grids

Efficient PDE Optimization under Uncertainty using Adaptive Model Reduction and Sparse Grids

Efficient PDE Optimization under Uncertainty using Adaptive Model Reduction and Sparse Grids Matthew J. Zahr Advisor: Charbel Farhat Computational and Mathematical Engineering Stanford University Joint work with: Kevin Carlberg (Sandia CA),

444 views • 28 slides
Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse

Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse

Two-Level Approximation Error-Aware Trust-Region Model Management Numerical Experiments Adaptive Stochastic Collocation for PDE-Constrained Optimization under Uncertainty using Sparse Grids and Model Reduction Matthew J. Zahr Advisor: Charbel

612 views • 46 slides
Sparse Recovery and Fourier Sampling Eric Price MIT Eric Price (MIT) Sparse Recovery and

Sparse Recovery and Fourier Sampling Eric Price MIT Eric Price (MIT) Sparse Recovery and

Sparse Recovery and Fourier Sampling Eric Price MIT Eric Price (MIT) Sparse Recovery and Fourier Sampling 1 / 37 The Fourier Transform Conversion between time and frequency domains Frequency Domain Time Domain Fourier Transform

2.15k views • 187 slides
Generating Sparse Representations by Adaptive Multiscale Approximations Angela Kunoth University

Generating Sparse Representations by Adaptive Multiscale Approximations Angela Kunoth University

Generating Sparse Representations by Adaptive Multiscale Approximations Angela Kunoth University of Cologne, Germany Angela Kunoth Generating Sparse Representations by Adaptive Multiscale Approximations 1 Generating Sparse Representations

737 views • 34 slides
Two problems from neural data analysis: Sparse entropy estimation and efficient adaptive

Two problems from neural data analysis: Sparse entropy estimation and efficient adaptive

Two problems from neural data analysis: Sparse entropy estimation and efficient adaptive experimental design Liam Paninski Department of Statistics and Center for Theoretical Neuroscience Columbia University http://www.stat.columbia.edu/

702 views • 38 slides
New Constructions of RIP Matrices with Fast Multiplication and Fewer Rows aka, sparse recovery

New Constructions of RIP Matrices with Fast Multiplication and Fewer Rows aka, sparse recovery

New Constructions of RIP Matrices with Fast Multiplication and Fewer Rows aka, sparse recovery from Fourier -like measurements with applications to fast Johnson-Lindenstrauss transforms , etc. Jelani Nelson, Eric Price, and Mary Wootters February

394 views • 24 slides
Sparse Recovery via Differential Inclusions Yuan Yao School of Mathematical Sciences Peking

Sparse Recovery via Differential Inclusions Yuan Yao School of Mathematical Sciences Peking

Outline Bregman Iteration Path Consistency Discussion Sparse Recovery via Differential Inclusions Yuan Yao School of Mathematical Sciences Peking University September 2, 2014 with Stanley Osher (UCLA), Feng Ruan (PKU &amp; Stanford),

1.31k views • 33 slides
The Noise Collector for sparse recovery in high dimensions Alexei Novikov Department of

The Noise Collector for sparse recovery in high dimensions Alexei Novikov Department of

The Noise Collector for sparse recovery in high dimensions Alexei Novikov Department of Mathematics Penn State University, USA with M.Moscoso (Madrid), G.Papanicolaou (Stanford), and C.Tsogka (UC Merced). Supported by NSF and AFOSR. Inverse

433 views • 26 slides
Recursive Low-rank and Sparse Recovery of Surveillance Video using Compressed Sensing Shuangjiang

Recursive Low-rank and Sparse Recovery of Surveillance Video using Compressed Sensing Shuangjiang

Recursive Low-rank and Sparse Recovery of Surveillance Video using Compressed Sensing Shuangjiang Li, Hairong Qi Department of Electrical Engineering and Computer Science University of Tennessee, Knoxville Oct. 1, 2014 8th ACM/IEEE

523 views • 37 slides
Adaptive sparse grids and quasi Monte Carlo for option pricing under the rough Bergomi model

Adaptive sparse grids and quasi Monte Carlo for option pricing under the rough Bergomi model

Adaptive sparse grids and quasi Monte Carlo for option pricing under the rough Bergomi model Chiheb Ben Hammouda Christian Bayer Ra ul Tempone 3rd International Conference on Computational Finance (ICCF2019), A Coru na 8-12 July, 2019

737 views • 36 slides
GLAD: Learning Sparse Graph Recovery Le Song Georgia Tech Joint work with Harsh Shrivastava,

GLAD: Learning Sparse Graph Recovery Le Song Georgia Tech Joint work with Harsh Shrivastava,

GLAD: Learning Sparse Graph Recovery Le Song Georgia Tech Joint work with Harsh Shrivastava, Xinshi Chen, Binghong Chen, Guanghui Lan Objective Recovering sparse conditional independence graph G from data &quot;# = 0 &quot;

566 views • 27 slides
Recovery of sparse signals from a mixture of linear samples Arya Mazumdar Soumyabrata Pal

Recovery of sparse signals from a mixture of linear samples Arya Mazumdar Soumyabrata Pal

Recovery of sparse signals from a mixture of linear samples Arya Mazumdar Soumyabrata Pal University of Massachusetts Amherst June 15, 2020 ICML 2020 A relationship between features and labels x : feature and y : label . Consider the tuple (

322 views • 21 slides
Stochastic optimization and sparse statistical recovery: An optimal algorithm for high dimensions

Stochastic optimization and sparse statistical recovery: An optimal algorithm for high dimensions

Stochastic optimization and sparse statistical recovery: An optimal algorithm for high dimensions Alekh Agarwal Microsoft Research Joint work with Sahand Negahban and Martin Wainwright Workshop on Optimization and Statistical Learning 2013, Les

695 views • 42 slides
Computing Sparse Representations in O(NlogN) time May 3, 2013 Tsung-Han Lin and H.T. Kung

Computing Sparse Representations in O(NlogN) time May 3, 2013 Tsung-Han Lin and H.T. Kung

Computing Sparse Representations in O(NlogN) time May 3, 2013 Tsung-Han Lin and H.T. Kung Workshop on Signal Processing with Adaptive Sparse Structured Representations (SPARS 2013), July 2013 Hierarchical Feature Extraction Deep learning

83 views • 7 slides
Greedy Algorithms for Joint Sparse Recovery Jeff Blanchard with Mike Davies, Michael Cermak,

Greedy Algorithms for Joint Sparse Recovery Jeff Blanchard with Mike Davies, Michael Cermak,

Compressed Sensing Rank Awareness Greedy Algorithms Greedy Algorithms for Joint Sparse Recovery Jeff Blanchard with Mike Davies, Michael Cermak, David Hanle, Yirong Jing Grinnell College, Iowa, USA University of Edinburgh, UK Funding

474 views • 31 slides
Fast L0-based Sparse Signal Recovery Nick Kingsbury and Yingsong Zhang Signal Processing Group

Fast L0-based Sparse Signal Recovery Nick Kingsbury and Yingsong Zhang Signal Processing Group

Fast L0-based Sparse Signal Recovery Nick Kingsbury and Yingsong Zhang Signal Processing Group Department of Engineering ACVT seminar, Adelaide May 2011 Outline Introduction 1 L0 reweighted-L2 Minimization 2 IRLS Basic Model Basic

642 views • 34 slides
Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are zero lower triangular (?) dense few elements are zero These are structured sparse matrices. May be mapped into a 1D array so that a mapping

391 views • 6 slides
Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of Sparse Matrices diagonal tridiagonal lower triangular (?) These are structured sparse matrices. May be mapped into a 1D array so that a mapping

698 views • 11 slides
Adaptive Signal Recovery by Convex Optimization Dmitrii Ostrovskii CWI, Amsterdam 19 April 2018

Adaptive Signal Recovery by Convex Optimization Dmitrii Ostrovskii CWI, Amsterdam 19 April 2018

Adaptive Signal Recovery by Convex Optimization Dmitrii Ostrovskii CWI, Amsterdam 19 April 2018 Signal denoising problem Recover complex signal x = ( x ) , = n , ..., n , from noisy observations y = x + , = n ,

697 views • 43 slides

More recommend