sparse pca
play

Sparse PCA refusing to graduate :-) Aviad Rubinstein (UC Berkeley) - PowerPoint PPT Presentation

Sep 17, 2023 •113 likes •166 views

Best* Case Approximability of Sparse PCA refusing to graduate :-) Aviad Rubinstein (UC Berkeley) Joint work with Siu-On Chan and Dimitris Papailliopoulos Sparse Principal Component Analysis max s.t. 2 = 1 and 0

  1. Best* Case Approximability of Sparse PCA refusing to graduate :-) Aviad Rubinstein (UC Berkeley) Joint work with Siu-On Chan and Dimitris Papailliopoulos

  2. Sparse Principal Component Analysis max 𝑦 ⊤ 𝐵𝑦 s.t. 𝑦 2 = 1 and 𝑦 0 ≤ 𝑙 (and 𝐵 is PSD) 7.1 1.3 ⋯ 4.5 −2.6 −3.4 ⋯ 6.2 ⋮ ⋮ ⋱ ⋮ 3.1 9.2 ⋯ −4.8 “Yes we SPCA!” -Obama, 2008

  3. Sparse Spiked Covariance Model (“average case”) rank 1 ⋯ 0 𝑙 × 𝑙 block 𝐵 = 𝐽 𝑜 + +noise ⋮ ⋱ ⋮ 0 ⋯ 0 • Cool sample-complexity / computational-complexity tradeoff [Berthet & Rigollet ’13 , Wang et al ’14 , Gao et al ’15 , Kraugthgamer et al ’15 , Ma & Wigderson ’15] • Good news: trivial algorithm gives (1 − 𝑝 1 ) -approximation • Bad news: this is not what your data looks like!

  4. Our results: “best - case” analysis Computationally intractable even when given the exact covariance matrix: • NP-hard to approximate to within (1 − 𝜗) • SSE-hard to approximate to within any 𝑑 • Quasi-quasi-poly ( 𝑓 𝑓 lnln𝑜 ) integrality gap • 𝑜 −1/3 -approximation algorithm “Vive la SPCA!” -Napoleon, 1808

Recommend

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are zero lower triangular (?) dense few elements are zero These are structured sparse matrices. May be mapped into a 1D array so that a mapping

391 views • 6 slides
Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of Sparse Matrices diagonal tridiagonal lower triangular (?) These are structured sparse matrices. May be mapped into a 1D array so that a mapping

698 views • 11 slides
= + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit,

= + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit,

Sparse processing / Compressed sensing Model : y = Ax + n , x is sparse = + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit, LASSO (convex objective function) Matching pursuit (greedy method)

210 views • 17 slides
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup: Data/signal in n-dimensional

418 views • 25 slides
Sparse Representations The Sparse Representations (SRs) framework was inspired by studies of

Sparse Representations The Sparse Representations (SRs) framework was inspired by studies of

Manifold Based Sparse Representation for Robust Expression Recognition without Neutral Subtraction Raymond Ptucha, Grigorios Tsagkatakis, Andreas Savakis Department of Computer Engineering, Rochester Institute of Technology, Rochester, NY Nov

411 views • 11 slides
Tutorial: TF-Ranking for sparse features Tutorial: TF-Ranking for sparse features This tutorial

Tutorial: TF-Ranking for sparse features Tutorial: TF-Ranking for sparse features This tutorial

Tutorial: TF-Ranking for sparse features Tutorial: TF-Ranking for sparse features This tutorial is an end-to-end walkthrough of training a TensorFlow Ranking (TF-Ranking) neural network model which incorporates sparse textual features.

257 views • 24 slides
Extremal results for sparse pseudorandom graphs Yufei Zhao Massachusetts Institute of Technology

Extremal results for sparse pseudorandom graphs Yufei Zhao Massachusetts Institute of Technology

Extremal results for sparse pseudorandom graphs Yufei Zhao Massachusetts Institute of Technology Joint work with David Conlon and Jacob Fox Sparse extensions Extending classical results to sparse settings. For example: Sparse extensions

882 views • 55 slides
Parallel Numerical Algorithms Chapter 4 Sparse Linear Systems Section 4.1 Direct Methods

Parallel Numerical Algorithms Chapter 4 Sparse Linear Systems Section 4.1 Direct Methods

Sparse Matrices Sparse Triangular Solve Cholesky Factorization Sparse Cholesky Factorization Parallel Numerical Algorithms Chapter 4 Sparse Linear Systems Section 4.1 Direct Methods Michael T. Heath and Edgar Solomonik Department of

906 views • 63 slides
Outline Background Sparse Coding Semi-supervised Learning with Sparse Coding

Outline Background Sparse Coding Semi-supervised Learning with Sparse Coding

Sparse-coded Net Model and Applications Y. Gwon, M. Cha, W. Campbell, H.T. Kung, C. Dagli IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2016) September 16, 2016 This work is sponsored by the

416 views • 17 slides
CNBC Matlab Mini-Course Sparse Matrices Sparse matrices provide an efficient means to store

CNBC Matlab Mini-Course Sparse Matrices Sparse matrices provide an efficient means to store

CNBC Matlab Mini-Course Sparse Matrices Sparse matrices provide an efficient means to store matrices that are mostly empty. David S. Touretzky s = sparse(1000, 1000) October 2019 s(2, 11) = 2 s(992, 875) = 3 Day 3: The Really Cool Stuff

550 views • 6 slides
sparse matrices and graphs L. Olson Department of Computer Science University of Illinois at

sparse matrices and graphs L. Olson Department of Computer Science University of Illinois at

sparse matrices and graphs L. Olson Department of Computer Science University of Illinois at Urbana-Champaign 1 objectives Convert a graph into a sparse matrix Go over a few sparse matrix storage formats Give an example of lower

513 views • 24 slides
Exploiting GPU Caches in Sparse Matrix Vector Multiplication Yusuke Nagasaka Tokyo Institute of

Exploiting GPU Caches in Sparse Matrix Vector Multiplication Yusuke Nagasaka Tokyo Institute of

Exploiting GPU Caches in Sparse Matrix Vector Multiplication Yusuke Nagasaka Tokyo Institute of Technology Sparse Matrix Generated by FEM, being as the graph data Often require solving sparse linear equation fast Iterative method :

629 views • 30 slides
Sparse and TV Kaczmarz solvers and the linearized Bregman method Dirk Lorenz, Frank Schpfer,

Sparse and TV Kaczmarz solvers and the linearized Bregman method Dirk Lorenz, Frank Schpfer,

Sparse and TV Kaczmarz solvers and the linearized Bregman method Dirk Lorenz, Frank Schpfer, Stephan Wenger, Marcus Magnor, March, 2014 Sparse Tomo Days, DTU Motivation Split feasibility problems Sparse Kaczmarz and TV-Kaczmarz Application

1.04k views • 67 slides
MLSS 06 - Canberra Elements Hierarchical Basis Sparse Grids Sparse Grids Combination

MLSS 06 - Canberra Elements Hierarchical Basis Sparse Grids Sparse Grids Combination

MLSS 06 - Canberra Sparse Grids Jochen Garcke (Very) Short Course on Finite MLSS 06 - Canberra Elements Hierarchical Basis Sparse Grids Sparse Grids Combination Technique Jochen Garcke Regression / Classification via Function

977 views • 68 slides
Parameter efficient training of deep convolutional neural networks by dynamic sparse

Parameter efficient training of deep convolutional neural networks by dynamic sparse

Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization Hesham Mostafa (Intel AI) Xin Wang (Intel AI, Cerebras Systems) Easy : post-training (sparse) compression Hard : direct training of sparse

727 views • 9 slides
Lecture 5 : Sparse Models Homework 3 discussion (Nima) Sparse Models Lecture - Reading :

Lecture 5 : Sparse Models Homework 3 discussion (Nima) Sparse Models Lecture - Reading :

Lecture 5 : Sparse Models Homework 3 discussion (Nima) Sparse Models Lecture - Reading : Murphy, Chapter 13.1, 13.3, 13.6.1 - Reading : Peter Knee, Chapter 2 Paolo Gabriel (TA) : Neural Brain Control After class - Project groups

571 views • 28 slides
High-Dimensional Pattern Recognition via Sparse Representation Allen Y. Yang University of

High-Dimensional Pattern Recognition via Sparse Representation Allen Y. Yang University of

Introduction Sparse Representation Sparse Optimization Wearable Augmented Reality Conclusion High-Dimensional Pattern Recognition via Sparse Representation Allen Y. Yang University of California, Berkeley and Atheer, Inc. UC Berkeley, June

498 views • 47 slides
Taming the Beast: Topic imaging Predictive approach Sparse Machine Learning for Large Text

Taming the Beast: Topic imaging Predictive approach Sparse Machine Learning for Large Text

Sparse ML for Text 1/33 L. El Ghaoui Information Overload Taming the Beast: Topic imaging Predictive approach Sparse Machine Learning for Large Text Corpora Visualizaations Beyond co-occurence Examples Research Agenda Sparse PCA Laurent

488 views • 47 slides
Sparse sampling sampling design in design in Sparse population PK/PD studies studies

Sparse sampling sampling design in design in Sparse population PK/PD studies studies

Sparse sampling sampling design in design in Sparse population PK/PD studies studies population PK/PD Sylvie Retout & France Mentr INSERM U738, Universit Paris 7, UFR de Mdecine, Hpital Bichat, Paris. 1 Outline Outline

674 views • 21 slides
Submanifold Sparse Convolutional Networks for Sparse, Locally Dense Particle Image Analysis

Submanifold Sparse Convolutional Networks for Sparse, Locally Dense Particle Image Analysis

Submanifold Sparse Convolutional Networks for Sparse, Locally Dense Particle Image Analysis Laura Domine (Stanford / SLAC) Kazuhiro Terao (SLAC) 2018 CPAD Instrumentation Frontier Workshop 1 Outline 1. Particle image analysis &

799 views • 31 slides
Sparse CCA using Lasso Anastasia Lykou & Joe Whittaker Department of Mathematics and

Sparse CCA using Lasso Anastasia Lykou & Joe Whittaker Department of Mathematics and

Introduction CCA Lasso Sparse CCA Summary Sparse CCA using Lasso Anastasia Lykou & Joe Whittaker Department of Mathematics and Statistics, Lancaster University July 23, 2008 Introduction CCA Lasso Sparse CCA Summary Outline

447 views • 34 slides
Parallel Sparse Matrix-Vector and Matrix- Transpose-Vector Multiplication using Compressed Sparse

Parallel Sparse Matrix-Vector and Matrix- Transpose-Vector Multiplication using Compressed Sparse

Parallel Sparse Matrix-Vector and Matrix- Transpose-Vector Multiplication using Compressed Sparse Blocks Aydn Bulu, UCSB Jeremy T. Fineman (MIT) Matteo Frigo (Cilk Arts) John R. Gilbert (UCSB) Charles E. Leiserson (MIT & Cilk Arts) 1

523 views • 21 slides
Cache&aware)Sparse)Matrix)Formats)) for)Kepler)GPU

Cache&aware)Sparse)Matrix)Formats)) for)Kepler)GPU

Cache&aware)Sparse)Matrix)Formats)) for)Kepler)GPU Yusuke&Nagasaka,&Akira&Nukada,&Satoshi&Matsuoka& Tokyo&Ins8tute&of&Technology& & Sparse)Matrix

563 views • 27 slides
From simple structure to sparse components: a comparative introduction Nickolay T. Trendafilov

From simple structure to sparse components: a comparative introduction Nickolay T. Trendafilov

ERCIM12, Session ES11: Sparse dimension reduction, 1-3 December 2012, Conference Centre, Oviedo, Spain From simple structure to sparse components: a comparative introduction Nickolay T. Trendafilov Department of Mathematics and Statistics

595 views • 30 slides

More recommend