matrix completion from fewer entries
play

Matrix Completion from Fewer Entries Raghunandan Keshavan, Andrea - PowerPoint PPT Presentation

May 29, 2023 •370 likes •913 views

Matrix Completion from Fewer Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Stanford University March 30, 2009 Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion Outline The problem, a look at the data, and

  1. Matrix Completion from Fewer Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Stanford University March 30, 2009 Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  2. Outline The problem, a look at the data, and some results (slides) 1 Proofs (blackboard) 2 arXiv:0901.3150 Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  3. The problem, a look at the data, and some results Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  4. Netflix dataset: A big (!) matrix 2 1 3 1 4 4 5 4 4 3 5 2 · 10 4 movies 4 1 5 4 4 1 3 3 4 4 M = 1 4 4 5 3 4 1 2 1 2 1 3 4 4 4 2 5 · 10 5 users 10 8 ratings Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  5. A big (!) matrix 2 1 3 ? 1 4 4 5 4 4 3 5 ? 2 · 10 4 movies 4 1 5 4 4 1 3 3 4 4 M = 1 4 4 ? 5 3 4 1 2 ? 1 2 1 3 ? 4 4 4 2 5 · 10 5 users 10 6 queries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  6. You get a prize if. . . RMSE < 0 . 8563 ; − ) Is this possible? Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  7. You get a prize if. . . RMSE < 0 . 8563 ; − ) Is this possible? Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  8. You get a prize if. . . RMSE < 0 . 8563 ; − ) Is this possible? Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  9. A model: Incoherent low-rank matrices Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  10. The observations 13421532161432614361436514327147171542154437171521726547152481582524858141258141841852423233334448148 24312412365126251454231542321542121432413512442422555231552162561272662262626713242442515252233333341 5125125653426356254412545346532671735712351663571237213533333333172671238127638172681871881 24312412365126251454231542321542143214324135124424225534242444245231552162561272662262626711515252241 24312412365126251454231542321542143214324135124423323212144422555231552162561272662262626711515252241 24312412365126251454231542321542143214324135124424225552315521625612726622621412412212626711515252241 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543154311315464566366531253151353116‘161466 24312412365126251454231542321542143214324135124424225552315521625612726622626267115143434343225252241 24312412365126251454231542321542143214324135124424225552315521625612726622623452352566626711515252241 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543154311315312345334646653151353116‘161466 n movies 24312412365126251454231542321542143214324135124424225552315521343466663562561272662262626711515252241 24312412365126251454231542321542143214324135124424223434543453555231552162561272662262626711515252241 41315426514236152461547‘614542422471‘6567157157‘65147‘353534543361524154315431131531253151353116‘161466 24312412365126251454231542321542143214343453453452413512442422555231552162561272662262626711515252241 24312412365126251454231542321542143245345354551432413512442422555423155216256127266226262671151525241 41315426514236152461547‘614542422471‘346567157157‘65147‘61524154315431131531253151353116‘16146453454356 M = 24312412365126251454231542321542143214324135133133111124424225552315521625461272662262626711515252241 24312412365126251454231542321542143334211233321432413512442422555231552162561272662262626711515252241 24312412365126251454231542321542143214324135124424225552315521625612721231‘13132662262626711515252241 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543154311131232333311531253151353116‘161466 24312412365126251454231542321542143214324135124424225552315521625612726622626267115152522443744747441 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543154311343344453551531253151353116‘161466 143265421542715765127651543151221652465236125436541625143615243162534535666461r5261463416452646161611 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543154366363443135131531253151353116‘161466 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543144444345554431131531253151353116‘161466 24312412365126251454231542321542143214324135124424225552315521625446346466661272662262626711515252241 24312412365126251454231542321542143214324135124424225552315521625446436666661272662262626711515252241 41315426514236152461547‘614542422471‘6567157157‘65147‘615241545345346664315431131531253151353116‘161466 24312412365126251454231542321542143214324135124424225552315521625464363423361272662262626711515252241 24312412365126251454231542321542143214324135124424225552315521624534466433561272662262626711515252241 41315426514236152461547‘614542422471‘6567157157‘65147‘615241543135353453445431131531253151353116‘161466 24312412365126251454231542321542343434445514321432413512442422555231552162561272662262626711515252241 24312412365126251454231542434444453332154214321432413512442422555231552162561272662262626711515252241 n α users Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  11. The observations 2 1 3 1 4 4 5 4 4 3 5 n movies 4 1 5 4 4 M E = 1 3 3 4 4 1 4 4 5 3 4 1 2 1 2 1 3 4 4 4 2 n α users n ǫ unif. random positions Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  12. You need some structure! r V T r n α M = U n r ≪ n Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  13. You need some structure! r V T r n α M = U n r ≪ n Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  14. Unstructured factors A1. Bounded entries √ r . | M ia | ≤ M max = µ 0 A2. Incoherence r r � � U 2 V 2 ik ≤ µ 1 r , ak ≤ µ 1 r . k =1 k =1 [Cand´ es, Recht 2008] Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  15. Metric ( RMSE )   1 / 2   � 1 D (M , ˆ | M ia − ˆ M ia | 2 M) ≡  n 2 M 2  max i , a Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  16. Previous work Theorem (Cand´ es, Recht, 2008) If ǫ ≥ C r n 1 / 5 log n then whp 1. M is unique given the observed entries. 2. M is the unique minimum of a SDP. cf. also [Recht, Fazel, Parrilo 2007] Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  17. Previous work Theorem (Cand´ es, Recht, 2008) If ǫ ≥ C r n 1 / 5 log n then whp 1. M is unique given the observed entries. 2. M is the unique minimum of a SDP. cf. also [Recht, Fazel, Parrilo 2007] Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  18. Previous work Theorem (Cand´ es, Recht, 2008) If ǫ ≥ C r n 1 / 5 log n then whp 1. M is unique given the observed entries. 2. M is the unique minimum of a SDP. cf. also [Recht, Fazel, Parrilo 2007] Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  19. Previous work Theorem (Cand´ es, Recht, 2008) If ǫ ≥ C r n 1 / 5 log n then whp 1. M is unique given the observed entries. 2. M is the unique minimum of a SDP. cf. also [Recht, Fazel, Parrilo 2007] Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  20. Previous work Theorem (Cand´ es, Recht, 2008) If ǫ ≥ C r n 1 / 5 log n then whp 1. M is unique given the observed entries. 2. M is the unique minimum of a SDP. cf. also [Recht, Fazel, Parrilo 2007] Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  21. Great, but. . . 1. n 1 / 5 observations for 1 bit of information? 2. RMSE = 0? O ( n 4 ... 6 ). Substitute n = 10 5 . . . 3. SDP = Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  22. Great, but. . . 1. n 1 / 5 observations for 1 bit of information? 2. RMSE = 0? O ( n 4 ... 6 ). Substitute n = 10 5 . . . 3. SDP = Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  23. Great, but. . . 1. n 1 / 5 observations for 1 bit of information? 2. RMSE = 0? O ( n 4 ... 6 ). Substitute n = 10 5 . . . 3. SDP = Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  24. Great, but. . . 1. n 1 / 5 observations for 1 bit of information? 2. RMSE = 0? O ( n 4 ... 6 ). Substitute n = 10 5 . . . 3. SDP = Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  25. O ( n ) entries are enough (practice) Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  26. A movie Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  27. Rank = 1: Bayes optimal vs. Belief Propagation 1 n=100 n=1000 n=10000 0.8 0.6 D 0.4 0.2 0 0 2 4 6 8 10 12 14 ǫ Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  28. Rank = 2: Belief Propagation n=100 1.4 n=1000 n=10000 1.2 1 0.8 D 0.6 0.4 0.2 0 0 2 4 6 8 10 12 ǫ Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  29. Rank = 3: Belief Propagation 1.8 n=100 n=1000 1.6 n=10000 1.4 1.2 1 D 0.8 0.6 0.4 0.2 0 0 2 4 6 8 10 12 14 16 18 ǫ Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  30. Rank = 4: Belief Propagation 2.2 n=100 2 n=1000 n=10000 1.8 1.6 1.4 1.2 D 1 0.8 0.6 0.4 0.2 0 0 5 10 15 20 ǫ Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

  31. O ( n ) entries are enough (theory) Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Matrix Completion

Recommend

The Parameterized Complexity of Matrix Completion Robert Ganian Joint work with: Eduard Eiben

The Parameterized Complexity of Matrix Completion Robert Ganian Joint work with: Eduard Eiben

The Parameterized Complexity of Matrix Completion Robert Ganian Joint work with: Eduard Eiben Iyad Kanj Sebastian Ordyniak Stefan Szeider Matrix Completion: Basic Measures Input: Matrix over GF(p) with missing entries 0 0 2 1 2 1

681 views • 38 slides
Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh

Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh

Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Stanford University Physics of Algorithms Santa Fe - Aug 31, 2009 R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 1 /

491 views • 47 slides
Introductory Matrix Operations Matrix Entries Defn. For matrix A , notation a ij means the en-

Introductory Matrix Operations Matrix Entries Defn. For matrix A , notation a ij means the en-

Introductory Matrix Operations Matrix Entries Defn. For matrix A , notation a ij means the en- try in row i and column j of A . matOpsONE: 2 Matrix Addition and Scalar Multiplication Matrix addition requires the two ma- Defn. trices have the

230 views • 8 slides
Sum of matrix entries of representations of the symmetric group and its asymptotics Dario De

Sum of matrix entries of representations of the symmetric group and its asymptotics Dario De

Sum of matrix entries of representations of the symmetric group and its asymptotics Sum of matrix entries of representations of the symmetric group and its asymptotics Dario De Stavola 13 October 2015 Advisor: Valentin Fray Affiliation:

851 views • 68 slides
Singularity Degree of PSD Matrix Completion Shin-ichi Tanigawa CWI and Kyoto July 29, 2016 1 /

Singularity Degree of PSD Matrix Completion Shin-ichi Tanigawa CWI and Kyoto July 29, 2016 1 /

Singularity Degree of PSD Matrix Completion Shin-ichi Tanigawa CWI and Kyoto July 29, 2016 1 / 13 Positive Semidefinite Matrix Completion PSD completion problem ( G , c ) Given G = ( V , E ) with V = { 1 , . . . , n } and edge weight c : E

883 views • 41 slides
Matrix Completion and Matrix Concentration Lester Mackey Collaborators: Ameet Talwalkar ,

Matrix Completion and Matrix Concentration Lester Mackey Collaborators: Ameet Talwalkar ,

Matrix Completion and Matrix Concentration Lester Mackey Collaborators: Ameet Talwalkar , Michael I. Jordan , Richard Y. Chen , Brendan Farrell , Joel A. Tropp , and Daniel Paulin Stanford University UCLA

651 views • 43 slides
L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP? Word embeddings Topic models Information extraction FastText Why complete the matrix? Label Features Label Label Label Label

420 views • 20 slides
Implement Distributed Alternating Least Squares Algorithm for Matrix Completion Varun Gandhi

Implement Distributed Alternating Least Squares Algorithm for Matrix Completion Varun Gandhi

Implement Distributed Alternating Least Squares Algorithm for Matrix Completion Varun Gandhi (vg292) Computer Laboratory Netflix Problem V: m*n matrix complete the matrix W: m*r (row-factor matrix) H: r*n

250 views • 6 slides
Sub-Gaussian Estimators of the Mean of a Random Matrix with Entries Possessing Only Two Moments

Sub-Gaussian Estimators of the Mean of a Random Matrix with Entries Possessing Only Two Moments

Sub-Gaussian Estimators of the Mean of a Random Matrix with Entries Possessing Only Two Moments Stas Minsker University of Southern California July 21, 2016 ICERM Workshop Simple question: how to estimate the mean? Assume that X 1 , . . . , X

1k views • 50 slides
Matrix Completion and Matrix Concentration Lester Mackey, Ameet Talwalkar, Michael I. Jordan

Matrix Completion and Matrix Concentration Lester Mackey, Ameet Talwalkar, Michael I. Jordan

Matrix Completion and Matrix Concentration Lester Mackey, Ameet Talwalkar, Michael I. Jordan University of California, Berkeley Richard Chen, Brendan Farrell, Joel Tropp Caltech October 8, 2012 Part I Divide-Factor-Combine Jordan (UC

613 views • 39 slides
Eigenvalues and Eigenvectors Suppose A is an n n symmetric matrix with real entries. The

Eigenvalues and Eigenvectors Suppose A is an n n symmetric matrix with real entries. The

Eigenvalues and Eigenvectors Suppose A is an n n symmetric matrix with real entries. The function from R n to R defined by x x t Ax is called a quadratic form. We can maximize x T Ax subject to x T x = || x || 2 = 1 by

742 views • 33 slides
Empirical Testing of Sparse Approximation and Matrix Completion Algorithms Jared Tanner Workshop

Empirical Testing of Sparse Approximation and Matrix Completion Algorithms Jared Tanner Workshop

Sparse Approximation Phase Transitions Matrix completion Empirical Testing of Sparse Approximation and Matrix Completion Algorithms Jared Tanner Workshop on Sparsity, Compressed Sensing and Applications University of Oxford

655 views • 34 slides
Lecture 15: Exact Tensor Completion Joint Work with David Steurer Lecture Outline Part I:

Lecture 15: Exact Tensor Completion Joint Work with David Steurer Lecture Outline Part I:

Lecture 15: Exact Tensor Completion Joint Work with David Steurer Lecture Outline Part I: Matrix Completion Problem Part II: Matrix Completion via Nuclear Norm Minimization Part III: Generalization to Tensor Completion

664 views • 44 slides
Nonlinear algebra and matrix completion Daniel Irving Bernstein Massachusetts Institute of

Nonlinear algebra and matrix completion Daniel Irving Bernstein Massachusetts Institute of

Nonlinear algebra and matrix completion Daniel Irving Bernstein Massachusetts Institute of Technology and ICERM dibernst@mit.edu dibernstein.github.io Daniel Irving Bernstein Nonlinear algebra and matrix completion 1 / 20 Funding and

258 views • 22 slides
Authority and Co-cite, Hub and Co-reference Given the adjacency matrix A (with entries 0 or 1) a i

Authority and Co-cite, Hub and Co-reference Given the adjacency matrix A (with entries 0 or 1) a i

Authority and Co-cite, Hub and Co-reference Given the adjacency matrix A (with entries 0 or 1) a i = A T Aa i 1 a i = ( A T A ) i a 0 h i = AA T h i 1 h i = ( AA T ) i h 0 Co-citation: the number of pages co-cite P i and P j Co-reference:

482 views • 13 slides
Low Rank Matrix Completion: A Smoothed 0 -Search Wei Dai Jointly with Guangyu Zhou and

Low Rank Matrix Completion: A Smoothed 0 -Search Wei Dai Jointly with Guangyu Zhou and

Low Rank Matrix Completion: A Smoothed 0 -Search Wei Dai Jointly with Guangyu Zhou and Xiaochen Zhao Imperial College London Queen Mary University 2012 Zhou, Zhao, and Dai (Imperial College) Matrix Completion: 0 -Search Queen Mary 2012

784 views • 33 slides
Low-rank Matrix Completion via Convex Optimization Ben Recht Center for the Mathematics of

Low-rank Matrix Completion via Convex Optimization Ben Recht Center for the Mathematics of

Low-rank Matrix Completion via Convex Optimization Ben Recht Center for the Mathematics of Information Caltech Recommender Systems Netflix Prize One million big ones! Given 100 million ratings on a scale of 1 to 5, predict 3

404 views • 24 slides
Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Fran

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Fran

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Fran cois Caron Department of Statistics, Oxford STATLEARN 2014, Paris April 7, 2014 Joint work with Adrien Todeschini, Marie Chavent (INRIA, U. of

699 views • 45 slides
Data Mining and Matrices 04 Matrix Completion Rainer Gemulla, Pauli Miettinen May 02, 2013

Data Mining and Matrices 04 Matrix Completion Rainer Gemulla, Pauli Miettinen May 02, 2013

Data Mining and Matrices 04 Matrix Completion Rainer Gemulla, Pauli Miettinen May 02, 2013 Recommender systems Problem Set of users Set of items (movies, books, jokes, products, stories, ...) Feedback (ratings, purchase,

477 views • 35 slides
Maximum Likelihood Matrix Completion Under Sparse Factor Models: Error Guarantees and Efficient

Maximum Likelihood Matrix Completion Under Sparse Factor Models: Error Guarantees and Efficient

Maximum Likelihood Matrix Completion Under Sparse Factor Models: Error Guarantees and Efficient Algorithms Jarvis Haupt Department of Electrical and Computer Engineering University of Minnesota Institute for Computational and Experimental

596 views • 40 slides
Sparse Sensing in Colocated MIMO Radar: A Matrix Completion Approach Athina P. Petropulu

Sparse Sensing in Colocated MIMO Radar: A Matrix Completion Approach Athina P. Petropulu

Sparse Sensing in Colocated MIMO Radar: A Matrix Completion Approach Athina P. Petropulu Department of Electrical and Computer Engineering Rutgers, the State University of New Jersey Acknowledgments Shunqiao Sun, D. Kalogerias, W. Bajwa,

474 views • 22 slides
Median Matrix Completion: from Embarrassment to Optimality Xiaojun Mao School of Data Science

Median Matrix Completion: from Embarrassment to Optimality Xiaojun Mao School of Data Science

Median Matrix Completion: from Embarrassment to Optimality Xiaojun Mao School of Data Science Fudan University, China June 15, 2020 Joint work with Dr. Weidong Liu (Shanghai Jiao Tong University, China) and Dr. Raymond K. W. Wong (Texas

507 views • 22 slides
ADAPTIVE MATRIX COMPLETION FOR FAST VISIBILITY COMPUTATIONS WITH MANY LIGHTS RENDERING Sunrise

ADAPTIVE MATRIX COMPLETION FOR FAST VISIBILITY COMPUTATIONS WITH MANY LIGHTS RENDERING Sunrise

ADAPTIVE MATRIX COMPLETION FOR FAST VISIBILITY COMPUTATIONS WITH MANY LIGHTS RENDERING Sunrise Wang and Nicolas Holzschuch 1 Many-Lights Problem Direct lighting problem Large numbers of point lights Instant Radiosity (Keller

309 views • 29 slides
An Improved Matrix Completion Algorithm For Categorical Variables: Application to Active Learning

An Improved Matrix Completion Algorithm For Categorical Variables: Application to Active Learning

An Improved Matrix Completion Algorithm For Categorical Variables: Application to Active Learning of Drug Responses Huangqingbo Sun, Robert F. Murphy Workshop on Real World Experiment Design and Active Learning at ICML 2020 Drug Discovery

92 views • 6 slides

More recommend