Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea - PowerPoint PPT Presentation

Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Stanford University Physics of Algorithms Santa Fe - Aug 31, 2009 R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 1 / 24

Motivating Example : Recommender System Netflix Challenge 2 1 3 1 4 4 5 4 4 3 5 2 · 10 4 movies 4 1 5 4 4 1 3 3 4 4 1 M = 4 4 5 3 4 1 2 1 2 1 3 4 4 4 2 5 · 10 5 users 10 8 ratings R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 2 / 24

The Model R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 3 / 24

Matrix Completion Problem V T Σ r n Low-rank M = U n α r 1. Low-rank matrix M = U Σ V T . U T U = n , V T V = n α, Σ = diag (Σ 1 , Σ 2 , . . . , Σ r ) 2. Uniformly random sample E ⊂ [ n ] × [ n α ] given its size | E | . [ k ] = { 1 , 2 , . . . , k } R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 4 / 24

Matrix Completion Problem Sample M E n n α 1. Low-rank matrix M = U Σ V T . U T U = n , V T V = n α, Σ = diag (Σ 1 , Σ 2 , . . . , Σ r ) 2. Uniformly random sample E ⊂ [ n ] × [ n α ] given its size | E | . [ k ] = { 1 , 2 , . . . , k } R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 4 / 24

Matrix Completion Problems � � For any estimation � √ mn || M − � 1 M , let RMSE = M || F || A || 2 ij A 2 F = P ij R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 5 / 24

Matrix Completion Problems � � For any estimation � √ mn || M − � 1 M , let RMSE = M || F Q1. How many samples do we need to get RMSE ≤ δ ? (1 + α ) rn � | E | = O ( nr ) Q2. How many samples do we need to recover M exactly ? n log n � | E | = O ( n log n ) Q3. What if the samples are corrupted by noise? R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 5 / 24

Matrix Completion Problems Q1. How many samples do we need to get RMSE ≤ δ ? (1 + α ) rn � | E | = O ( nr ) Q2. How many samples do we need to recover M exactly ? n log n � | E | = O ( n log n ) Q3. What if the samples are corrupted by noise? R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 5 / 24

Pathological Example M = e 1 e T 1 �    1 0 · · · 0     0 0 · · · 0    n    . . . ... . . .    . . .  � 0 0 · · · 0 P (observing M 11 ) = | E | n 2 R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 6 / 24

Incoherence Property M is ( µ 0 , µ 1 )-incoherent if √ r ; , A 1 . M max ≤ µ 0 Σ 1 r r � � U 2 V 2 A 2 . ia ≤ µ 1 r , ja ≤ µ 1 r . a =1 a =1 [Cand´ es, Recht 2008 [1]] R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 7 / 24

Previous Work Theorem (Cand´ es, Recht 2008 [1]) Let M be an n × n α matrix of rank r satisfying ( µ 0 , µ 1 )-incoherence condition. If | E | ≥ C ( α, µ 0 , µ 1 ) rn 6 / 5 log n , then w.h.p. Semidefinite Programming reconstructs M exactly. R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 8 / 24

Main Contributions Questions Main Results � � 1 2 nr 1. RMSE ≤ C ( α ) | E | 2. Exact Reconstruction | E | = O ( n log n ) 3. Noisy Reconstruction | E | = O ( n log n ) M || F ≤ C n √ α r √ mn || M − � 1 | E | || Z E || 2 ( N = M + Z ) 4. Complexity? O ( | E | r log n ) R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 9 / 24

The Algorithm and Main Theorems R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 10 / 24

Na¨ ıve Approach � M ij if ( i , j ) ∈ E , M E ij = 0 otherwise . n � M E = x k σ k y T k k =1 Rank- r projection : r � P r ( M E ) ≡ n 2 α x k σ k y T k | E | k =1 R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 11 / 24

Na¨ ıve Approach Fails Define : deg ( row i ) ≡ # of samples in row i . � For | E | = O ( n ), spurious singular values of Ω( log n / (log log n )). Solution : Trimming  0 if deg ( row i ) > 2 E [ deg ( row i )] ,  M E � 0 if deg ( col j ) > 2 E [ deg ( col i )] , ij =  M E otherwise . ij R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 12 / 24

The Algorithm OptSpace Input : sample positions E , sample values M E , rank r Output : estimation � M M E be the output; Trim M E , and let � 1: Compute rank- r projection P r ( � M E ) = X 0 S 0 Y T 2: 0 ; 3: R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 13 / 24

Main Result Theorem (Keshavan, Montanari, Oh, 2009 [2]) Let M be an n × n α matrix of rank-r bounded by M max . Then, w.h.p, � nr 1 || M − P r ( � M E ) || F = RMSE ≤ C ( α ) | E | , nM max R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 14 / 24

Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea - PowerPoint PPT Presentation

Matrix Completion from a Few Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh Stanford University Physics of Algorithms Santa Fe - Aug 31, 2009 R.Keshavan, A.Montanari, S.Oh (Stanford) Physics of Algorithms 2009 Aug 31, 2009 1 /

Matrix Completion from Fewer Entries Raghunandan Keshavan, Andrea Montanari and Sewoong Oh

The Parameterized Complexity of Matrix Completion Robert Ganian Joint work with: Eduard Eiben

Introductory Matrix Operations Matrix Entries Defn. For matrix A , notation a ij means the en-

Sum of matrix entries of representations of the symmetric group and its asymptotics Dario De

Singularity Degree of PSD Matrix Completion Shin-ichi Tanigawa CWI and Kyoto July 29, 2016 1 /

Matrix Completion and Matrix Concentration Lester Mackey Collaborators: Ameet Talwalkar ,

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

Implement Distributed Alternating Least Squares Algorithm for Matrix Completion Varun Gandhi

Sub-Gaussian Estimators of the Mean of a Random Matrix with Entries Possessing Only Two Moments

Matrix Completion and Matrix Concentration Lester Mackey, Ameet Talwalkar, Michael I. Jordan

Eigenvalues and Eigenvectors Suppose A is an n n symmetric matrix with real entries. The

Empirical Testing of Sparse Approximation and Matrix Completion Algorithms Jared Tanner Workshop

Lecture 15: Exact Tensor Completion Joint Work with David Steurer Lecture Outline Part I:

Nonlinear algebra and matrix completion Daniel Irving Bernstein Massachusetts Institute of

Authority and Co-cite, Hub and Co-reference Given the adjacency matrix A (with entries 0 or 1) a i

Low Rank Matrix Completion: A Smoothed 0 -Search Wei Dai Jointly with Guangyu Zhou and

Low-rank Matrix Completion via Convex Optimization Ben Recht Center for the Mathematics of

Probabilistic Low-Rank Matrix Completion with Adaptive Spectral Regularization Algorithms Fran

Data Mining and Matrices 04 Matrix Completion Rainer Gemulla, Pauli Miettinen May 02, 2013

Maximum Likelihood Matrix Completion Under Sparse Factor Models: Error Guarantees and Efficient

Sparse Sensing in Colocated MIMO Radar: A Matrix Completion Approach Athina P. Petropulu

Median Matrix Completion: from Embarrassment to Optimality Xiaojun Mao School of Data Science

ADAPTIVE MATRIX COMPLETION FOR FAST VISIBILITY COMPUTATIONS WITH MANY LIGHTS RENDERING Sunrise

An Improved Matrix Completion Algorithm For Categorical Variables: Application to Active Learning