matrix factorization
play

Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown - PowerPoint PPT Presentation

Sep 05, 2022 •154 likes •1.33k views

Matrix Factorization March 17, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie Pavlick HTAs: Josh Levin, Diane Mutako, Sol Zitter 1 Announcements 2 Today Matrix Factorization with SVD Applications to: Topic

  1. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 doc1 in new feature space 1 0 1 0 1 doc4 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  2. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 weight of component 1 for 1 0 1 0 1 doc4 doc 1 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  3. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 weight of component 1 over 1 0 1 0 1 doc4 all the data cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  4. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 component 1 1 0 1 0 1 doc4 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  5. cong parli the US UK ress ame Singular Value 1 1 1 1 0 doc1 Decomposition (SVD) 1 0 1 0 1 doc2 1 1 0 1 0 doc3 contribution of “the” to 1 0 1 0 1 doc4 component 1 cong parlia the US UK ress ment -0.65 -0.34 -0.51 -0.34 -0.31 d1 -0.60 -0.39 0.70 0.00 3.06 0.00 0.00 0.00 0.00 0.02 -0.54 0.34 -0.54 0.56 d2 -0.48 0.50 -0.12 -0.71 0.00 1.81 0.00 0.00 0.00 d3 -0.43 -0.58 -0.69 0.00 -0.42 0.02 0.79 0.02 -0.44 0.00 0.00 0.57 0.00 0.00 d4 -0.48 0.50 -0.12 0.71 -0.63 0.27 0.00 0.37 0.63 0.00 0.00 0.00 0.00 0.00 -0.04 0.73 0.00 -0.68 0.04 U D V

  6. Singular Value Decomposition (SVD) x = x M U D V m x n m x m m x n n x n

  7. 67

  8. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V m x n m x l l x l l x n

  9. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V m x n m x l l x l l x n keep only first l components

  10. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V m x n m x l l x l l x n keep only first l components “best l-rank approximation of M”

  11. d e t a Singular Value c n u r T Decomposition (SVD) x = x M U D V ||M - UDV|| 2 as small as possible m x n m x l l x l l x n keep only first l components “best l-rank approximation of M”

  12. 72

  13. Dimensionality Reduction • “Low Rank Assumption”: we typically assume that our features contain a large amount of redundant information • We can throw away a lot of principle components without losing too much of the signal needed for our task

  14. Clicker Question!

  15. Clicker Question! In practice, is this assumption of low rank valid? a) Yes b) No c) Yeah, sure, why not?

  16. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  17. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  18. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  19. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”*

  20. Matrices IRL • Data is noisy, so M is most likely full-rank • We assume that M is close to a low rank matrix, and we approximate the matrix it is close to • Viewed as a “de-noised” version of M • “Original matrix exhibits redundancy and noise, low-rank reconstruction exploits the former to remove the latter”* *Matrix and Tensor Factorization Methods for Natural Language Processing. (ACL 2015)

  21. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  22. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  23. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  24. Matrices IRL • Data is also often incomplete…missing values, new observations, etc. • Can we use SVD for this? • Yes! Though we need to make a few changes…

  25. Matrix Completion to all the ballad of mud- boys i roma buster okja bound loved scruggs… before user1 1 0 1 user2 0 0 1 user3 1 0 1 0 1 0 user4 user5 1

  26. Matrix Completion to all the ballad of mud- boys i roma buster okja bound loved scruggs… before 1 0 user1 1 0 1 user2 0 0 1 1 user3 1 0 1 0 1 0 user4 user5 1 “people also liked…”

  27. Matrix Completion

  28. Matrix Completion M ≈ UDV = M’

  29. Matrix Completion M ≈ UDV = M’ original completed

  30. Matrix Completion M ≈ UDV = M’ original completed problems?

  31. Matrix Completion x x = M U D V Exact SVD assumes M is complete…

  32. Matrix Completion x x = M U D V …just gradient descent that MF!

  33. MF with Gradient Descent x x = M U D V

  34. MF with Gradient Descent = x M U V

  35. MF with Gradient Descent = x M U V Not properly SVD (fewer guarantees, e.g. components not orthonormal) but good enough

  36. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> MF with Gradient Descent = x M U V X ( M ij − u i · v j ) 2 min U,V ij

  37. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> MF with Gradient Descent = x M U V X ( M ij − u i · v j ) 2 min U,V ij But! Only consider cases when M ij is observed!

  38. Clicker Question!

  39. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> Clicker Question! X ( M ij − u i · v j ) 2 min U,V ij Compute the loss given this setting of U and V… 2 1 1 2 3 2 2 0 0 M U V a) 14 b) 10 c) 6

  40. <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> <latexit sha1_base64="FxlQFTpt3XT/cP7nfIKG0d4whfk=">ACGXicbVDLSgMxFM34rPU16tJNsAgVtMwUQZdFN26ECvYBnXHIpGmbNskMSaZQhv6G3/FjQtFXOrKvzFtZ6GtBy73cM69JPeEMaNKO863tbS8srq2ntvIb25t7+zae/t1FSUSkxqOWCSbIVKEUFqmpGmrEkiIeMNMLB9cRvDIlUNBL3ehQTn6OuoB2KkTZSYDsepyJIa6ewPoaeSniQ0v4YFm9n/QwmAYUebkcaDoP+yUM5sAtOyZkCLhI3IwWQoRrYn147wgknQmOGlGq5Tqz9FElNMSPjvJcoEiM8QF3SMlQgTpSfTi8bw2OjtGEnkqaEhlP190aKuFIjHpJjnRPzXsT8T+vlejOpZ9SESeaCDx7qJMwqCM4iQm2qSRYs5EhCEtq/gpxD0mEtQkzb0Jw509eJPVyXVK7t15oXKVxZEDh+AIFIELkAF3IAqAEMHsEzeAVv1pP1Yr1bH7PRJSvbOQB/YH39ACajnxs=</latexit> Clicker Question! X ( M ij − u i · v j ) 2 min U,V ij Compute the loss given this setting of U and V… 1 2 2 1 x 1 2 = 2 4 3 2 0 0 2 0 0 M U V M’ a) 14 b) 10 c) 6

Recommend

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP?

L101: Matrix Factorization In a nutshell Matrix factorization/completion you know? In NLP? Word embeddings Topic models Information extraction FastText Why complete the matrix? Label Features Label Label Label Label

420 views • 20 slides
Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender

Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender

Introduction Matrix factorization Learning Matrix Factorization Models Online Updates Evaluation Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender Systems. Steffen Rendle, Lars Schmidt-Thieme

903 views • 21 slides
TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th

TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th

Introduction Non negative-Matrix Factorization Spectrogram model Decomposition of musical spectrograms Conclusion TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th International Conference on Digital

750 views • 26 slides
Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department

Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department

Matrix Factorization and Factorization Machines for Recommender Systems Chih-Jen Lin Department of Computer Science National Taiwan University Talk at SDM workshop on Machine Learning Methods on Recommender Systems, May 2, 2015 Chih-Jen Lin

1.14k views • 54 slides
Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon

Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon

Non-negative Matrix Factorization Fast Coordinate Descent methods for Non-Negative Matrix Factorization Inderjit S. Dhillon University of Texas at Austin SIAM Conference on Applied Linear Algebra Valencia, Spain June 19, 2012 Joint work with

682 views • 25 slides
Chapter IX: Matrix factorizations* 1. The general idea 2. Matrix factorization methods 3. Latent

Chapter IX: Matrix factorizations* 1. The general idea 2. Matrix factorization methods 3. Latent

Chapter IX: Matrix factorizations* 1. The general idea 2. Matrix factorization methods 3. Latent topic models 4. Dimensionality reduction *Zaki &amp; Meira, Ch. 8; Tan, Steinbach &amp; Kumar, App. B; Manning, Raghavan &amp; Schtze, Ch. 18

798 views • 53 slides
Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer,

Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer,

Matrix Factorization with Binary Components Uniqueness in a randomized model Felix Krahmer, TU M unchen Joint work with: Matthias Hein, Saarland University, David James , University of G ottingen Matrix Factorization given data

1.49k views • 44 slides
Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop

Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop

Matrix Factorization For Topic Models Dr. Derek Greene Insight Latent Space Workshop Non-negative Matrix Factorization NMF : an unsupervised family of algorithms that simultaneously perform dimension reduction and clustering. Also

407 views • 16 slides
CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson

CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson

CuMF: Large-Scale Matrix Factorization on Just One Machine with GPUs Wei Tan, IBM T. J. Watson Research Center wtan@us.ibm.com Agenda Why accelerate recommendation (matrix factorization) using GPUs? What are the challenges? How cuMF

403 views • 22 slides
Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang,

Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang,

Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang, Jun Wang Infinia ML Inc., Fudan University, East China Normal University Non-negative Matrix Factorization Represent data with non-negative basis

330 views • 7 slides
Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla December 17, 2011 Peter J. Haas Yannis Sismanis Christina Teflioudi Faraz Makari Outline Matrix Factorization Stochastic Gradient Descent

1.24k views • 75 slides
A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen

A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen

A Model For Mixed Linear-Tropical Matrix Factorization James Hook, Sanjar Karaev, Pauli Miettinen University of Birmingham: 18th June 2018 Low-Rank Approximate Factorization Given a matrix A R n m , an approximate factorization of rank k

207 views • 17 slides
Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com

Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com

cuMF_sgd: Fast and Scalable Matrix Factorization on GPUs Wei Tan IBM T. J. Watson Research Center wtan@us.ibm.com http://github.com/cumf http://researcher.ibm.com/person/us-wtan GTC 2017 Agenda Why accelerate matrix factorization?

571 views • 21 slides
Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla

Large-Scale Matrix Factorization with Distributed Stochastic Gradient Descent Rainer Gemulla August 23, 2011 Peter J. Haas Yannis Sismanis Erik Nijkamp Outline Matrix Factorization Stochastic Gradient Descent Distributed SGD with MapReduce

1.05k views • 81 slides
Structured sparse methods for matrix factorization Francis Bach Sierra team, INRIA - Ecole

Structured sparse methods for matrix factorization Francis Bach Sierra team, INRIA - Ecole

Structured sparse methods for matrix factorization Francis Bach Sierra team, INRIA - Ecole Normale Sup erieure March 2011 Joint work with R. Jenatton, J. Mairal, G. Obozinski Structured sparse methods for matrix factorization Outline

1.07k views • 73 slides
Nonnegative matrix factorization and applications in audio signal processing C edric F

Nonnegative matrix factorization and applications in audio signal processing C edric F

Nonnegative matrix factorization and applications in audio signal processing C edric F evotte Laboratoire Lagrange, Nice Machine Learning Crash Course Genova, June 2015 1 Outline Generalities Matrix factorisation models Nonnegative

581 views • 42 slides
Matrix Factorization and Lifting Palle Jorgensen and Myung-Sin Song The University of Iowa,

Matrix Factorization and Lifting Palle Jorgensen and Myung-Sin Song The University of Iowa,

Matrix Factorization and Lifting Palle Jorgensen and Myung-Sin Song The University of Iowa, Southern Illinois University Edwardsville January 7, 2011 Introduction Develop and refine a procedure which uses factorization of families of

544 views • 35 slides
Divide-and-Conquer Matrix Factorization Lester Mackey Collaborators: Ameet Talwalkar

Divide-and-Conquer Matrix Factorization Lester Mackey Collaborators: Ameet Talwalkar

Divide-and-Conquer Matrix Factorization Lester Mackey Collaborators: Ameet Talwalkar Michael I. Jordan Stanford University UCLA UC Berkeley December 14, 2015 Mackey (Stanford) Divide-and-Conquer Matrix Factorization

891 views • 42 slides
Multimodal Visualization Based On Non-negative Matrix Factorization Jorge Camargo Juan Caicedo

Multimodal Visualization Based On Non-negative Matrix Factorization Jorge Camargo Juan Caicedo

Multimodal Visualization Based On Non-negative Matrix Factorization Multimodal Visualization Based On Non-negative Matrix Factorization Jorge Camargo Juan Caicedo Fabio Gonzlez BioIngenium Research Group National University of Colombia

406 views • 29 slides
Some Recent Advances in Nonnegative Matrix Factorization and their Applications to Hyperspectral

Some Recent Advances in Nonnegative Matrix Factorization and their Applications to Hyperspectral

Some Recent Advances in Nonnegative Matrix Factorization and their Applications to Hyperspectral Unmixing Nicolas Gillis https://sites.google.com/site/nicolasgillis/ Universit e de Mons Department of Mathematics and Operational Research

1.15k views • 72 slides
Boolean matrix factorization meets consecutive ones property Nikolaj T atti &amp; Pauli

Boolean matrix factorization meets consecutive ones property Nikolaj T atti &amp; Pauli

Boolean matrix factorization meets consecutive ones property Nikolaj T atti &amp; Pauli Miettinen Boolean matrix factorisation Given a matrix A , find matrices B and C s.t. A B C T SVD, NMF, PCA, ICA, In

481 views • 25 slides
Automatic relevance determination in nonnegative matrix factorization with the -divergence

Automatic relevance determination in nonnegative matrix factorization with the -divergence

Background ARD in NMF Results Automatic relevance determination in nonnegative matrix factorization with the -divergence Vincent Y. F. Tan 1 and C evotte 2 edric F 2 CNRS LTCI; T 1 University of Wisconsin-Madison el ecom ParisTech

357 views • 8 slides
5.3 Diagonalization The goal here is to develop a useful factorization A = PDP 1 , when A is n

5.3 Diagonalization The goal here is to develop a useful factorization A = PDP 1 , when A is n

5.3 Diagonalization The goal here is to develop a useful factorization A = PDP 1 , when A is n n . We can use this to compute A k quickly for large k . The matrix D is a diagonal matrix (i.e. entries off the main diagonal are all zeros). D

278 views • 10 slides
Using noncommutative polynomial optimization for matrix factorization ranks Sander Gribling

Using noncommutative polynomial optimization for matrix factorization ranks Sander Gribling

Using noncommutative polynomial optimization for matrix factorization ranks Sander Gribling (CWI/QuSoft) David de Laat (CWI/QuSoft) Monique Laurent (CWI/Tilburg/QuSoft) SIAM Conference on Optimization, 25 May 2017, Vancouver Symmetric matrix

1.08k views • 93 slides

More recommend