Greedy Orthogonal Pivoting for Non-negative Matrix Factorization Kai Zhang, Jun Liu, Jie Zhang, Jun Wang Infinia ML Inc., Fudan University, East China Normal University
Non-negative Matrix Factorization β’ Represent data with non-negative basis [Lee & Seung, 2000][Ding et al. 2006] π β ππΌ 2 min π,πΌβ₯0 Coefficients Basis (rows) π° Γ β π β β πΓπ πΏ β’ Applications β’ Signal separation, Image classification, Gene expression analysis, Clusteringβ¦
Orthogonal NMF π β ππΌ 2 min β’ Motivation π,πΌβ₯0 π‘. π’. π β² π = π½ β NMF optimization is ill-posed β Task Preferences (cluster indicator matrix) β’ Existing Methods β Multiplicative updates [Ding et. al. 2006] β Soft orthogonality constraints [Shiga et al. 2014, Lin 2007] β Clustering-based formulation [Pompili et al. 2014] β’ Challenges β Zero-locking problem β Level of orthogonality hard to control
Greedy Orthogonal Pivoting Algorithm β’ A Group-coordinate-descent with adaptive updating variables and closed-form iterations β’ Exact orthogonality, easy to implement, faster convergence (batch-mode and randomized version)
Empirical Observations β’ Avoid zero-locking multiplicative updates β when starting from a feasible (sparse) solution, GOPA avoids pre-mature convergence GOPA β’ Faster Convergence GOPA GOPA GOPA
Future Work β’ Adaptive control of sparsity (or orthogonality) β’ New way of decomposition into sub-problems β’ Probabilistic error guarantee
Recommend
More recommend
