Automatic relevance determination in nonnegative matrix - PowerPoint PPT Presentation

Background ARD in NMF Results Automatic relevance determination in nonnegative matrix factorization with the β -divergence Vincent Y. F. Tan 1 and C´ evotte 2 edric F´ 2 CNRS LTCI; T´ 1 University of Wisconsin-Madison el´ ecom ParisTech Paris, France NIPS “Sparse Low Rank” workshop December 2011 V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Nonnegative matrix factorization (NMF) Given a nonnegative matrix V of dimensions F × N , NMF is the problem of finding a factorization V ≈ WH where W and H are nonnegative matrices of dimensions F × K and K × N , respectively. V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Nonnegative matrix factorization (NMF) Given a nonnegative matrix V of dimensions F × N , NMF is the problem of finding a factorization V ≈ WH where W and H are nonnegative matrices of dimensions F × K and K × N , respectively. Constrained optimization problem: � W , H ≥ 0 D ( V | WH ) = min d ([ V ] fn | [ WH ] fn ) fn where d ( x | y ) is a scalar cost function. Objective of this work is to identify the “right” value of K . V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Automatic relevance determination in NMF Inspired by Bayesian PCA (Bishop, 1999): each “component” k is assigned a relevance (= variance) parameter φ k . φ 1 φ K h 1 h K + ... + ≈ V w 1 w K Half-Gaussian or exponential priors on w k and h k . � φ − 1 exp − φ − 1 � φ − 1 exp − φ − 1 E.g. , p ( w k | φ k ) = p ( h k | φ k ) = k w fk , k h kn k k n f V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Automatic relevance determination in NMF After a few manipulations, we are essentially left with the minimization of K � C ( W , H ) = D β ( V | WH ) + ρ log ( � w k � + � h k � + ε ) k =1 where ◮ D β ( V | WH ) is the measure of fit (in this work, β -divergence) ◮ � x � = 1 2 � x � 2 2 (half-Gaussian priors) or � x � = � x � 1 (exponential priors). V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Swimmer decomposition results 8 data samples (among 256) Estimated W using with exponential priors / ℓ 1 penalization V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Audio decomposition results � ����� � � � � � � � � � � � � � � � � � audio signal 0.5 0 −0.5 2 4 6 8 10 12 14 time (s) log power spectrogram 500 400 frequency 300 200 100 100 200 300 400 500 600 frame V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF

Background ARD in NMF Results Audio decomposition results IS−NMF ARD IS−NMF 0.05 0.05 0.04 0.04 0.03 0.03 0.02 0.02 0.01 0.01 0 0 0 5 10 15 20 0 5 10 15 20 Figure: Histograms of standard deviation values of all K = 18 components produced by Itakura-Saito NMF and ARD Itakura-Saito NMF (with ℓ 2 penalization). ARD IS-NMF only retains the 6 “right” components. Check our full-length technical report available on arxiv. V. Y. F. Tan (Univ. of Wisconsin) and C. F´ evotte (CNRS) Automatic relevance determination in β -NMF