Ordinal Non-negative Matrix Factorization for Recommendation International Conference on Machine Learning Olivier Gouvert 1 Thomas Oberlin 2 evotte 1 C´ edric F´ 1 IRIT, Universit´ e de Toulouse, CNRS, France 2 ISAE-SUPAERO, Universit´ e de Toulouse, France
Introduction OrdNMF Experimental Results Conclusion Collaborative Filtering (CF) ◮ Based only on the feedbacks of users on items ◮ Y : feedback matrix, of size U × I y ui : feedback of a user u ∈ { 1 , . . . , U } on an item i ∈ { 1 , . . . , I } Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 2 of 14
Introduction OrdNMF Experimental Results Conclusion Collaborative Filtering (CF) ◮ Based only on the feedbacks of users on items ◮ Y : feedback matrix, of size U × I y ui : feedback of a user u ∈ { 1 , . . . , U } on an item i ∈ { 1 , . . . , I } ◮ Ordinal data : nominal data which exhibit a natural ordering [Stevens et al., 1946]: • Explicit feedbacks: bad ≺ average ≺ good ≺ excellent • Implicit feedbacks: quantized play counts • ... without loss of generality y ui ∈ { 0 , . . . , V } Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 2 of 14
Introduction OrdNMF Experimental Results Conclusion Non-negative Matrix Factorization (NMF) ◮ Approximation: Y ≈ WH T [Lee and Seung, 1999] • W ≥ 0 of size U × K : preferences of the users • H ≥ 0 of size I × K : attributes of the items H T ≈ ˆ Y W Y Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 3 of 14
Introduction OrdNMF Experimental Results Conclusion Non-negative Matrix Factorization (NMF) ◮ Approximation: Y ≈ WH T [Lee and Seung, 1999] • W ≥ 0 of size U × K : preferences of the users • H ≥ 0 of size I × K : attributes of the items H T ≈ ˆ Y W Y ◮ How to process ordinal data? Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 3 of 14
Introduction OrdNMF Experimental Results Conclusion How to process ordinal data? ◮ Threshold models: • Quantization of a continuous latent variable • Some examples: [Chu and Ghahramani, 2005, Paquet et al., 2012, Hernandez-Lobato et al., 2014] Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 4 of 14
Introduction OrdNMF Experimental Results Conclusion How to process ordinal data? ◮ Threshold models: • Quantization of a continuous latent variable • Some examples: [Chu and Ghahramani, 2005, Paquet et al., 2012, Hernandez-Lobato et al., 2014] ◮ Contributions: • NMF for ordinal data (OrdNMF) ◮ Non-negative constraints ◮ Multiplicative noise ◮ Link with Poisson factoriaztion (PF) [Gopalan et al., 2015] • Efficient variational algorithm ◮ Augmentation trick ◮ Scales with the number of non-zero values • Excellent flexibility of OrdNMF Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 4 of 14
Introduction OrdNMF Experimental Results Conclusion Ordinal Non-negative Matrix Factorization (OrdNMF) ◮ Approximation: Y ≈ G b ( WH T ) • Y ordinal matrix • W ≥ 0 and H ≥ 0 10 ◮ Quantization of the non-negative numbers v = G b ( x ) 8 G b : → { 0 , . . . , V } R + 6 v x �→ v such that x ∈ [ b v − 1 , b v ) 4 2 where b is an increasing sequence of thresholds 0 10 0 10 2 10 4 10 6 x Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 5 of 14
Introduction OrdNMF Experimental Results Conclusion Ordinal Non-negative Matrix Factorization (OrdNMF) ◮ Approximation: Y ≈ G b ( WH T ) • Y ordinal matrix • W ≥ 0 and H ≥ 0 10 ◮ Quantization of the non-negative numbers v = G b ( x ) 8 G b : → { 0 , . . . , V } R + 6 v x �→ v such that x ∈ [ b v − 1 , b v ) 4 2 where b is an increasing sequence of thresholds 0 10 0 10 2 10 4 10 6 x ◮ Goal: joint estimation of W , H and b Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 5 of 14
Introduction OrdNMF Experimental Results Conclusion Ordinal Non-negative Matrix Factorization (OrdNMF) ◮ Generative model: W X H x ui = [ WH T ] ui · ε ui G b y ui = G b ( x ui ) Y ◮ Multiplicative noise: ε non-negative random variable with c.d.f. F ε Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 6 of 14
Introduction OrdNMF Experimental Results Conclusion Ordinal Non-negative Matrix Factorization (OrdNMF) ◮ Generative model: W X H x ui = [ WH T ] ui · ε ui G b y ui = G b ( x ui ) Y ◮ Multiplicative noise: ε non-negative random variable with c.d.f. F ε ◮ Cumulative distribution function: P [ y ui ≤ v | W , H ] = P [ G b ( x ui ) ≤ v | W , H ] [ WH T ] ui · ε ui < b v � � = P � � b v = P ε ui < [ WH T ] ui � b v � = F ε [ WH T ] ui Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 6 of 14
Introduction OrdNMF Experimental Results Conclusion Inverse-Gamma OrdNMF ◮ Generative model: W X H x ui = [ WH T ] ui · ε ui G b y ui = G b ( x ui ) Y ◮ Inverse-gamma noise: ε ui ∼ IG(1 , 1) Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 7 of 14
Introduction OrdNMF Experimental Results Conclusion Inverse-Gamma OrdNMF ◮ Generative model: W X H x ui = [ WH T ] ui · ε ui G b y ui = G b ( x ui ) Y ◮ Inverse-gamma noise: ε ui ∼ IG(1 , 1) ◮ Cumulative distribution function: P [ y ui ≤ v | W , H ] = e − [ WH T ] ui b − 1 v or P [ y ui > v | W , H ] = 1 − e − [ WH T ] ui b − 1 v Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 7 of 14
Introduction OrdNMF Experimental Results Conclusion Interpretation ◮ V dependent Bernoulli models: � 1 − e − [ WH T ] ui b v − 1 � { y ui > v } ∼ Bern , v ∈ { 0 , . . . , V − 1 } • V = 1 : Bernoulli-Poisson factorization (BePoF) [Acharya et al., 2015] • ... Poisson factorization (PF) [Gopalan et al., 2015] applied on binary data Y with V = 3 (4 classes) Y > 0 Y > 1 Y > 2 Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 8 of 14
Introduction OrdNMF Experimental Results Conclusion Bayesian Inference ◮ Bayesian inference: • A priori: w uk ∼ Gamma( α W , β W u ) and h ik ∼ Gamma( α H , β H i ) • Variational inference (VI): p ( W , H | Y ) ≈ q ( W ) q ( H ) Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 9 of 14
Introduction OrdNMF Experimental Results Conclusion Bayesian Inference ◮ Bayesian inference: • A priori: w uk ∼ Gamma( α W , β W u ) and h ik ∼ Gamma( α H , β H i ) • Variational inference (VI): p ( W , H | Y ) ≈ q ( W ) q ( H ) ◮ Log-likelihood, with ∆ v = b − 1 v − 1 − b − 1 v : − [ WH T ] ui b − 1 � 0 , if v = 0 log P [ y ui = v | W , H ] = − [ WH T ] ui b − 1 v + log(1 − e − [ WH T ] ui ∆ v ) , if v > 0 Non-conjugate model Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 9 of 14
Introduction OrdNMF Experimental Results Conclusion Model Augmentation ◮ Trick: model augmentation similar to [Acharya et al., 2015] � if y ui = 0 δ 0 , n ui | y ui , W , H ∼ ZTP([ WH T ] ui ∆ y ui ) , if y ui > 0 • Joint likelihood: generalized Kullback-Leibler divergence • Scales with the number of non-zero in Y Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 10 of 14
Introduction OrdNMF Experimental Results Conclusion Model Augmentation ◮ Trick: model augmentation similar to [Acharya et al., 2015] � if y ui = 0 δ 0 , n ui | y ui , W , H ∼ ZTP([ WH T ] ui ∆ y ui ) , if y ui > 0 • Joint likelihood: generalized Kullback-Leibler divergence • Scales with the number of non-zero in Y ◮ Threshold optimization: working on the decrement sequence ∆ (defined as ∆ v = b − 1 v − 1 − b − 1 v ) rather than on the threshold sequence b • Very simple update rules for b Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 10 of 14
Introduction OrdNMF Experimental Results Conclusion Experimental Results ◮ MovieLens dataset • Ratings of users on movies on a scale from 1 to 10 • Class 0 : absence of a rating ◮ Splitting of Y : • Y train : 80% of the non-zero values • Y test : remaining 20% Ordinal Non-negative Matrix Factorization for Recommendation Gouvert O., Oberlin T. and F´ evotte C. ICML 2020 11 of 14
Recommend
More recommend
