Mixed Factorization for Collaborative Recommendation with Heterogeneous Explicit Feedbacks Weike Pan , Shanchuan Xia, Zhuode Liu, Xiaogang Peng and Zhong Ming ∗ panweike@szu.edu.cn, shaun.xia@outlook.com, zero.lzd@gmail.com, patrickpeng@126.com, mingz@szu.edu.cn College of Computer Science and Software Engineering Shenzhen University, Shenzhen, China Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 1 / 37
Introduction Problem Definition Collaborative Recommendation with Heterogeneous Explicit Feedbacks (CR-HEF) Input: 5-star grade scores R = { ( u , i , r ui ) } , where r ui ∈ G = { 0 . 5 , 1 , 1 . 5 , . . . 5 } R = { ( u , i , ˜ r ui ) } , where like/dislike binary ratings ˜ r ui ∈ B = { like , dislike } ˜ Goal: rating prediction (of the missing 5-star grade scores) Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 2 / 37
Introduction Challenges How can we integrate two different types of feedbacks in a principled way? Collective factorization such as CMF [Singh and Gordon, 2008]: two jointly conducted factorization tasks are loosely coupled, which may not fully leverage the binary ratings to the grade scores. Integrative factorization such as SVD++ [Koren, 2008]: leveraging the implicit feedbacks to the grade scores in such an integrative manner may not well capture the implicit-feedback-dependent effect. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 3 / 37
Introduction Overall of Our Solution (1/2) Transfer by Mixed Factorization (TMF) We first take the CR-HEF problem from a transfer learning view, in which grade scores are taken as target data and binary ratings are taken as auxiliary data. We then propose a novel and generic mixed factorization based transfer learning framework, which consists of collective factorization and integrative factorization as different assembly components. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 4 / 37
Introduction Overall of Our Solution (2/2) The following methods are special cases of our TMF RSVD [Koren, 2008]: { e 1 , e 2 } CMF [Singh and Gordon, 2008]: { e 1 , e 2 , e 3 , e 4 } iTCF [Pan and Ming, 2014]: { e 1 , e 2 , e 3 , e 4 , e 5 } Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 5 / 37
Introduction Advantages of Our Solution TMF unifies collective factorization and integrative factorization in one single transfer learning framework, which enables both feature-based and instance-based preference learning and transfer in a principled way. TMF is expected to transfer more knowledge from binary ratings to grade scores than collective factorization, and to model binary-rating-dependent and -independent effect more accurately than integrative factorization. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 6 / 37
Introduction Notations (1/3) Table: Some notations. n user number m item number u ∈ { 1 , 2 , . . . , n } user ID i , j ∈ { 1 , 2 , . . . , m } item ID r ui observed grade score of user u on item i r ui observed binary rating of user u on item i ˜ R = { ( u , i , r ui ) } grade score records (training data) R = { ( u , i , ˜ ˜ r ui ) } binary rating records (training data) p = |R| number of grade scores p = | ˜ ˜ R| number of binary ratings Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 7 / 37
Introduction Notations (2/3) Table: Some notations (cont.). items liked by user u P u items disliked by user u N u Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 8 / 37
Introduction Notations (3/3) Table: Some notations (cont.). µ ∈ R global average rating value b u ∈ R user bias b i ∈ R item bias d ∈ R number of latent dimensions U u · , W u · ∈ R 1 × d user-specific latent feature vector U , W ∈ R n × d user-specific latent feature matrix V i · , P j · , N j · ∈ R 1 × d item-specific latent feature vector V , P , N ∈ R m × d item-specific latent feature matrix T E = { ( u , i , r ui ) } grade score records of test data r ui predicted preference of user u on item i ˆ T iteration number in the algorithm Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 9 / 37
Related Work Transfer Learning for Heterogeneous Feedbacks Transfer learning approaches Model-based transfer Feature-based transfer Instance-based transfer Transfer learning algorithm styles Adaptive knowledge transfer Collective knowledge transfer Integrative knowledge transfer Mixed knowledge transfer: Collective knowledge transfer + Integrative knowledge transfer TMF: Feature-based transfer + Instance-based transfer + Mixed knowledge transfer Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 10 / 37
Related Work Factorization for Collaborative Recommendation Different problem settings Explicit feedbacks Implicit feedbacks Explicit feedbacks and implicit feedbacks Heterogeneous explicit feedbacks TMF: for heterogeneous explicit feedbacks Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 11 / 37
Method Prediction Rule for Grade Scores For grade scores, the prediction rule of user u on item i , r ui = U u · V T P u · V T N u · V T i · + b u + b i + µ, i · + ¯ i · + ¯ ˆ (1) where ¯ P u · and ¯ N u · are virtual user profiles from binary feedbacks: 1 P u · δ P w p P j · , ¯ � = (2) � |P u | j ∈P u 1 N u · ¯ δ N w n � N j · , = (3) � |N u | j ∈N u where δ P , δ N ∈ { 1 , 0 } , and w p , w n are weight. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 12 / 37
Method Prediction Rule for Binary Ratings For binary ratings, the prediction rule of user u on item i , r ui = W u · V T ˆ ˜ i · . (4) Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 13 / 37
Method Objective Function for Grade Scores For grade scores, we have the objective function, n m y ui [ 1 r ui ) 2 + reg ( U u · , V i · , b u , b i , P , N )] 2 ( r ui − ˆ � � min (5) Θ u = 1 i = 1 where reg ( U u · , V i · , b u , b i , P , N ) = α u 2 � U u · � 2 + α v 2 � V i · � 2 + β u 2 � b u � 2 + 2 � b i � 2 + δ P j ∈P u � P � 2 j ∈N u � N � 2 β v α p F + δ N α n � � F is the 2 2 regularization term used to avoid overfitting. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 14 / 37
Method Objective Function for Binary Ratings For binary ratings, we have the objective function, n m y ui [ 1 r ui ) 2 + reg ( W u · , V i · )] 2 ( r ui − ˆ � � ˜ ˜ min (6) Θ u = 1 i = 1 2 � W u · � 2 + α v 2 � V i · � 2 is the regularization where reg ( W u · , V i · ) = α w term used to avoid overfitting. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 15 / 37
Method Overall Objective Function We have the overall objective function, n m n m y ui f ui + λ y ui ˜ f ui � � � � ˜ min (7) Θ u = 1 u = 1 i = 1 i = 1 where f ui = 1 2 ( r ui − ˆ r ui ) 2 + reg ( U u · , V i · , b u , b i , P , N ) and f ui = 1 2 ( r ui − ˆ r ui ) 2 + reg ( W u · , V i · ) . ˜ ˜ Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 16 / 37
Method Gradient for Grade Scores (1/2) Denoting f ui = 1 2 ( r ui − ˆ r ui ) 2 + reg ( U u · , V i · , b u , b i , P , N ) , we have, ∂ f ui ∂µ = − e ui , ∇ µ = (8) ∂ f ui ∇ b u = − e ui + β u b u , = (9) ∂ b u ∂ f ui ∇ b i = − e ui + β v b i , = (10) ∂ b i ∂ f ui ∇ U u · = − e ui V i · + α u U u · , = (11) ∂ U u · Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 17 / 37
Method Gradient for Grade Scores (2/2) ∂ f ui ∇ V i · = − e ui ( ρ U u · + ( 1 − ρ ) W u · + ¯ P u · + ¯ N u · ) + α v V i · , (12) = ∂ V i · ∂ f ui 1 ∇ P j · = δ P ( − e ui w p V i · + α p P j · ) , j ∈ P u , = (13) ∂ P j · � |P u | ∂ f ui 1 ∇ N j · = δ N ( − e ui w n V i · + α n N j · ) , j ∈ N u , = (14) ∂ N j · � |N u | where e ui = ( r ui − ˆ r ui ) , and ρ U u · + ( 1 − ρ ) W u · is used to introduce rich interactions as that in iTCF [Pan and Ming, 2014]. Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 18 / 37
Method Gradient for Binary Ratings r ui ) 2 + reg ( W u · , V i · ) , we have, Denoting ˜ f ui = 1 2 ( r ui − ˆ ˜ ∂ ˜ f ui ∇ W u · e ui V i · + α w W u · ) , = λ ( − ˜ = (15) ∂ W u · f ui ∂ ˜ ∇ V i · e ui ( ρ W u · + ( 1 − ρ ) U u · ) + α v V i · ) , = λ ( − ˜ = (16) ∂ V i · e ui = (˜ r ui − ˆ r ui ) . Note that “dislike” is converted as ˜ r ui = 1 and where ˜ ˜ r ui = 5. “like” is converted as ˜ Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 19 / 37
Method Update Rule We have the update rules, θ = θ − γ ∇ θ, (17) where γ is the learning rate, and θ can be µ , b u , b i , U u · , V i · , W u · , P j · , N j · . Pan et al., (CSSE, SZU) Transfer by Mixed Factorization (TMF) Information Sciences 2016 20 / 37
Recommend
More recommend
