Learning Hierarchical Representation Model for Next Basket - PowerPoint PPT Presentation

Learning Hierarchical Representation Model for Next Basket Recommendation Pengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Shengxian Wan, Xueqi Cheng CAS Key Lab of Network Data Science and Technology Institute of Computing Technology,Chinese Academy of Sciences

Outline • Task • Background • Motivation • Our model structure of HRM connections to previous methods • Experiments • Summary 2

Problem: Next basket recommendation Next basket recommendation : given a sequence of purchases, what items are purchased sequentially? transaction i-1 transaction i transaction i+1 potato maternity milk baby-car beers butter ? nursing bread bread bottle earphone phone battery 3

Background • Sequence models MC (Markov Chain) [Zimdars et al. UAI01, Chen et al. KDD12] • Collaborative Filtering NMF (Non negative Matrix Factorization ) [Daniel D. Lee et al. NIPS01] • Hybrid methods FPMC (Factorized Personalized Markov Chain) [S Rendle et al. WWW10] 4

Weakness of Previous methods Sequence models based on Markov Assumption ： MC transactions of u 1 weakness of MC: Lack users` general interests a a b d transactions of u 2 a b c a,b,c, d represent items 5

Weakness of Previous methods Collaborative filtering matrix factorization ： NMF transactions of u 1 weakness of NMF: Lack sequence behaviors a a b d Global one : transactions of u 2 a b Global set c 6

Weakness of Previous methods Hybrid method ： Factorized Personalized Markov Chain [S Rendle et al. WWW10 best paper] transactions of u 1 weakness of FPMC: linear combination of different factors a a v j b v i d v i transactions of u 2 v j a b v c u U represent users, V represent items 7

Weakness of Previous methods Hybrid method ： Factorized Personalized Markov Chain [S Rendle et al. WWW10 best paper] transactions of u 1 weakness of FPMC: linear combination of different factors a a b + General taste Sequential behavior d user next item next item last item transactions of u 2 a b I,L ,v 1 L,I > <v i c linear I,L ,v 2 L,I > <v i combination Independent Influence! I,L ,v 3 L,I > <v i 8

Weakness of Previous methods Is that linear combination enough for a good recommendation? pumpkin potato cucumber Last transaction next transaction candy chocolate chips Last transaction next transaction Halloween! + pumpkin candy 9

Motivation We need a model that is capable of incorporating more complicated interactions among multiple factors. This becomes the major motivation of our work. 10

Hierarchical Representation Model  The structure of HRM Level 2 Level 1 general interest sequential behavior 11

Aggregation Method  Linear method average pooling  Nonlinear method max pooling other types of operators(top-k average pooling, k-max pooling, hadamard pooling) (V is a set of input vectors to be aggregated)

Learning and Prediction The probability of purchasing one item next transaction: exponent of item and users` hybrid interest sum of all items: too large! Objective function all users all trans all items Negative sampling negative count sample distribution 13

Connection to Previous methods  Degradation To MC softmax Select copy : copy item when constructing the transaction select copy representation from item vectors, the operation randomly selects one item vector and copies it select copy = 14

Connection to Previous methods  Degradation To MF softmax Select copy : always select and copy user vector in the second select copy layer, ignoring the sequential information = 15

Connection to Previous methods  Degradation To FPMC softmax avg pooling Avg Pooling is used , each instance corresponds to 1 negative sample avg pooling 16

Experiments  Data sets retails ecommerce Ta-Feng BeiRen T-Mall #transactions 67964 91294 1805 #items 7982 5845 191 #users 9238 9321 292 #avg.transaction 5.9 5.8 1.2 size #avg.transaction 7.4 9.7 5.6 per user 17

Experiments  Evaluation Metric  F1-score: harmonic mean of precision and recall  Hit-ratio: coverage  NDCG: a ranking measure 18

Experiments  Comparison among Different HRMs  observation Avg pooling perform worst When apply max pooling on any layer, the performance improved a little When apply max pooling on all layers, HRM performed best 19

Experiments  Comparison with baselines top popular sequential behavior general interest hybrid method  observation Top method performed worst NMF and MC performed better than top method FPMC performed better than NMF and MC HRM performed best 20

Experiments  Comparison over groups  observation NMF perform better than MC on active group, while MC performs better than NMF on inactive group HRM performed best 21

Experiments  The Impact of Negative Sampling  observation More negative count we choose ,the more F1-score we obtain The sampling number k increases, the performance gain between two consecutive trials decreases 22

Summary A next basket recommendation task A Hierarchical Representation Model • model both sequential behavior and users` general taste • Aggregation operators to connect two level factors. • HRM can produce multiple recommendation models by introducing different aggregation operations Furture works • More aggregations operations will be analyzed • Integrate other types of information, e.g. timestamp of transaction 23

Thank You ！ EMAIL:wangpengfei@software.ict.ac.cn 24

• Poin twise mutual information (PMI) is a widely used word similarity measure

Weakness of Previous methods the sum of score in general recommend and score of sequential recommend 27