Collaborative Metric Le Learning An Andy Hs Hsieh eh, Lo Longqi - PowerPoint PPT Presentation

Collaborative Metric Le Learning An Andy Hs Hsieh eh, Lo Longqi Yang, , Yi Yin Cui, Ts Tsung-Yi Yi Lin , Serge Be Belongie, D , Debo borah Es h Estri rin Connected Experience Lab, Cornell Tech AOL CONNECTED CORNELL TECH EXPERIENCES LAB 1

Co Collaborative Me Metri ric Le Learn rning • A different perspective on collaborative filtering • Better accuracy • Extremely efficient Top-K recommendations • Easy to interpret and extend 2

Us User er-It Item em Matr trix ix Items Users 3

Ma Matri rix Factori rization (MF MF) Items Items ≈ Users Users 4

Im Implic plicit it Feedbac eedback ? ? • Ubiquitous in today’s online services ? ? • Only positive feedback is available ? • Traditional MF does not work ? ? ? Click Thumbs up Like ? ? ? 5

Ma Matri rix Factori rization for r Imp mplicit Feedback • Weighted Regularized Matrix Factorization ( WR WRMF MF ) [Hu08] • Probabilistic Matrix Factorization ( PM PMF ) [Salakhutdinov08] • Bayesian Personalized Ranking ( BPR BPR ) [Rendle09] and many more … 6

Think Beyond Matrix Th ? ? Implicit Explicit ? ? No longer about estimating ratings • ? But about modeling the relationships • ? ? ? between different user/item pairs ? ? ? 7

Th Think Beyond Matrix Implicit Explicit No longer about estimating ratings • But about modeling the relationships • between different user/item pairs 8

Me Metri ric Le Learn rning Unknown relationships Known relationships 9

Co Collaborative Me Metri ric Le Learn rning • Learn a joint user-item distance metric. • The Euclidean distances reflect the relationships between users/items. 10

Ba Based ed o on th the i e inher eren ent T t Triangu gular In Ineq equality o ty of Metric c Learning – If If A is cl close to B, a , and nd B is cl close to to C, the , then n A is cl close to C. • Fit the model with implicit feedback 1. An user is pulled closer to the items she liked 2. Other similar users are pulled closer. 3. The items users liked are also pulled closer. • Top-K recommendations are simply KNN search (a well-optimized task) 11

Ba Based ed o on th the i e inher eren ent T t Triangu gular In Ineq equality o ty of Metric c Learning – If If A is cl close to B, a , and nd B is cl close to to C, the , then n A is cl close to C. • Fit the model with implicit feedback 1. An user is pulled closer to the items she liked 2. Other similar users are pulled closer. 3. The items users liked are also pulled closer. • Top-K recommendations are simply KNN search (a well-optimized task) 12

Ba Based ed o on th the i e inher eren ent T t Triangu gular In Ineq equality o ty of Metric c Learning – If If A is cl close to B, a , and nd B is cl close to to C, the , then n A is cl close to C. • Fit the model with implicit feedback 1. An user is pulled closer to the items she liked 2. Other similar users are pulled closer. 3. The items users liked are also pulled closer. • Top-K recommendations are simply KNN search (a well-optimized task) 13

Co Collaborative La Large Ma Margin Nearest Neighbor r Before After Safety Margin User Positive item Imposter Gradients * The outline of figure is inspired by Weinberger, Kilian Q., John Blitzer, and Lawrence Saul. "Distance metric learning for large margin nearest neighbor 14 classification." Advances in neural information processing systems 18 (2006): 1473.

Pitf Pitfalls alls of Matr trix ix Fac actoriz izatio tion (Dot-Pr Product) • Dot-Product violates triangle inequality misleading embedding. 15

Pitfalls Pitf alls of Matr trix ix Fac actoriz izatio tion (Dot-Pr Product) • Dot-Product violates triangle inequality misleading embedding. $ 𝑊 𝑊 % = 0: does not reflect that # they are both liked by 𝑉 * $ 𝑉 % = 0 : does not reflect that 𝑉 # they both share the same interest as 𝑉 * 16

Co Collaborative Me Metri ric Le Learn rning Emb mbedding • Euclidian distance faithfully reflects the relative relationships. 17

In Integ egrating ting It Item em Fea eatur tures es • Use a learnable function (e.g. Multi-Layer Perceptron) to project features into user-item embedding. • Treat the projections as a prior for items' locations. 18

Ev Evaluation • 6 Datasets from Different Domains Papers - CiteULike • Pa Books - BookCrossing • Bo • Ph Photography - Flickr ticles - Medium • Ar Arti Movies - MovieLens • Mo • Mu Musi sic - EchoNest 19

Accuracy (Recall@5 @50) Recall@50 Improvements Over BPR (%) 100 80 60 * 40 * * 20 * 0 CiteULike BookCX Flickr Medium MovieLens EchoNest -20 -40 WRMF WARP CML 20 * Indicate that CML > the second best algorithm is statistically significant according to Wilcoxon signed rank test

Ac Accur uracy y (wi (with h Item Featur ures) s) Recall@50 Improvements Over Factorization Machine (%) 120 100 80 60 * 40 * 20 * 0 CiteULike BookCX Flickr Medium MovieLens -20 VBPR CDL CML+F * Indicate that CML > the second best algorithm is statistically significant according to Wilcoxon signed rank test 21

Ef Efficiency 8x faster • All optimized with LSHs • CML’s throughput is improved by 106x with only 2% reduction in accuracy • Over 8x faster than (optimized) MF models given the same accuracy ‘s are brute force search 22

Em Embe beddi dding ng Interpr pretabi bility A A B C B C 23

Co Conclusi sions • The notion of user-item matrix and matrix factorization becomes less applicable with implicit feedback. • CML is a metric learning model that has • better accuracy, efficiency, interpretability, and extensibility. • Applying metric-based algorithms, such as K-means, and SVMs, to other recommendation problems. 24

Thank you! AOL CONNECTED CORNELL TECH EXPERIENCES LAB 25