2020-1 UROP 2020.03.12 2016-12146 Seri Lee Goal Multi-Behavior - PowerPoint PPT Presentation

2020-1 UROP 2020.03.12 2016-12146 Seri Lee

Goal Multi-Behavior Recommendation “Given user behavior data of multiple types, predict users’ next behaviors of target type.”

Approach • Implement an RNN-based recommendation algorithm for a single behavior type. • Extend the algorithm to further utilize other types of behaviors by using attention mechanisms.

Previous work (1) Learning recommender systems from multi- behavior data

Limitations • NMTR cannot capture sequential patterns since it does not consider the time sequence of behaviors. • New algorithm should capture sequential patterns by using Recurrent Neural Network.

Previous work (2) ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation

Attention-Based Heterogeneous Behaviors Modeling Framework

Raw Feature Spaces object behavior timestamp U = { ( a j , o j , t j ) | j = 1,2,…, m } behavior groups according to target object types G = { bg 1 , bg 2 , …, bg n } bg i ∩ bg j = ∅ U = ∪ n i =1 bg i group-specific neural nets to build up behavior embedding

Attention-Based Heterogeneous Behaviors Modeling Framework

Behavior Embedding Spaces embedding building block u ij = f i ( a j , o j , t j ) u ij = emb i ( o j ) + lookup t i ( bucketize i ( t j )) + lookup a i ( a j ) output: list of vectors in all behavior groups B = { u bg 1 , u bg 2 , …, u bg n }

Attention-Based Heterogeneous Behaviors Modeling Framework

Latent Semantic Spaces projection function to fix-length encoding vectors (put them into same semantic space) S = concat (0) ( F M 1 ( u bg 1 ), F M 2 ( u bg 2 ), …, F M n ( u bg n )) overall space of dimension size S all projected behavior embedding in each spaces S k = F P k ( S ) projection function (single layer perceptron, ReLu activation function)

Attention-Based Heterogeneous Behaviors Modeling Framework

Self-Attention Layer goal: capture the inner-relationships among each semantic space self-attention score vector A k = softmax ( a ( S k , S ; θ )) score function a ( S k , S ; θ k ) = S k W k S T <bilinear scoring function> attention C k = A k F Q k ( S ) vectors of projection function: single space k layer perceptron+ReLU concatenated & reorganized C = 𝔊 self ( concat (1) ( C 1 , C 2 , …, C K )) feedforward network with one hidden layer

Attention-Based Heterogeneous Behaviors Modeling Framework

⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ ⃗ Downstream Application Network : point-wise / pair-wise fully connected nn vanilla attention h t = F M g ( t ) ( q t ) s k = F P k ( h t ) c k = softmax ( a ( s k , C ; θ k )) F Q k ( C ) final context vector e t u = F vanilla ( concat (1) (( c 1 , c 2 , …, c K ))) final loss function: sigmoid cross entropy loss − ∑ y t log ( σ ( f ( h t , e t u ))) + (1 − y t ) log (1 − σ ( f ( h t , e t u ))) t , u ranking function

Future Work • Implement ATRank with the given Dataset.

Dataset • https://www.kaggle.com/mkechinov/ecommerce- behavior-data-from-multi-category-store/data# • eCommerce behavior data from multi category store • behavior: view, cart, remove_from_cart, purchase • object behavior: purchase