Music Recommendation Based on Multiple Contextual Similarity - PowerPoint PPT Presentation

Music ¡Recommendation ¡Based ¡on ¡ Multiple ¡Contextual ¡Similarity ¡Information Chih-­‑Ming ¡Chen, ¡Ming-­‑Feng ¡Tsai ¡ Department ¡of ¡Computer ¡Science ¡& ¡Program ¡in ¡Digital ¡Content ¡and ¡Technology ¡ National ¡Chengchi ¡University ¡ Jen-­‑Yu ¡Liu, ¡Yi-­‑Hsuan ¡Yang ¡ Research ¡Center ¡for ¡Information ¡Technology ¡Innovation ¡ Academia ¡Sinica ¡ Taipei, ¡Taiwan WI-­‑IAT-­‑2013 1

Our Studies 1 2 3 User Age Factorization Machine User B Artist E Music D Article( User% User C Artist L Music G User F Artist P Music F User B Artist E Music D User K Artist S User C Artist L Music G Match the emotions User F Artist P Music F Group-of-feature User K Artist S Multiple similarity information (Mimic the CF/CB method) 2

Music-related Dataset Initial dataset Experimental dataset • 1,928,868 Listening records • 225,652 Listening records • 674,932 Users • 19,596 Users • 72,913 Songs • 30,260 Songs * keep only the users who have more than 10 listening records 3

LiveJournal Example Post Title Time Content Mood tag Listening music ( 132 pre-defined mood tags ) ( connect to Last.fm ) 4

Recommendation Strategy EchoNest API ANEW word Article( Lexicon User% (Emotional status) User Perspective (Audio features) Emotion Perspective Ranking Music Perspective Model Model the relationship 5

Match the Emotions M. Bradley and P. J. Lang, “Affective norms for english words ANEW: Instruction manual and affective ratings.” (A ff ective norm for English words) EchoNest API ANEW word Article( Lexicon User% TF-IDF weights (Emotional status) Description Vale lenc nce Arousa ousal l Dom omina inanc nce dream 6.73 4.53 5.53 User lonely 2.71 4.51 2.95 admired 7.74 6.11 7.53 Perspective (Audio features) good 7.47 5.43 6.41 Emotion Featur ture Dim imension nsion hate 2.12 6.95 5.05 Perspective Danceability 1 • Valance: pleasant to unpleasant Loudness 1 Key 1 Ranking Music • Arousal: calm to excited Mode 1 Perspective Model Tempo 1 • Dominance: control std_of_pitches 12 (3-dimensional vector) mean_of_pitches 12 Model the relationship std_of_timbre 12 mean_of_timbre 12 (53-dimensional vector) 6

Our Ranking Approach n n n κ X X X X y ( x ) = w 0 + ˆ w i x i + x j x j 0 v jf v j 0 f i =1 j =1 j 0 = j +1 f =1 Global Bias Feature Weights Weights of pair of features [Rendle, ICDM 2010] • Factorization Machine （ FM ） • A competitive model for ranking problem. • Easy to embed various kinds of feature in the data. • Capable of learning the interactions from pair of features . 7

The Data Format n n n κ X X X X y ( x ) = w 0 + ˆ w i x i + x j x j 0 v jf v j 0 f i =1 j =1 j 0 = j +1 f =1 Global Bias Feature Weights Weights of pair of features Age … User Music Artist Audio 4 1 0 0 0 12 1 0 0 0 1 0 0 0.2 0.7 0.1 … 0 1 0 0 0 12 0 1 0 0 0 1 0 0.8 0.1 0.1 … Rating 2 0 1 0 0 18 0 1 0 0 0 1 0 0.8 0.1 0.1 … 1 0 1 0 0 18 0 0 1 0 0 0 1 0.3 0.6 0.1 … … … … … … … … … … … … … … … … … … Y A B C D G A B C A B C A B C D 8

Features Interaction User Age Music Artist Audio ? Age … User Music Artist Audio 3 1 0 0 0 12 1 0 0 0 1 0 0 0.2 0.7 0.1 … 0 1 0 0 0 12 0 1 0 0 0 1 0 0.8 0.1 0.1 … Rating 2 0 1 0 0 18 0 1 0 0 0 1 0 0.8 0.1 0.1 … 1 0 1 0 0 18 0 0 1 0 0 0 1 0.3 0.6 0.1 … … … … … … … … … … … … … … … … … … Y A B C D G A B C A B C A B C D 9

Similarity Information • Enable the missing connections between the features User Age Music Artist Audio ? User B Similar Users User C Simulate the user-based KNN method. User F v n O ( T i ) ∩ O ( T j ) u X u s ij = d ( p, q ) = ( p i − q i ) 2 . t | O ( T i ) | 1 − α | O ( T j ) | α i =1 User K Modi fi ed version of L2 Distance Function Cosine Similarity 10

Example User User similarity 1 0 0 0 0 0 0 0.8 0 0.9 3 1 0 0 0 0 2 0 0 0.8 0 0.9 Rating … … … 0 1 0 0 0 0 0 0 0 0.85 0 1 0 1 0 0 0 0 0 0 0.85 0 … … … … … … … … … … … A B C D E A B C D E Music Music similarity 1 0 0 0 0 0 0.8 0 0.7 0 0 1 0 0 0 0.85 0 0 0 0 … … … 0 1 0 0 0 0.85 0 0 0 0 0 0 1 0 0 0.7 0 0 0 0.9 … … … … … … … … … … A B C D E A B C D E 11

User & Music Similarity ( Mean Average Precision ) • U: User Features MAP@10 Recall • US: User similarity • S: Song U + S (baseline) 0.3817 0.5216 • SS: Song similarity U + S + US 0.4310 0.5712 U + S + SS 0.4635 0.6194 U + S + US + SS 0.4712 0.6251 12

j Aj 1 � j Bj Feature Similarity • It is also applicable to other kinds of feature (under FM) User Age Music Artist Audio User B Age 15 Artist E Music D Audio K • Some users prefer the songs similar in melody, • Some users prefer the songs similar in lyrics. Audio-based KNN • … User C Age 18 Music G Artist L Age-based KNN Artist-based KNN User F Music F Music-based KNN User K User-based KNN (Mimic many state-of-the-art CF/CB algorithms) 13

Results for Feature Similarity • U: User Features MAP@10 Recall • S: Song U + S 0.3817 0.5216 U + S + A 0.5025 0.6538 • A: Artist U + S + A + AS 0.5125 0.6640 • M: Mood tag U + S + M 0.4635 0.6194 • Au: Audio information U + S + M + MS 0.4712 0.6251 U + S + Au 0.4254 0.5809 • VAD: Emotional status U + S + Au + AuS 0.4576 0.6114 • R: Region U + S + VAD 0.4438 0.5905 0.4511 0.5935 U + S + VAD + VADS U + S + R 0.4283 0.5723 U + S + R +RS 0.4382 0.5834 14

Some Issues Music Article Audio User Non-informative connection Music Features Age Mode VAD Region Tempo Age 15 Mood Emotion Loudness Age 16 Similar Users Danceability Age 18 … High dimensions Confused interactions • High computation cost • High complexity

Example for Group-of-Feature User Music Group A Mode Age Temp Group C User B Loudn User C Group B Dance x User F … User K • Reduce computation cost • Reduce complexity 16

Grouping Method n n n κ X X X X y ( x ) = w 0 + ˆ w i x i + x j x j 0 v jf v j 0 f i =1 j =1 j 0 = j +1 f =1 Interaction between each pair of features n n n κ X X X X y ( x ) = w 0 + ˆ w i x i + x j x j 0 v jf v j 0 f j 0 / i =1 f =1 j ∈ G ( j ) ∈ G ( j ) This way can eliminate the inner interaction (If the two features are in the same group) 17

Mean Average Precision n o i s i c e r P e g Reduce computation cost a r e & v A Keep the performance n a e M 18

Training Loss Training loss of Training Data 0 . 25 FMs r GroupingFM o r 0 . 24 r E e Fast convergence r a 0 . 23 u RMSE & q S Prevent over- fi tting n 0 . 22 a e M t 0 . 21 o o R 0 . 20 0 20 40 60 80 100 120 Iteration 19

Conclusion • Music ¡Recommendation ¡ • Match ¡the ¡music ¡by ¡capturing ¡the ¡ emotions ¡ • Recommendation ¡Model ¡ • Factorization ¡Machine ¡is ¡used ¡for ¡ranking ¡purpose ¡ • Integrate ¡the ¡ multiple ¡similarity ¡information ¡ • Apply ¡the ¡ group-­‑of-­‑feature ¡concept ¡to ¡FM ¡model Thank You! Chih-Ming Chen changecandy@gmail.com 20