sentibert
play

SentiBERT: A Transferable Transformer-based Architecture for - PowerPoint PPT Presentation

Apr 06, 2023 •1.09k likes •1.28k views

SentiBERT: A Transferable Transformer-based Architecture for Compositional Sentiment Semantics Da Yin 1 , Tao Meng 2 , Kai-Wei Chang 2 1 Peking University 2 University of California, Los Angeles 1 Motivation Sentiment composition is

  1. SentiBERT: A Transferable Transformer-based Architecture for Compositional Sentiment Semantics Da Yin 1 , Tao Meng 2 , Kai-Wei Chang 2 1 Peking University 2 University of California, Los Angeles 1

  2. Motivation ● Sentiment composition is challenging. positive . neutral negative Frenetic but not really funny Frenetic but not really funny. 2

  3. Motivation ● How to encode sentiment composition in a contextual encoder? ● Can semantic composition learned from SST transfer to related tasks? Better capture = + sentiment composition 3

  4. Model Sentiment Phrase Node BERT Semantics Prediction Composition 4

  5. Model BERT Sentiment Semantics Composition ● Layer 1: Attention to Tokens Phrase Node ● Layer 2: Attention to Children Prediction 5

  6. Training Objectives 6

  7. Experiments ● Tasks: ○ SST-phrase Evaluated under supervised learning ○ SST-5 protocol ○ SST-2, SST-3 ○ Twitter Sentiment Analysis Test transferability ○ Contextual Emotion Detection (EmoContext) ○ Emotion Intensity Classification (EmoInt) 7

  8. Experiments ● Results: ○ For sentiment semantic composition: SST-phrase (Accuracy) SST-5 (Accuracy) 0.7 0.58 0.57 0.69 0.56 0.68 0.55 0.67 0.54 0.53 0.66 0.52 0.65 0.51 0.5 0.64 0.49 0.63 0.48 0.62 0.47 BERT BERT w/ Tree-LSTM BERT BERT w/ Tree-LSTM SentiBERT RoBERTa SentiBERT RoBERTa SentiBERT w/ RoBERTa SentiBERT w/ RoBERTa More results and discussion are in the paper 8

  9. Experiments ● Results: ○ For transferability: EmoInt (Pearson Correlation) EmoContext (F1) 0.675 0.75 0.67 0.745 0.665 0.74 0.66 0.655 0.735 0.65 0.73 0.645 0.64 0.725 BERT SentiBERT BERT SentiBERT RoBERTa SentiBERT w/ RoBERTa RoBERTa SentiBERT w/ RoBERTa More results and discussion are in the paper 9

  10. Analysis -- Performance v.s. Sentiment Switch positive . neutral negative Frenetic but not really funny ● Local difficulty: the number of sentiment switches between a phrase and its children ● Global difficulty: the number of sentiment switches in the entire constituency tree 10

  11. Analysis ● Results: Local Difficulty Global Difficulty More results and discussion are in the paper 11

  12. Case Study More examples are in the paper 12

  13. Conclusion ● We present SentiBERT to better capture compositional sentiment semantics ● SentiBERT can transfer the compositional sentiment semantics learned on SST to other related tasks Thanks! GitHub: https://github.com/WadeYin9712/SentiBERT 13

Recommend

More recommend