EMNLP 2018 Brussels, Belgium, November 4, 2018 Modeling Empathy and Distress in Reaction to News Stories Sven Buechel 2* Anneke Buffone 1* Barry Slaff 1 Lyle Ungar 1 João Sedoc 1 * equal contribution 2 1 JULIE Lab World Well-Being Project University of Pennsylvania Friedrich-Schiller-Universität Jena http://www.julielab.de http://www.wwbp.org Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 1
EMNLP 2018 Brussels, Belgium, November 4, 2018 https://xkcd.com/660/ CC-BY-NC 2.5 Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 2
EMNLP 2018 Brussels, Belgium, November 4, 2018 EMPATHY TIPS FOR ENGINEERS https://xkcd.com/660/ CC-BY-NC 2.5 Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 3
EMNLP 2018 Brussels, Belgium, November 4, 2018 Why Empathy? • Crucial for how we experience the world and communicate within it • Great potential for social media analysis • Scarce work on empathy modeling in written language Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 4
EMNLP 2018 Brussels, Belgium, November 4, 2018 Why Empathy? • Crucial for how we experience the world and communicate within it • Great potential for social media analysis • Scarce work on empathy modeling in written language Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 5
EMNLP 2018 Brussels, Belgium, November 4, 2018 Shortcomings of Prior Work • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 6
EMNLP 2018 Brussels, Belgium, November 4, 2018 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 7
EMNLP 2018 Brussels, Belgium, November 4, 2018 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 8
EMNLP 2018 Brussels, Belgium, November 4, 2018 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 9
EMNLP 2018 Brussels, Belgium, November 4, 2018 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 10
EMNLP 2018 Brussels, Belgium, November 4, 2018 Empathic Concern vs. Personal Distress • Consensus in psych. that there are multiple forms of empathy • We follow most popular distinction by Batson et al. (1987) – Empathic Concern positive, other-focused – Personal Distress negative, self-focused • Post-hoc analysis shows both are distinct ( r =.45) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 11
EMNLP 2018 Brussels, Belgium, November 4, 2018 Empathic Concern vs. Personal Distress • Consensus in psych. that there are multiple forms of empathy • We follow most popular distinction by Batson et al. (1987) – Empathic Concern positive, other-focused – Personal Distress negative, self-focused • Post-hoc analysis shows that both are distinct ( r =.45) Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 12
EMNLP 2018 Brussels, Belgium, November 4, 2018 Problem Definition Message Empathy Distress I‘m sorry to hear about Dakota‘s parents. [...] 4.8 3.1 • Given a natural language utterance ... • ... predict empathy and distress of the writer from [1, 7] Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 13
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 14
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional r 1 r 2 r 3 r final Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 15
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 16
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and ratings Reliable through multi- • item scales Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 17
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and rating r Reliable through multi- • item scales Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 18
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and rating r Reliable through multi- • item scales Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 19
EMNLP 2018 Brussels, Belgium, November 4, 2018 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and rating r 1 r 2 r 3 r final Reliable through • multi-item scales Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 20
EMNLP 2018 Brussels, Belgium, November 4, 2018 Use of Multi-Item Scales Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 21
EMNLP 2018 Brussels, Belgium, November 4, 2018 Use of Multi-Item Scales Empathic Concern Mean Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 22
EMNLP 2018 Brussels, Belgium, November 4, 2018 Corpus Creation Process crowd workers take Qualtrics online survey manual review upload final stimulus - read articles corpus - rate E mpathy/ D istress - write M essage 418 online news articles (300 to 800 chars) 1680 ( M , E , D )-triples Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Ungar, and João Sedoc Modeling Empathy and Distress in Reaction to News Stories 23
Recommend
More recommend
