WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Modeling Empathy and Distress in Written Language Sven Buechel Jena University Language and Information Engineering (JULIE) Lab Friedrich-Schiller-University Jena, Jena, Germany https://julielab.de Sven Buechel Modeling Empathy and Distress 1
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 The People Behind This Work João Sedoc Barry Slaff Yoni Nachmany Anneke Buffone Lyle Ungar Sven Buechel Modeling Empathy and Distress 2
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Outline • A Gold Standard for Empathy and Distress • Generating Lexica for Empathy and Distress • WASSA 2020 Shared Task on Empathy and Distress Sven Buechel Modeling Empathy and Distress 3
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Outline Ø A Gold Standard for Empathy and Distress • Generating Lexica for Empathy and Distress • WASSA 2020 Shared Task on Empathy and Distress Sven Buechel Modeling Empathy and Distress 4
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Why Empathy? https://xkcd.com/660/ CC-BY-NC 2.5 Sven Buechel Modeling Empathy and Distress 5
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Why Empathy? EMPATHY TIPS FOR ENGINEERS https://xkcd.com/660/ CC-BY-NC 2.5 Sven Buechel Modeling Empathy and Distress 6
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Why Empathy? Psychology & Medicine well-being, longevity but also physiological stress and emotional pain Social Science charity and volunteering AI natural language user interfaces Sven Buechel Modeling Empathy and Distress 7
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Prior Work Is Scarce What we talk about today Xiao et al. (2012) Gibson et al. (2015) Khanpour et al. (2017) Buechel et al. (2018) written spoken Sven Buechel Modeling Empathy and Distress 8
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Shortcomings of Prior Work • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel Modeling Empathy and Distress 9
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel Modeling Empathy and Distress 10
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel Modeling Empathy and Distress 11
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel Modeling Empathy and Distress 12
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Contribution • No publicly available gold standard Ø First publicly available gold standard (CC-BY) • 3rd person ground truth Ø Annotations by the experiencer (new annotation methodology) • Disconnected from psychological theory and research Ø Distinguish two different types of empathy (in line with psych. research) Sven Buechel Modeling Empathy and Distress 13
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Empathic Concern vs. Personal Distress Distinction by Batson et al. (1987) Sven Buechel Modeling Empathy and Distress 14
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Empathic Concern vs. Personal Distress Distinction by Batson et al. (1987) Empathic Concern positive, other-focused: “feeling FOR someone” Sven Buechel Modeling Empathy and Distress 15
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Empathic Concern vs. Personal Distress Distinction by Batson et al. (1987) Empathic Concern Personal Distress positive, other-focused: “feeling FOR someone” negative, self-focused: “feeling WITH someone” Sven Buechel Modeling Empathy and Distress 16
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Empathic Concern vs. Personal Distress Distinction by Batson et al. (1987) Empathic Concern Personal Distress positive, other-focused: “feeling FOR someone” negative, self-focused: “feeling WITH someone” Post-hoc analysis shows that both are distinct ( r =.45) Sven Buechel Modeling Empathy and Distress 17
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Problem Definition Message Empathy Distress I‘m sorry to hear about Dakota‘s parents. [...] 4.8 3.1 • Given a natural language utterance ... • ... predict empathy and distress of the writer from [1, 7] • Momentary state, not personality trait Sven Buechel Modeling Empathy and Distress 18
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional Sven Buechel Modeling Empathy and Distress 19
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional r 1 r 2 r 3 r final Sven Buechel Modeling Empathy and Distress 20
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Sven Buechel Modeling Empathy and Distress 21
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and ratings Reliable through multi- • item scales Sven Buechel Modeling Empathy and Distress 22
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and rating r Reliable through multi- • item scales Sven Buechel Modeling Empathy and Distress 23
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and rating r Reliable through multi- • item scales Sven Buechel Modeling Empathy and Distress 24
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Annotation Methodology Traditional Annotators “guess“ • r 1 r 2 r 3 experiencer‘s feelings Models biased • Experiencer r final towards annotators Proposed Annotation as psych. • experiment Stimulus Experiencer Experiencer produces • message and rating r 1 r 2 r 3 r final Reliable through • multi-item scales Sven Buechel Modeling Empathy and Distress 25
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Use of Multi-Item Scales Sven Buechel Modeling Empathy and Distress 26
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 Use of Multi-Item Scales Empathic Concern Mean Sven Buechel Modeling Empathy and Distress 27
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 E MPATHIC R EACTIONS : Creation Process crowd workers take Qualtrics online survey manual review upload final stimulus - read articles corpus - rate E mpathy/ D istress - write M essage 418 online news articles (300 to 800 chars) 1860 ( M , E , D )-triples Sven Buechel Modeling Empathy and Distress 28
WASSA 2019 Invited Talk Minneapolis, MN, USA, June 6, 2019 E MPATHIC R EACTIONS : Creation Process crowd workers take Qualtrics online survey manual review upload final stimulus - read articles corpus - rate E mpathy/ D istress - write M essage 418 online news articles (300 to 800 chars) 1860 ( M , E , D )-triples Split-half reliability around r=.9 for both empathy and distress J Sven Buechel Modeling Empathy and Distress 29
Recommend
More recommend
