Introduction The Entity Grid Evaluation Natural Language Understanding Lecture 17: Entity-based Coherence Mirella Lapata School of Informatics University of Edinburgh mlap@inf.ed.ac.uk March 24, 2015 Mirella Lapata Natural Language Understanding 1
Introduction The Entity Grid Evaluation 1 Introduction 2 The Entity Grid Discourse Representation Entity Transitions Ranking Model 3 Evaluation Text Ordering Summarization Reading: Barzilay and Lapata (2008). Mirella Lapata Natural Language Understanding 2
Introduction The Entity Grid Evaluation Coherence in Text Coherence: is a property of well-written texts; makes them easier to read and understand; ensures that sentences are meaningfully related; and that the reader can work out what expressions mean; the text is thematically organized; temporally organized; rather than a random concatenation of sentences. In this lecture, we will discuss Barzilay and Lapata’s (2008) entity-based model of coherence. Mirella Lapata Natural Language Understanding 3
Introduction The Entity Grid Evaluation Coherence in Text Summary B Summary A Former Chilean dictator Augusto Pinochet, was arrested in London Britain said he did not have on 14 October 1998. Pinochet, diplomatic immunity. The 82, was recovering from surgery. Spanish authorities contend that The arrest was in response to an Pinochet may have committed extradition warrant served by a crimes against Spanish citizens in Spanish judge. Pinochet was Chile. Baltasar Garzon filed a charged with murdering request on Wednesday. Chile thousands, including many said, President Fidel Castro said Spaniards. Pinochet is awaiting a Sunday he disagreed with the hearing, his fate in the balance. arrest in London. American scholars applauded the arrest. Mirella Lapata Natural Language Understanding 4
Introduction The Entity Grid Evaluation Entity-based Coherence The way entities are introduced and discussed influences coherence (Grosz et al., 1995). Entities in an utterance are ranked according to salience. Is an entity pronominalized or not? Is an entity in a prominent syntactic position? Each utterance has one center ( ≈ topic or focus). Coherent discourses have utterances with common centers. Entity transitions capture degrees of coherence (e.g., in Centering theory continue > shift ). Notions of salience, utterance, ranking are left unspecified. Mirella Lapata Natural Language Understanding 5
Introduction The Entity Grid Evaluation Entity-based Local Coherence John went to his favorite John went to his favorite music store to buy a piano. music store to buy a piano. He had frequented the store It was a store John had fre- for many years. quented for many years. He was excited that he could He was excited that he could finally buy a piano. finally buy a piano. He arrived just as the store It was closing just as John was closing for the day. arrived. Mirella Lapata Natural Language Understanding 6
Introduction The Entity Grid Evaluation Entity-based Local Coherence John went to his favorite John went to his favorite music store to buy a piano. music store to buy a piano. He had frequented the store It was a store John had fre- for many years. quented for many years. He was excited that he could He was excited that he could finally buy a piano. finally buy a piano. He arrived just as the store It was closing just as John was closing for the day. arrived. Mirella Lapata Natural Language Understanding 6
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid Can we compute a discourse representation automatically? Does it capture coherence characteristics? What linguistic information matters for coherence? Is it robust across domains and genres? What is an appropriate coherence model? View coherence rating as a machine learning problem. Learn a ranking function without manual involvement. Apply to text-to-text generation tasks. Inspired from entity-based theories, not a direct implementation of any theory in particular. Mirella Lapata Natural Language Understanding 7
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid 1 Former Chilean dictator Augusto Pinochet, was arrested in London on 14 October 1998. 2 Pinochet, 82, was recovering from surgery. 3 The arrest was in response to an extradition warrant served by a Spanish judge. 4 Pinochet was charged with murdering thousands, including many Spaniards. 5 He is awaiting a hearing, his fate in the balance. 6 American scholars applauded the arrest. Mirella Lapata Natural Language Understanding 8
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid ✞ ☎ 1 Former Chilean dictator Augusto Pinochet , was arrested in ✝ ✆ ✞ ☎ ✞ ☎ London on 14 October 1998. ✝ ✆ ✝ ✆ ✞ ☎ Pinochet , 82, was recovering from ✄ � 2 surgery . ✝ ✆ ✂ ✁ ✞ ☎ ✞ ☎ was in ✄ � 3 The arrest response to an extradition warrant ✝ ✆ ✝ ✆ ✂ ✁ ✞ ☎ served by a Spanish judge . ✝ ✆ ✞ ☎ ✞ ☎ 4 Pinochet was charged with murdering thousands , includ- ✝ ✆ ✝ ✆ ✞ ☎ ing many Spaniards . ✝ ✆ ✞ ☎ ✞ ☎ ✞ ☎ 5 He is awaiting a hearing , Pinochet’s fate in ✝ ✆ ✝ ✆ ✝ ✆ ✞ ☎ the balance . ✝ ✆ ✞ ☎ ✄ � 6 American scholars applauded the arrest . ✝ ✆ ✂ ✁ Mirella Lapata Natural Language Understanding 9
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid ✞ ☎ 1 Former Chilean dictator Augusto Pinochet S , was arrested ✝ ✆ ✞ ☎ ✞ ☎ in London X on 14 October X 1998. ✝ ✆ ✝ ✆ ✞ ☎ Pinochet S , 82, was recovering from ✄ � 2 surgery X . ✝ ✆ ✂ ✁ ✞ ☎ ✞ ☎ The arrest S was in ✄ � 3 response X to an extradition warrant X ✝ ✆ ✝ ✆ ✂ ✁ ✞ ☎ served by a Spanish judge S . ✝ ✆ ✞ ☎ ✞ ☎ 4 Pinochet O was charged with murdering thousands O , in- ✝ ✆ ✝ ✆ ✞ ☎ cluding many Spaniards O . ✝ ✆ ✞ ☎ ✞ ☎ ✞ ☎ 5 Pinochet S is awaiting a hearing O , his fate X in ✝ ✆ ✝ ✆ ✝ ✆ ✞ ☎ the balance X . ✝ ✆ ✞ ☎ ✄ � 6 American scholars S applauded the arrest O . ✝ ✆ ✂ ✁ Mirella Lapata Natural Language Understanding 9
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid 1 Pinochet S London X October X 2 Pinochet S surgery X 3 arrest S response X warrant X judge O 4 Pinochet O thousands O Spaniards O 5 Pinochet S hearing O Pinochet X fate X balance X 6 scholars S arrest O Mirella Lapata Natural Language Understanding 10
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid Thousands Pinochet Spaniards October Warrant Scholars Balance London Surgery Hearing Arrest Judge Fate 1 2 3 4 5 6 Mirella Lapata Natural Language Understanding 11
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid Extradition Thousands Spaniards Pinochet Scholars October Warrant Balance London Surgery Hearing Arrest Judge Fate 1 S 2 S 3 – 4 O 5 S 6 – Mirella Lapata Natural Language Understanding 11
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid Extradition Thousands Spaniards Pinochet Scholars October Warrant Balance London Surgery Hearing Arrest Judge Fate 1 S X X – – – – – – – – – – – 2 S – – X – – – – – – – – – – 3 – – – – S X X O – – – – – – 4 O – – – – – – – O O – – – – 5 S – – – – – – – – – O X X – 6 – – – – O – – – – – – – – S Mirella Lapata Natural Language Understanding 11
Introduction Discourse Representation The Entity Grid Entity Transitions Evaluation Ranking Model The Entity Grid S X X – – – – – – – – – – – S – – X – – – – – – – – – – – – – – S X X O – – – – – – O – – – – – – – O O – – – – S – – – – – – – – – O X X – – – – – O – – – – – – – – S Mirella Lapata Natural Language Understanding 12
Recommend
More recommend
