Parsing of natural language sentences to syntactic and semantic - PowerPoint PPT Presentation

Parsing of natural language sentences to syntactic and semantic graph representations Abschlussvortrag zum Forschungsprojekt Pius Meinert April 13, 2018

Overview Graph Representations Corpora Parsing Techniques Parser 1

Overview Graph Representations Corpora Parsing Techniques Parser Semantic: AMR, UCCA, depen- dency graphs Syntactic: Constituency tree derived, Use of syntactic infor- mation 1

Overview Graph Representations Corpora Parsing Techniques Parser AMR, UCCA, SemEval-2014/-2015: dependency graphs, Penn Tree- bank, TIGER Corpus 1

Overview Graph Representations Corpora Parsing Techniques Parser Maximum Subgraph Transition-Based Synchronous HRG 1

Overview Graph Representations Corpora Parsing Techniques Parser 1

Abstract Meaning Representation (AMR) [Ban+13] 2

Abstract Meaning Representation (AMR) [Ban+13] contrast possible say You I person dream include only — arg0-of arg1 arg2 arg3 2

Abstract Meaning Representation (AMR) [Ban+13] dream arg3 arg2 arg1 arg0-of — only include person rooted, directed, I You say possible contrast leaf-labeled edge-labeled and 2

Tree Defjnition unique directed path. [KJ15; KO16] 3 A tree is a directed graph G = ( V , A ) that has a vertex r , named root, such that every vertex v ∈ V is reachable from r via a

Dependency Graph [KJ15] bv arg3 arg1 arg1 arg2 arg2 bv arg1 arg1 itself The fertilizing from plant a prevent can thus gene 4

Dependency Graph [KJ15] arg1 singletons. [KO16] vertices. Nodes with in- and out-degree zero are called There exists an undirected path between every two pairs of Connectedness: arg3 arg1 arg1 arg2 arg2 bv arg1 unconnected bv itself fertilizing from plant a prevent can thus gene The 4

Dependency Graph [KJ15] arg1 root in a tree. [KO16] Nodes of in-degree zero, a graph’s equivalent to the unique Top nodes: arg3 arg1 arg1 arg2 arg2 bv arg1 bv unconnected, multi-rooted itself fertilizing from plant a prevent can thus gene The 4

Dependency Graph [KJ15] arg1 Nodes with in-degree greater than one. [WXP15; DCS17; BB17] Reentrant nodes: arg3 arg1 arg1 arg2 arg2 bv arg1 bv unconnected, multi-rooted, reentrancy itself fertilizing from plant a prevent can thus gene The 4

Dependency Graph - Noncrossing [KJ15] bv arg3 arg1 arg1 arg2 arg2 bv arg1 arg1 itself The fertilizing from plant a prevent can thus gene 5

Dependency Graph - Noncrossing [KJ15] itself arg3 arg1 arg1 arg2 arg2 bv arg1 arg1 bv fertilizing from plant a prevent can thus gene The (CCGbank, Prage Semantic Dependencies, etc.). [KJ15; SCW17] 5 Coverage ranges from 48 % to 78 % for various graph banks

Dependency Graph - 1-Endpoint-Crossing [PKM13] das mer em Hans es huus hälfed aastriche 6

Dependency Graph - 1-Endpoint-Crossing [PKM13] das mer em Hans es huus hälfed aastriche 6

Dependency Graph - 1-Endpoint-Crossing [PKM13] used in [Cao+17]. das mer em Hans es huus hälfed aastriche 6 Account for 95 . 7 − 97 . 7 % of the dependency structures that are

Dependency Graph - Book Embedding [SCW17] arg2 arg2 arg2 arg2 arg1 arg1 arg1 arg1 The arg1 buy to wants Mark that company 7

Dependency Graph - Book Embedding [SCW17] arg2 arg2 arg2 arg2 arg1 arg1 arg1 arg1 The arg1 buy to wants Mark that company 7

Dependency Graph - Book Embedding [SCW17] arg2 arg2 arg2 arg2 arg1 arg1 arg1 arg1 The arg1 buy to wants Mark that company 7

Dependency Graph - Book Embedding [SCW17] wants arg2 arg2 arg2 arg1 arg1 arg1 arg2 arg1 arg1 buy to Mark Coverage with respect to different page numbers: that company The 3 2 1 coverage PN 7 48 − 78 % 20 − 49 % 0 . 3 − 1 . 7 %

Evaluation Metric - AMR Representations AMR graph go–1 boy want-01 ARG0 instance instance instance 8 ARG0 ARG1

Evaluation Metric - AMR Representations AMR graph go–1 boy want-01 ARG0 instance instance instance PENMAN notation (w / want-01 :arg0 (b / boy) :arg1 (g / go-01) :arg0 b) 8 ARG0 ARG1

Evaluation Metric - AMR Representations instance ARG0(c, b) logic format AMR graph instance instance ARG0 want-01 boy go–1 8 instance(a, want-01) ∧ instance(b, boy) ∧ instance(c, go-01) ∧ ARG0 ARG1 ARG0(a, b) ∧ ARG1(a, c) ∧

Evaluation Metric - Smatch [CK13] The boy wants the football ARG1(x, z) The boy wants to go ARG0(c, b) 9 instance(x, want-01) ∧ instance(a, want-01) ∧ instance(y, boy) ∧ instance(b, boy) ∧ instance(z, football) ∧ instance(c, go-01) ∧ ARG0(x, y) ∧ ARG0(a, b) ∧ ARG1(a, c) ∧

Evaluation Metric - Smatch [CK13] The boy wants to go inter-annotator agreement study: ARG0(c, b) The boy wants the football 9 ARG1(x, z) instance(x, want-01) ∧ instance(a, want-01) ∧ instance(y, boy) ∧ instance(b, boy) ∧ instance(z, football) ∧ instance(c, go-01) ∧ ARG0(x, y) ∧ ARG0(a, b) ∧ ARG1(a, c) ∧ Smatch score ranges from 0 . 79 to 0 . 83.

Graph Parsing Techniques Maximum Subgraph “all pairs” approach [BM06] - Consider all possible (weighted) arcs and fjnd the maximum spanning connected subgraph. Transition-based “stepwise” approach [BM06] - Build the graph step by step by applying transitions to the current confjguration. Synchronous Hyperedge Replacement Grammar (SHRG) HRGs as “an intuitive generalization of context free grammars (CFGs) from strings to hypergraphs.” [Jon+12; Hab92] 10

Maximum Subgraph - Problem Defjnition[SCW17] Input Implicit Output Example Maximum Subgraph = Maximum Spanning Tree 11 directed, weighted graph G = ( V , A ) (complete) sentence s , class of graphs G subgraph G ′ = ( V , A ′ ⊆ A ) with maximum total weight such that G ′ belongs to G G ′ ( s ) = arg max H ∈G ( s , G ) � p ∈ H ScorePart ( s , p ) if class of graphs G is the class of all trees,

Maximum Subgraph - Learning and Features [MN07] 12 G ′ ( s ) = arg max H ∈G ( s , G ) � p ∈ H ScorePart ( s , p )

Maximum Subgraph - Learning and Features [MN07] Global learning Optimize entire graph score, not only single arc attachments. 12 G ′ ( s ) = arg max H ∈G ( s , G ) � p ∈ H ScorePart ( s , p )

Maximum Subgraph - Learning and Features [MN07] Global learning Optimize entire graph score, not only single arc attachments. Local features Restricted to a limited number of arcs (to keep inference and learning tractable). 12 G ′ ( s ) = arg max H ∈G ( s , G ) � p ∈ H ScorePart ( s , p )

JAMR [Fla+14] First published AMR parser. It solves the task by means of two phases: Concept identifjcation: Match spans of words to concept graph fragments. Relation identifjcation: Find the maximum spanning connected subgraph over those graph fragments. 13

JAMR - Concept Identifjcation Phase [Fla+14] visit-01 op3 op2 op1 name “City” “York” “New” city name 14 The want-01 boy City York New visit to wants boy ∅ ∅

JAMR - Concept Identifjcation Phase [Fla+14] b4 op3 op2 op1 name “City” “York” “New” city name visit-01 want-01 boy The b5 b6 b3 b2 boy wants to visit New York City c: w: b: k = 6 b0 b1 14 ∅ ∅ i = 1 θ ⊤ f ( w b i − 1 : b i , b i − 1 , b i , c i ) score ( b , c ; θ ) = � k Solve by dynamic programming: O ( n 2 ) .

JAMR - Relation Identifjcation Phase [Fla+14] 1. Initialization: Include all edges and vertices given by the concept identifjcation phase. 2. Pre-processing: Reduce the set of edges considered to one edge per pair of nodes: Either the edge given by the fjrst phase or the highest scoring one. 3. Core algorithm: First, add all positive edges and then greedily add the least negative edge that connects two components until the graph is connected. 15

sentence w to an initial state. Transition-Based - Transition System [WXP15] 16 A transition system for parsing is a tuple S = ( S , T , s 0 , S t ) where • S is a set of parsing states (confjgurations). • T is a set of parsing actions (transitions), each of which is a function t : S → S . • s 0 is an initialization function, mapping each input • S t ⊆ S is a set of terminal states.

Transition-Based - Parsing Algorithm [WXP15] Output: parsed graph G 3: 4: 5: 6: end while 7: return G 17 Input: sentence w = w 0 ... w n 1: s ← s 0 ( w ) 2: while s / ∈ S t do T ← all possible actions according to s bestT ← arg max t ∈T score ( t , s ) s ← apply bestT to s

Transition-Based - Learning and Features [MN07] 18 bestT ← arg max t ∈T score ( t , s )

Transition-Based - Learning and Features [MN07] Local learning Optimization only for single transitions, not transition sequences. 18 bestT ← arg max t ∈T score ( t , s )

Transition-Based - Learning and Features [MN07] Local learning Optimization only for single transitions, not transition sequences. Global features Features may be based on whole graph built so far/entire transition history. 18 bestT ← arg max t ∈T score ( t , s )