AMR dependency parsing with a typed semantic algebra Jonas - PowerPoint PPT Presentation

AMR dependency parsing with a typed semantic algebra Jonas Groschwitz*^, Matthias Lindemann*, Meaghan Fowlie*, Mark Johnson^, Alexander Koller* *Saarland University ^Macquarie University ACL 2018 Melbourne, Australia July 17

Matthias Lindemann Saarland University Meaghan Fowlie Saarland University Mark Johnson Macquarie University Alexander Koller Saarland University 2

Abstract Meaning Representation (AMR) Banarescu et al. 2013 spell try ARG1 ARG0 ARG1 A R G 0 witch cast The witch tried to cast a spell 3

Abstract Meaning Representation (AMR) Banarescu et al. 2013 spell try ARG1 ARG0 ARG1 A R G 0 witch cast Parsing The witch tried to cast a spell 4

5

Classic AMR parser (e.g. JAMR 2014) Step 1: Predict nodes Step 2: Predict edges spell spell try try ARG1 ARG0 ARG1 A R G 0 witch witch cast cast The witch tried to cast a spell 6

Not just nodes and edges spell try ARG1 ARG0 ARG1 A R G 0 witch cast 7

Not just nodes and edges Control verb spell try ARG1 ARG0 ARG1 A R G 0 cast witch Transitive verb Noun 7

Hidden compositional structure Principle of compositionality: the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them. APP S APP O APP O O try A ARG1 R ARG0 G ⊥ ⊥ ⊥ 1 witch spell A R G 0 cast S S O[S] The witch tried to cast a spell 8

Hidden compositional structure Principle of compositionality: the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them. APP S APP O APP O O try witch A ARG1 R ARG0 G 1 spell A R G 0 cast S S O[S] The witch tried to cast a spell 8

Hidden compositional structure Principle of compositionality: the meaning of a complex expression is determined by the meanings of its constituent expressions and the rules used to combine them. APP S APP O APP O O try witch A ARG1 R ARG0 G 1 spell A R G 0 cast S S O[S] The witch tried to cast a spell • Widely accepted in linguistics, long history (Frege 1800s) • Use this knowledge to guide machine learning! 8

APP S APP O spell try APP O witch cast O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] The witch tried to cast a spell 9

APP S APP O spell try APP O witch cast O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] difficult The witch tried to cast a spell 9

APP S APP O spell try APP O witch cast O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] equivalent App O App S App O dependencies! O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] easy (easier) The witch tried to cast a spell 9

APP S APP O spell try APP O witch cast O try ARG1 1 0 G G R R A witch A spell ARG0 cast Part 1 S S O[S] equivalent App O App S App O dependencies! O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] Part 2 easy (easier) The witch tried to cast a spell 9

spell try Apply-Modify (AM) Algebra G. et al, IWCS 2017 witch cast O 1 G R A A R G 0 S cast *HR algebra, Courcelle & Engelfriet 2012 10

spell try Apply-Modify (AM) Algebra G. et al, IWCS 2017 witch cast • Empty argument slots are labeled with sources* S,O,… (subject, object,…) O 1 G R A A R G 0 S cast *HR algebra, Courcelle & Engelfriet 2012 10

spell try Apply-Modify (AM) Algebra G. et al, IWCS 2017 witch cast • Empty argument slots are labeled with sources* S,O,… (subject, object,…) • Have ‘apply’ operation for each source, e.g. APP O App O spell = ARG1 O 1 A R G 0 G S cast R spell A A R G 0 S cast head argument *HR algebra, Courcelle & Engelfriet 2012 10

spell try Typed AM Algebra witch cast spell ARG1 A R G 0 S cast cast Has type [S] 11

spell try Typed AM Algebra witch cast try A ARG0 R G 1 spell O[S] S Object must have type [S] ARG1 A R G 0 S cast cast Has type [S] 11

spell try Typed AM Algebra witch cast App O try spell try = A ARG0 R ARG1 G ARG0 ARG1 1 spell O[S] S S A R G 0 cast Object must have type [S] ARG1 Matching sources automatically merge A R G 0 S cast cast Has type [S] 11

spell try Apply-Modify Algebra witch cast App S spell try = ARG1 ARG0 ARG1 try spell A R G 0 witch cast ARG1 ARG0 ARG1 witch The witch tried to cast a spell S A R G 0 cast 12

Types control reentrancies ✘ App O = sleep ARG0 O 1 G sleep R A 1 ARG0 G R A R G 0 A S cast S A R G 0 S cast *cast to sleep Has type [S] 13

Types control reentrancies ✘ App O = sleep ARG0 O 1 G sleep R A 1 ARG0 G R A R G 0 A S cast S A R G 0 S cast *cast to sleep Has type [S] Object must have type [ ] 13

spell try Types control reentrancies witch cast App O ✘ try try = A ARG0 R G A 1 ARG0 R G 1 S witch O[S] S witch Object must have type [S] *tried to witch Has type [ ] 14

AM Dependency Trees dependencies define operations, but not their order App S App O O ARG1 witch spell ARG0 cast S The witch cast a spell 15

AM Dependency Trees dependencies define operations, but not their order App S App O O ARG1 witch spell ARG0 cast S The witch cast a spell here: order does not matter 15

AM Dependency Trees App O App S App O O try ARG1 ARG1 ARG0 witch spell ARG0 cast S S O[S] The witch tried to cast a spell here: need APP O before APP S to get reentrancies 16

AM Dependency Trees App O App S App O O try ARG1 ARG1 ARG0 witch spell ARG0 cast S S O[S] The witch tried to cast a spell here: need APP O before APP S to get reentrancies • Always need to resolve reentrancies first • Types encode reentrencies ➡ use type system to determine operation order 16

AM Dependency Trees App O App S App O O try ARG1 ARG1 ARG0 witch spell ARG0 cast S S O[S] The witch tried to cast a spell Building instructions for an AMR that we know how to predict 16

APP S spell try APP O APP O witch cast O try ARG1 1 0 G G R Part 1 R A witch A spell ARG0 cast S S O[S] equivalent App O App S App O O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] Part 2 easy (easier) The witch tried to cast a spell 17

Model 1. Supertagging: score graph fragments for each word O try ARG1 1 0 G ⊥ G R R A witch A spell ARG0 cast S S O[S] 2. Dependency model: score operations ✘ App O App S 3. Decoding : find highest-scoring well-typed tree App O App S App O O try ARG1 1 0 G G R R A witch A spell ARG0 cast S S O[S] The witch tried to cast a spell 18

1. Supertagging E.g. Lewis et al. (2014) for CCG try A R train to predict ARG0 G 1 O[S] S probability distribution P 0 P 1 P 2 P n over graph lexicon … bidirectional LSTM word embeddings w 0 w 1 w 2 w n The witch tried … spell 19

2. Dependency Model Kiperwasser & Goldberg (2016) for syntactic dependencies App S train to predict witch tried probability distribution over operations P 2 1 → … bidirectional LSTM word embeddings w 0 w 1 w 2 w n The witch tried … spell 20

AMR Corpus Required training data spell App S App O ARG1 O 1 A R G 0 G witch cast R A witch spell ARG0 S cast The witch cast a spell The witch cast a spell 21

AMR Corpus Required training data spell App S App O ARG1 O 1 A R G 0 G witch cast R A witch spell ARG0 S cast The witch cast a spell The witch cast a spell O Heuristics 1 G R A witch spell ARG0 S cast • Alignments The witch cast a spell • Attaching edges • Source names • Source annotations 21

AMR Corpus Required training data spell App S App O ARG1 determine O 1 A R G 0 G witch cast R A witch spell ARG0 S cast The witch cast a spell The witch cast a spell O Heuristics 1 G R A witch spell ARG0 S cast • Alignments The witch cast a spell • Attaching edges • Source names • Source annotations 21

3. Decoding Find the best well-typed dependency tree 22

3. Decoding Find the best well-typed dependency tree • ill-typed trees do not evaluate to AMRs • ill-typed trees to not match our linguistic intuitions 22