n eu g en
play

N EU G EN Text Generation from Meaning Representations Yannis - PowerPoint PPT Presentation

Dec 11, 2022 •118 likes •879 views

N EU G EN Text Generation from Meaning Representations Yannis Konstas Joint work with Mark Yatskar, Luke Zettlemoyer and Yejin Choi (UW) ~ Yonatan Bisk and Daniel Marcu (ISI) Motivation Motivation Machine-generated

  1. N EU G EN � Text Generation from Meaning Representations Yannis Konstas � � Joint work with Mark Yatskar, Luke Zettlemoyer and Yejin Choi (UW) ~ Yonatan Bisk and Daniel Marcu (ISI)

  2. Motivation

  3. Motivation Machine-generated Representation

  4. Motivation Machine-generated Representation

  5. Motivation Machine-generated Representation

  6. Motivation Machine-generated Representation hk source block: ms target block: W small pos RP: scale:

  7. Motivation Machine-generated Representation hk source block: Place the heineken block west of the ms target block: mercedes block. W small pos RP: scale:

  8. Motivation Machine-generated Representation hk source block: ups target block: W big pos RP: scale:

  9. Motivation Machine-generated Representation hk source block: Place the heineken block in the first ups open space to the left of ups block. target block: W big pos RP: scale:

  10. Motivation

  11. Motivation Machine-generated Representation

  12. Motivation Machine-generated Representation Sentence Compression

  13. Motivation Machine-generated Representation Sentence Compression Sentence Fusion

  14. Motivation Machine-generated Representation Sentence Compression Sentence Fusion Paraphrasing

  15. NLG Pipeline Communicative Goal A A A A m Input Content Planning Content Selection Document Planning Sentence Planning Framework Lexicalization Reordering/Linearization Splitting/Aggregation Surface Realisation Text

  16. NLG Pipeline Communicative Goal A A A A m Input Content Planning Content Selection Document Planning Sentence Planning Framework Lexicalization Reordering/Linearization Splitting/Aggregation Surface Realisation Text

  17. NLG Pipeline Communicative Goal A A A A m Input Content Planning Content Selection Document Planning Sentence Planning Framework - Neural Encoder Lexicalization - RNN Decoder Reordering/Linearization - Graph Alignment Splitting/Aggregation Surface Realisation Text

  18. Joint Generator ? ? Temperature Cloud Sky Cover S ? S Time Min Mean Max Time Percent (%) 06:00-21:00 9 15 21 06:00-09:00 25-50 ... Cloudy, with temperatures ... 09:00-12:00 50-75 S Training between 10 and 20 degrees. S South wind around 20 mph. Wind Speed S Wind Direction ... ... Time Mode Time Min Mean Max 06:00-21:00 S 06:00-21:00 15 20 30 ... F 0 , 1 (temp 1 ,min) S → R( start ) (1) PCFG Grammar FS 0 , 1 (temp 1 ,start) R( r i .t ) → FS( r j , start )R( r j .t ) F 0 , 1 (temp 1 ,max) R( r i .t ) → FS( r j , start ) FS( r, r.f i ) → F( r, r.f j )FS( r, r.f j ) FS( r, r.f i ) → F( r, r.f j ) F 0 , 2 (temp 1 ,min) F( r, r.f ) → W( r, r.f )F( r, r.f ) FS 0 , 2 (temp 1 ,start) (2) Hypergraph F( r, r.f ) → W( r, r.f ) F 0 , 2 (temp 1 ,max) Representation W( r, r.f ) → α W( r, r.f ) → g( f.v ) FS 1 , 2 (temp 1 ,start)   mostly cloudy the morning ? 11 am mostly cloudy after ?   (3) k-best decoding   mostly cloudy then becoming ? FS 0 , 5 (skyCover 1 .t ,start)   via integration · · ·   mostly cloudy mostly clouds     Testing cloudy ,   F 0 , 2 (skyCover 1 .t ,%) W 4 , 5 (skyCover 1 .t ,time) · · ·   morning 11 am W 0 , 1 (skyCover 1 .t ,%) W 1 , 2 (skyCover 1 .t ,%)     after       mostly mostly · · · cloudy cloudy         Konstas and Lapata, 2012, 2013 sunny sunny     · · · · · ·

  19. Joint Generator ? ? Temperature Cloud Sky Cover S ? S Time Min Mean Max Time Percent (%) 06:00-21:00 9 15 21 06:00-09:00 25-50 ... Cloudy, with temperatures ... 09:00-12:00 50-75 S Training between 10 and 20 degrees. S South wind around 20 mph. Wind Speed S Wind Direction ... ... Time Mode Time Min Mean Max 06:00-21:00 S 06:00-21:00 15 20 30 ... F 0 , 1 (temp 1 ,min) S → R( start ) (1) PCFG Grammar FS 0 , 1 (temp 1 ,start) R( r i .t ) → FS( r j , start )R( r j .t ) F 0 , 1 (temp 1 ,max) R( r i .t ) → FS( r j , start ) FS( r, r.f i ) → F( r, r.f j )FS( r, r.f j ) FS( r, r.f i ) → F( r, r.f j ) F 0 , 2 (temp 1 ,min) F( r, r.f ) → W( r, r.f )F( r, r.f ) FS 0 , 2 (temp 1 ,start) (2) Hypergraph F( r, r.f ) → W( r, r.f ) F 0 , 2 (temp 1 ,max) Representation W( r, r.f ) → α W( r, r.f ) → g( f.v ) FS 1 , 2 (temp 1 ,start)   mostly cloudy the morning ? 11 am mostly cloudy after ?   (3) k-best decoding   mostly cloudy then becoming ? FS 0 , 5 (skyCover 1 .t ,start)   via integration · · ·   mostly cloudy mostly clouds     Testing cloudy ,   F 0 , 2 (skyCover 1 .t ,%) W 4 , 5 (skyCover 1 .t ,time) · · ·   morning 11 am W 0 , 1 (skyCover 1 .t ,%) W 1 , 2 (skyCover 1 .t ,%)     after       mostly mostly · · · cloudy cloudy         Konstas and Lapata, 2012, 2013 sunny sunny     · · · · · ·

  20. Joint Generator ? ? Temperature Cloud Sky Cover S ? S Time Min Mean Max Time Percent (%) 06:00-21:00 9 15 21 06:00-09:00 25-50 ... Cloudy, with temperatures ... 09:00-12:00 50-75 S Training between 10 and 20 degrees. S South wind around 20 mph. Wind Speed S Wind Direction ... ... Time Mode Time Min Mean Max 06:00-21:00 S 06:00-21:00 15 20 30 ... F 0 , 1 (temp 1 ,min) S → R( start ) (1) PCFG Grammar FS 0 , 1 (temp 1 ,start) R( r i .t ) → FS( r j , start )R( r j .t ) F 0 , 1 (temp 1 ,max) R( r i .t ) → FS( r j , start ) FS( r, r.f i ) → F( r, r.f j )FS( r, r.f j ) FS( r, r.f i ) → F( r, r.f j ) F 0 , 2 (temp 1 ,min) F( r, r.f ) → W( r, r.f )F( r, r.f ) FS 0 , 2 (temp 1 ,start) (2) Hypergraph F( r, r.f ) → W( r, r.f ) F 0 , 2 (temp 1 ,max) Representation W( r, r.f ) → α W( r, r.f ) → g( f.v ) FS 1 , 2 (temp 1 ,start)   mostly cloudy the morning ? 11 am mostly cloudy after ?   (3) k-best decoding   mostly cloudy then becoming ? FS 0 , 5 (skyCover 1 .t ,start)   via integration · · ·   mostly cloudy mostly clouds     Testing cloudy ,   F 0 , 2 (skyCover 1 .t ,%) W 4 , 5 (skyCover 1 .t ,time) · · ·   morning 11 am W 0 , 1 (skyCover 1 .t ,%) W 1 , 2 (skyCover 1 .t ,%)     after       mostly mostly · · · cloudy cloudy         Konstas and Lapata, 2012, 2013 sunny sunny     · · · · · ·

  21. More Complex Structure

  22. More Complex Structure Input (Graph or Tree) know-01 ARG0 ARG1 I planet ARG1-of inhabit-01 ARG0 man mod lazy

  23. More Complex Structure Input (Graph or Tree) know-01 ARG0 ARG1 I planet ARG1-of inhabit-01 ARG0 man mod lazy AMR DAG

  24. More Complex Structure Input (Graph or Tree) know-01 ARG0 ARG1 I planet ARG1-of inhabit-01 ARG0 man mod lazy AMR DAG λ -calculus Expression Tree

  25. More Complex Structure Input (Graph or Tree) know-01 ARG0 ARG1 I planet ARG1-of inhabit-01 ARG0 man mod lazy AMR DAG λ -calculus Expression Tree ECIs Tree

  26. More Complex Structure Input (Graph or Tree) Output (Text) know-01 ARG0 ARG1 I know a planet that is inhabited by a lazy man . I planet ARG1-of inhabit-01 ARG0 man mod lazy AMR DAG λ -calculus Expression Tree ECIs Tree

  27. More Complex Structure Input (Graph or Tree) Output (Text) know-01 ARG0 ARG1 I know a planet that is inhabited by a lazy man . I planet ARG1-of inhabit-01 I knew a planet that was inhabited by a lazy man . (lpp_1943.249) ARG0 man I have known a planet that was inhabited by mod a lazy man . lazy AMR DAG λ -calculus Expression Tree ECIs Tree

  28. More Complex Structure Input (Graph or Tree) Output (Text) know-01 ARG0 ARG1 I know a planet that is inhabited by a lazy man . I planet ARG1-of inhabit-01 I knew a planet that was inhabited by a lazy man . (lpp_1943.249) ARG0 man I have known a planet that was inhabited by mod a lazy man . lazy I know about a planet. It is inhabited by a lazy man . AMR DAG λ -calculus Expression Tree ECIs Tree

  29. N EU G EN Framework ARG1-of

  30. N EU G EN Framework G w Encoder Decoder ARG1-of

  31. N EU G EN Framework G w Encoder Decoder Encoder Bag of Words: Concepts and roles - ARG1-of Attention mechanism -

  32. N EU G EN Framework G w Encoder Decoder Encoder Decoder Bag of Words: Concepts and roles Left-to-right word LSTM - - ARG1-of Attention mechanism Greedy beam search - -

  33. N EU G EN Framework G w Encoder Decoder Reranker Encoder Decoder Bag of Words: Concepts and roles Left-to-right word LSTM - - ARG1-of Attention mechanism Greedy beam search - -

  34. N EU G EN Framework G w Encoder Decoder Reranker Encoder Decoder Bag of Words: Concepts and roles Left-to-right word LSTM - - ARG1-of Attention mechanism Greedy beam search - - Reranker

  35. N EU G EN Framework G w Encoder Decoder Reranker Encoder Decoder Bag of Words: Concepts and roles Left-to-right word LSTM - - ARG1-of Attention mechanism Greedy beam search - - Reranker CCG parser -> semantics -

Recommend

More recommend