sequence to sequence learning as beam search optimization
play

Sequence-to-Sequence Learning as Beam-Search Optimization Sam - PowerPoint PPT Presentation

Feb 23, 2023 •274 likes •878 views

Sequence-to-Sequence Learning as Beam-Search Optimization Sam Wiseman and Alexander M. Rush Seq2Seq as a General-purpose NLP/Text Generation Tool Machine Translation ???? Luong et al. [2015] Question Answering ? Conversation ? Parsing Vinyals et

  1. Sequence-to-Sequence Learning as Beam-Search Optimization Sam Wiseman and Alexander M. Rush

  2. Seq2Seq as a General-purpose NLP/Text Generation Tool Machine Translation ???? Luong et al. [2015] Question Answering ? Conversation ? Parsing Vinyals et al. [2015] Sentence Compression Filippova et al. [2015] Summarization ? Caption Generation ? Video-to-Text ? Grammar Correction ?

  3. Room for Improvement? Despite its tremendous success, there are some potential issues with standard Seq2Seq [Ranzato et al. 2016; Bengio et al. 2015] : (1) Train/Test mismatch (2) Seq2Seq models next-words, rather than whole sequences Goal of the talk : describe a simple variant of Seq2Seq — and corresponding beam-search training scheme — to address these issues.

  4. Review: Sequence-to-sequence (Seq2Seq) Models Encoder RNN (red) encodes source into a representation x Decoder RNN (blue) generates translation word-by-word

  5. Review: Seq2Seq Generation Details h 3 = RNN( w 3 , h 2 ) h 1 h 2 w 1 w 2 w 3 Probability of generating t ’th word: p ( w t | w 1 , . . . , w t − 1 , x ; θ ) = softmax( W out h t − 1 + b out )

  6. Review: Train and Test Train Objective : Given source-target pairs ( x, y 1: T ) , minimize NLL of each word independently, conditioned on gold history y 1: t − 1 � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t Test Objective : Structured prediction � y 1: T = arg max ˆ ln p ( w t | w 1: t − 1 , x ; θ ) w 1: T t Typical to approximate the arg max with beam-search

  7. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  8. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  9. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  10. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  11. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  12. Review: Beam Search at Test Time ( K = 3 ) a the red For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  13. Review: Beam Search at Test Time ( K = 3 ) a red the dog red blue For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  14. Review: Beam Search at Test Time ( K = 3 ) a red dog the dog dog red blue cat For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  15. Review: Beam Search at Test Time ( K = 3 ) a red dog smells the dog dog barks red blue cat walks For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  16. Review: Beam Search at Test Time ( K = 3 ) a red dog smells home the dog dog barks quickly red blue cat walks straight For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  17. Review: Beam Search at Test Time ( K = 3 ) a red dog smells home today the dog dog barks quickly Friday red blue cat walks straight now For t = 1 . . . T : For all k and for all possible output words w : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) Update beam: y (1: K ) y ( k ) ˆ ← K-arg max s ( w t , ˆ 1: t − 1 ) 1: t w 1: t

  18. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  19. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  20. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  21. Seq2Seq Issues Revisited Issue #1: Train/Test Mismatch (cf., Ranzato et al. [2016] ) � NLL ( θ ) = − ln p ( w t = y t | y 1: t − 1 , x ; θ ) t (a) Training conditions on true history (“Exposure Bias”) (b) Train with word-level NLL, but evaluate with BLEU-like metrics Idea #1: Train with beam-search Use a loss that incorporates (sub)sequence-level costs

  22. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  23. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  24. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  25. Idea #1: Train with Beam Search Replace NLL with loss that penalizes search-error: � � y ( K ) y ( K ) y ( K ) � L ( θ ) = ∆(ˆ 1: t ) 1 − s ( y t , y 1: t − 1 ) + s (ˆ , ˆ 1: t − 1 ) t t y ( K ) y 1: t is the gold prefix; ˆ is the K ’th prefix on the beam 1: t y ( k ) y ( k ) y ( k ) y ( k ) s (ˆ , ˆ 1: t − 1 ) is the score of history (ˆ , ˆ 1: t − 1 ) t t y ( K ) y ( K ) ∆(ˆ 1: t ) allows us to scale loss by badness of predicting ˆ 1: t

  26. Seq2Seq Issues Revisited Issue #2: Seq2Seq models next-word probabilities : y ( k ) y ( k ) y ( k ) s ( w t = w, ˆ 1: t − 1 ) ← ln p (ˆ 1: t − 1 | x ) + ln p ( w t = w | ˆ 1: t − 1 , x ) (a) Sequence score is sum of locally normalized word-scores; gives rise to “Label Bias” [Lafferty et al. 2001] (b) What if we want to train with sequence-level constraints? Idea #2: Don’t locally normalize

Recommend

Beam Search Shahrzad Kiani and Zihao Chen CSC2547 Presentation Beam Search Greedy Search: Always

Beam Search Shahrzad Kiani and Zihao Chen CSC2547 Presentation Beam Search Greedy Search: Always

Beam Search Shahrzad Kiani and Zihao Chen CSC2547 Presentation Beam Search Greedy Search: Always go to top 1 scored sequence (seq2seq) Beam Search: Maintain the top K scored sequences (this paper) Seq2Seq Train and Test Issues gold sequence

337 views • 14 slides
Sequence to Sequence models: Connectionist Temporal Classification 1 Sequence-to-sequence

Sequence to Sequence models: Connectionist Temporal Classification 1 Sequence-to-sequence

Deep Learning Sequence to Sequence models: Connectionist Temporal Classification 1 Sequence-to-sequence modelling Problem: A sequence goes in A different sequence comes out E.g. Speech recognition:

1.86k views • 172 slides
Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models Eldan Cohen

Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models Eldan Cohen

1 The Thirty-sixth International Conference on Machine Learning Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models Eldan Cohen J. Christopher Beck Poster: Pacific Ballroom #47 Motivation 2 u Most commonly

252 views • 9 slides
Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Deep Learning Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem: A sequence goes in A different sequence comes out E.g. Speech recognition: Speech goes in, a word

1.77k views • 162 slides
Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem:

Deep Learning Sequence to Sequence models: Attention Models 1 Sequence-to-sequence modelling Problem: A sequence goes in A different sequence comes out E.g. Speech recognition: Speech goes in, a word

2.23k views • 167 slides
Sequence-to-Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le,

Sequence-to-Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le,

Sequence to Sequence Learning with Neural Networks Sequence-to-Sequence Learning with Neural Networks Ilya Sutskever, Oriol Vinyals, Quoc V. Le, NIPS 2014 Introduced by Graham Neubig, NAIST 2014-11-01 1 Sequence to Sequence Learning with

360 views • 19 slides
Sequence to Sequence models: Connectionist Temporal Classification 5 March 2018 1

Sequence to Sequence models: Connectionist Temporal Classification 5 March 2018 1

Deep Learning Sequence to Sequence models: Connectionist Temporal Classification 5 March 2018 1 Sequence-to-sequence modelling Problem: A sequence 1 goes in A different sequence 1 comes out

1.44k views • 141 slides
16 Applications 1: Monolingual Sequence-to-sequence Prob- lems Up until now, we have largely

16 Applications 1: Monolingual Sequence-to-sequence Prob- lems Up until now, we have largely

16 Applications 1: Monolingual Sequence-to-sequence Prob- lems Up until now, we have largely used machine translation as an example of sequence-to-sequence learning tasks. However, as mentioned at the beginning of the course,

216 views • 8 slides
SEQ 3 : Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive

SEQ 3 : Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive

SEQ 3 : Differentiable Sequence-to-Sequence-to-Sequence Autoencoder for Unsupervised Abstractive Sentence Compression Christos Baziotis, Ion Androutsopoulos, Ioannis Konstas, Alexandros Potamianos Ed nburgh NLP University of Edinburgh Natural

772 views • 51 slides
Asynchronous sequence circuits An asynchronous sequence machine is a sequence circuit without

Asynchronous sequence circuits An asynchronous sequence machine is a sequence circuit without

Asynchronous sequence circuits An asynchronous sequence machine is a sequence circuit without flip-flops Asynchronous sequence machines are based on combinational gates with feedback Upon analysis it is assumed : Only one signal at a

1.34k views • 66 slides
Sequence 7 January 2019 OSU CSE 1 Sequence The Sequence component family allows you to

Sequence 7 January 2019 OSU CSE 1 Sequence The Sequence component family allows you to

Sequence 7 January 2019 OSU CSE 1 Sequence The Sequence component family allows you to manipulate strings of entries of any (arbitrary) type through direct access by position, similar to an array Another generic type like Queue and Set

533 views • 38 slides
Sequence to Sequence Models Matt Gormley Lecture 5 Sep. 11, 2019 1 Q&A Q: What did the

Sequence to Sequence Models Matt Gormley Lecture 5 Sep. 11, 2019 1 Q&A Q: What did the

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Sequence to Sequence Models Matt Gormley Lecture 5 Sep. 11, 2019 1 Q&A Q: What did the results of the

505 views • 46 slides
Sequence-to-Sequence Learning using Recurrent Neural Networks Jindich Helcl, Jindich

Sequence-to-Sequence Learning using Recurrent Neural Networks Jindich Helcl, Jindich

Sequence-to-Sequence Learning using Recurrent Neural Networks Jindich Helcl, Jindich Libovick March 4, 2020 NPFL116 Compendium of Neural Machine Translation Charles University Faculty of Mathematics and Physics Institute of Formal and

1.29k views • 79 slides
Attention Strategies for Multi-Source Sequence-to-Sequence Learning Jindich Libovick,

Attention Strategies for Multi-Source Sequence-to-Sequence Learning Jindich Libovick,

Attention Strategies for Multi-Source Sequence-to-Sequence Learning Jindich Libovick, Jindich Helcl Institute of Formal and Applied Linguistics Faculty of Mathematics and Physics Charles University August 2, 2017 Motivation

370 views • 16 slides
SEQUENCE ANALYSIS The term " sequence analysis " in biology implies subjecting a DNA or

SEQUENCE ANALYSIS The term " sequence analysis " in biology implies subjecting a DNA or

Sequence Analysis SEQUENCE ANALYSIS The term " sequence analysis " in biology implies subjecting a DNA or peptide sequence to sequence alignment, sequence databases, repeated sequence searches, or other bioinformatics methods on a

818 views • 20 slides
Attentive Sequence-to-Sequence Learning March 6, 2018 Jindich Helcl, Jindich Libovick

Attentive Sequence-to-Sequence Learning March 6, 2018 Jindich Helcl, Jindich Libovick

NPFL116 Compendium of Neural Machine Translation Attentive Sequence-to-Sequence Learning March 6, 2018 Jindich Helcl, Jindich Libovick Charles Univeristy in Prague Faculty of Mathematics and Physics Institute of Formal and Applied

562 views • 23 slides
Machine Translation using Deep Learning Methods Max Fomin Michael Zolotov Sequence to Sequence

Machine Translation using Deep Learning Methods Max Fomin Michael Zolotov Sequence to Sequence

Machine Translation using Deep Learning Methods Max Fomin Michael Zolotov Sequence to Sequence Learning with Neural Networks Learning Phrase Representations using RNN Encoder Decoder for Statistical Machine Translation Topics

909 views • 41 slides
Natural Language Processing with Deep Learning Sequence-to-sequence Models with Attention Navid

Natural Language Processing with Deep Learning Sequence-to-sequence Models with Attention Navid

Natural Language Processing with Deep Learning Sequence-to-sequence Models with Attention Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception Agenda Sequence-to-sequence

900 views • 64 slides
Introduction to sequence to sequence models N ATURAL LAN GUAGE GEN ERATION IN P YTH ON

Introduction to sequence to sequence models N ATURAL LAN GUAGE GEN ERATION IN P YTH ON

Introduction to sequence to sequence models N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath Halder Data Scientist Sequence to sequence generation Output a sequence given a sequence as input. Fixed length input. Fixed length output.

884 views • 39 slides
Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of

Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of

Garrett Bingham Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of video frames, Prev: Title Slide generate a variable-length natural language Problem description of the video. Next: Motivation

850 views • 15 slides
Lecture 4: Problem solving as search: Solution = a finite sequence of actions

Lecture 4: Problem solving as search: Solution = a finite sequence of actions

CS440/ECE448: Intro to Artificial Intelligence Tuesday s key concepts Lecture 4: Problem solving as search: Solution = a finite sequence of actions Heuristic search State graphs and search trees and local

185 views • 14 slides
CS224N/Ling284 Lecture 8: Machine Translation, Sequence-to-sequence and Attention Abigail See

CS224N/Ling284 Lecture 8: Machine Translation, Sequence-to-sequence and Attention Abigail See

Natural Language Processing with Deep Learning CS224N/Ling284 Lecture 8: Machine Translation, Sequence-to-sequence and Attention Abigail See Announcements We are taking attendance today Sign in with the TAs outside the auditorium

1.05k views • 79 slides
CSE 427 Comp Bio Sequence Alignment 1 Sequence Alignment What Why A Dynamic Programming

CSE 427 Comp Bio Sequence Alignment 1 Sequence Alignment What Why A Dynamic Programming

CSE 427 Comp Bio Sequence Alignment 1 Sequence Alignment What Why A Dynamic Programming Algorithm 2 Sequence Alignment Goal: position characters in two strings to best line up identical/similar ones with one another We can do this

941 views • 56 slides
CSE 421 Algorithms Sequence Alignment 1 Sequence Alignment What Why A Dynamic Programming

CSE 421 Algorithms Sequence Alignment 1 Sequence Alignment What Why A Dynamic Programming

CSE 421 Algorithms Sequence Alignment 1 Sequence Alignment What Why A Dynamic Programming Algorithm 2 Sequence Alignment Goal: position characters in two strings to best line up identical/similar ones with one another We can do

473 views • 28 slides

More recommend