Machine Learning 2 DS 4420 - Spring 2020 Sequence-2-sequence models - PowerPoint PPT Presentation

Machine Learning 2 DS 4420 - Spring 2020 Sequence-2-sequence models Byron C. Wallace

Today • Learning to map between sequences

Today • Learning to map between sequences

Input x 1 x 2 x T-1 x T …

Output y 1 y 2 y T-1 y T … MAGIC Input x 1 x 2 x T-1 x T …

Motivating Example: Translation

donde esta la bibliotheque? Spanish y 1 y 2 y T-1 y T … MAGIC English x 1 x 2 x T-1 x T … where is the library?

Motivating Example: Sequence Captioning Input Example credit: https://towardsdatascience.com/understanding-encoder-decoder-sequence-to-sequence-model-679e04af4346

Motivating Example: Sequence Captioning MAGIC Input Example credit: https://towardsdatascience.com/understanding-encoder-decoder-sequence-to-sequence-model-679e04af4346

Motivating Example: Sequence Captioning Output MAGIC Input Example credit: https://towardsdatascience.com/understanding-encoder-decoder-sequence-to-sequence-model-679e04af4346

MAGIC

Encoder-Decoder Models

Encoder x

z Encoder x

z Decoder Encoder x

y z Decoder Encoder x

Recall: Auto-Encoders Figure credit: https://stackabuse.com/autoencoders-for-image-reconstruction-in-python-and-keras/

RNN encoder x 1 x 2 x T-1 x T … where is the library? Slide derived from Matt Gormley’s: http://www.cs.cmu.edu/~mgormley/courses/10418/

RNN RNN RNN RNN x 1 x 2 x T-1 x T … where is the library? Slide derived from Matt Gormley’s: http://www.cs.cmu.edu/~mgormley/courses/10418/

RNN RNN RNN RNN x 1 x 2 x T-1 x T … where is the library? Slide derived from Matt Gormley’s: http://www.cs.cmu.edu/~mgormley/courses/10418/

h 1 h 2 h T-1 h T … Encoder RNN RNN RNN RNN x 1 x 2 x T-1 x T … where is the library? Slide derived from Matt Gormley’s: http://www.cs.cmu.edu/~mgormley/courses/10418/

RNN refresher Credit: Christopher Olah; https://colah.github.io/posts/2015-08-Understanding-LSTMs/

<latexit sha1_base64="ykFJ5c1d3j03cnvq6pmoTguSiIM=">ACXicbVDLSsNAFJ3UV62vqEs3g0WoCWpgm6EohuXFewD2hAm0k7dPJg5kYsIVs3/obF4q49Q/c+TdO0y60euDC4Zx7ufceLxZcgWV9GYWFxaXleJqaW19Y3PL3N5pqSiRlDVpJCLZ8YhigoesCRwE68SkcATrO2NriZ+45JxaPwFsYxcwIyCLnPKQEtuSYeuilk+AL7btqDIQOSVe5dOMr1Yzs7dM2yVbVy4L/EnpEymqHhmp+9fkSTgIVABVGqa1sxOCmRwKlgWamXKBYTOiID1tU0JAFTp/kuEDrfSxH0ldIeBc/TmRkCpceDpzoDAUM17E/E/r5uAf+6kPIwTYCGdLvITgSHCk1hwn0tGQYw1IVRyfSumQyIJBR1eSYdgz7/8l7RqVfukWrs5LdcvZ3EU0R7aRxVkozNUR9eogZqIogf0hF7Qq/FoPBtvxvu0tWDMZnbRLxgf3w+gmUc=</latexit> RNN refresher h t = f θ ( x t , h t − 1 ) Credit: Christopher Olah; https://colah.github.io/posts/2015-08-Understanding-LSTMs/

RNN refresher Credit: Christopher Olah; https://colah.github.io/posts/2015-08-Understanding-LSTMs/

donde esta la bibliotheque? y 1 y 2 y T-1 y T … h 1 h 2 h T-1 h T … Encoder RNN RNN RNN RNN x 1 x 2 x T-1 x T … where is the library? Slide derived from Matt Gormley’s: http://www.cs.cmu.edu/~mgormley/courses/10418/

RNN h 1 h 2 h T-1 h T … Encoder RNN RNN RNN RNN x T x 1 x 2 x T-1 … where is the library?

Decoder RNN START h 1 h 2 h T-1 h T … Encoder RNN RNN RNN RNN x T x 1 x 2 x T-1 … where is the library?

donde esta la bibliotheque? y 1 y 2 y T-1 y T … Decoder … RNN RNN RNN RNN START h 1 h 2 h T-1 h T … Encoder RNN RNN RNN RNN x T x 1 x 2 x T-1 … where is the library?

Let’s think about a concrete example…

https://colab.research.google.com/drive/ 1z_uyUjmXjpqbPwE7JWxYX116O0229aVK