A Hierarchical Encoder-Decoder for Paragraph Summarization Farzaneh - PowerPoint PPT Presentation

A Hierarchical Encoder-Decoder for Paragraph Summarization Farzaneh Mahdisoltani Department of Computer Science University of Toronto CSC2541, Fall 2016 Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 1 / 8

Paragraph Summarization Summarization Phenomena: ¯ Deletion Paraphrase Generalization Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 2 / 8

Related Work Extractive Summarization: Stitching words and phrases from the source Dorr et al. (2003), Cohn et al. (2008), Woodsend et al. (2010) Abstractive Summarization: Generating richer summaries based on understanding the input Attention based: Rush et al. (2015) Statistical machine translation for headline generation: Banko et. al.(2000) RNN with attention: Chopra et al.(2016) Encoder-Decoder: Hu et al.(2015) Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 3 / 8

Hierarchichal Encoder-Decoder Model for Summarization Paragraph S1 enc(S1) S2 enc(S2) S3 enc(S3) Sentence Paragraph enc(P) to Vector to Vector S4 enc(S4) S5 enc(S5) S6 enc(S6) enc(O1) O1 Word Sentence Level Level enc(O2) O2 Decoder Decoder enc(O3) O3 Summary Abstract Concepts Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 4 / 8

Hierarchichal Encoder-Decoder Model for Summarization Objective Function: � t P ( o t w | o < t w , o s , h p ) o 1 o 2 o 3 w w d d 1 d 2 d 3 w w w o 1 o 1 o 2 s w w d 1 p s h 1 h 2 start s s Sentence s i s j y j Level Word h 1 h 1 h 1 h 2 h 3 h 4 h 1 h 1 h 2 h 2 h 3 h 3 h 4 h 4 h 5 h 5 Level i i i i i i j j j j j j j j j j w i 1 w i 2 w i 3 w i 4 w j 1 v j 1 w j 2 v j 2 w j 3 v j 3 v j 4 w j 4 v j 5 w j 5 Encoder Decoder Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 5 / 8

Selecting the Correct Summary from Candidates Paragraph S1 enc(S1) S2 enc(S2) Paragrap S3 enc(S3) Sentence h to enc(P) to Vector S4 enc(S4) Vector S5 enc(S5) S6 enc(S6) enc(O1) O1 Paragrap Sentence enc(O1) O1 Paragrap enc(O2) h to enc(S1) O2 Sentence enc(O1) O1 to Vector Paragrap enc(O2) h to Vector enc(S2) O2 Sentence enc(O1) to Vector enc(O3) O1 O3 Paragrap enc(O2) h to enc(S3) O2 Vector Sentence to Vector enc(O3) O3 enc(O2) h to enc(S4) O2 Vector to Vector enc(O3) O3 Vector enc(O3) O3 Candidates Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 6 / 8

Preliminary Results Paragraph Representation Method Accuracy Random 25% Word2Vec 65% Skip-Thought Vectors 62% Bag of Words 67% RNN Encoder 57% Hierarchichal Encoder 54% Table: Training the encoder for multiple choice question Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 7 / 8

Thank You Farzaneh Mahdisoltani (University of Toronto) A Hierarchical Encoder-Decoder for Paragraph Summarization CSC2541, Fall 2016 8 / 8