A DOCUMENT SUMMARIZER FOR NOVICES REX RUBIN WHY A DOCUMENT - PowerPoint PPT Presentation

A DOCUMENT SUMMARIZER FOR NOVICES REX RUBIN

WHY A DOCUMENT SUMMARIZER?  Getting into a field of research is:  Daunting with the amount of information presented  Difficult to discern what is important and what isn’t  How a summarizer will help:  Present the most relevant information and remove the excess

EXTRACTION VS ABSTRACTION  Extraction[1]  Abstraction[1]  Pulls sentences straight  Creates sentences by from the input joining several together  Does not make its own  Works better for several sentences documents at once

TEXTRANK  Extraction based[2]  Creates a web of sentences  This web is used as an input for PageRank  PageRank will rank the sentences[3]  Gives the summary as the output

HOW TO IMPROVE THIS MODEL?  It is important to note the glossary should be of relevant terms compared to the original document  The way TextRank works, the glossary will allow for similar sentences to connect and score higher  This will help by giving more informative sentences  It is important to know that more informative does not mean easier to read

MY TEXTRANK MODIFICATION

RESEARCH QUESTION  Will including a glossary of related terms in the original document bring about more informative sentences?

HYPOTHESIS  Having a glossary included in the original document will bring out more informative sentences in the final summary

EXPERIMENT OVERVIEW  Two experimental groups:  Control Group (Y)  Test Group (X)  Have the groups take a test on the original document

MY SUMMARY  My summary was made using a document focused on cybersecurity and the glossary was filled with similar cybersecurity terms

PARTICIPANTS  Participants:  Union College students aged 18-22  Mixed group of CS students and non-CS students  2 Groups:  Control(Y) read the summary that was made through the original TextRank program  Test (X) read the summary that was made through my modified TextRank program

TEST GIVEN TO PARTICIPANTS  The test given to participants was based on the main points of the original document  Why the main points?  The main points should be in the summary  Question types  3 Multiples Choice  3 Open Answer

AVERAGE SCORES OF QUESTIONS 3.5 3.22 3.06 3 2.5 2 1.5 0.94 0.94 0.89 0.89 1 0.56 0.5 0.44 0.39 0.5 0.33 0.22 0.19 0.06 0 Question 1 Question 2 Question 3 Question 4 Question 5 Question 6 Total Score Multiple Choice: Open Answer: Data on the left is Y 3 1 and the right is X 4 2 6 5

AVERAGE SCORES OF QUESTIONS OUTLIERS REMOVED 4 3.625 3.5 3 3 2.5 2 1.5 1 1 1 1 1 0.5625 0.5 0.5 0.428571 0.5 0.375 0.1875 0.0714286 0 0 Question 1 Question 2 Question 3 Question 4 Question 5 Question 6 Total Score

DIFFERENCES IN RESULTS X-Y 0.4 0.33 0.3 0.2 0.16 0.13 0.11 0.11 0.1 0 0 Question 1 Question 2 Question 3 Question 4 Question 5 Question 6 Total Score -0.1 -0.2 -0.3 -0.4 -0.45 -0.5

DIFFERENCES X-Y OUTLIERS REMOVED 0.8 0.625 0.571429 0.6 0.375 0.4 0.2 0.1160714 0.0625 0 0 Question 1 Question 2 Question 3 Question 4 Question 5 Question 6 Total Score -0.2 -0.4 -0.5 -0.6

WAS MY HYPOTHESIS CORRECT? With these results, I can say my hypothesis is incorrect

SOMETHING ELSE?  Differences in 4 and 6 were significant Question 4 Question 6 1 1 0.89 0.89 0.9 0.9 0.8 0.8 0.7 0.7 0.56 0.6 0.6 0.5 0.45 0.5 0.44 0.4 0.4 0.33 0.3 0.3 0.2 0.2 0.1 0.1 0 0 X Average Y Average Difference Y-X X Average Y Average Difference X-Y

CITATIONS [1]Jan Pedersen Kupiec, Julian and Francine Chen. A trainable document summarizer. ACM SIGIR conference on Research and development in information retrieval, (15):68 – 73, 1995 [2] Paul Tarau Rada Mihalcea. Textrank: Bringing order into texts. 2011. [3] Herwig Unger Mario Kubek. Topic detection based on the pagerank’s clustering property. 2011.