question answering on web data
play

Question Answering on Web Data Silei Xu CS294S April 9, 2020 Joint - PowerPoint PPT Presentation

Aug 26, 2023 •325 likes •1.23k views

Question Answering on Web Data Silei Xu CS294S April 9, 2020 Joint work with Giovanni Campagna, Sina Semnani, Jian Li, and Monica S. Lam Commercial Assistants Alexa User hand-codes question/code 1 by 1 get me an upscale restaurants What are

  1. Question Answering on Web Data Silei Xu CS294S April 9, 2020 Joint work with Giovanni Campagna, Sina Semnani, Jian Li, and Monica S. Lam

  2. Commercial Assistants Alexa User hand-codes question/code 1 by 1 get me an upscale restaurants What are the restaurants around here? What is the best restaurant? search for Chinese restaurants

  3. Commercial Assistants Alexa User hand-codes question/code 1 by 1 get me an upscale restaurants What are the restaurants around here? What is the best restaurant? search for Chinese restaurants 100K Alexa skills Sep 2019

  4. Commercial Assistants Alexa User hand-codes question/code 1 by 1 100K Alexa skills get me an upscale restaurants Sep 2019 What are the restaurants around here? What is the best restaurant? search for Chinese restaurants 1.8 billion websites

  5. Genie: Synthesize Question/Code from a Schema User Schema Name Price Cuisine … Property Annotations Genie 500 Domain- Independent Templates What is the <prop> of <subject>? What is the <subject>’s <prop>?

  6. Genie: Synthesize Question/Code from a Schema User get me an upscale restaurants Schema What are the restaurants around here? Name Price Cuisine … What is the best restaurant? search for Chinese restaurants What is the best restaurant within 10 miles? Find restaurants that serve Chinese or Japanese food Property Annotations What is the best non-Chinese restaurant near here? Show me a cheap restaurant with 5-star review. Genie Are there any restaurant with at least 4.5 stars? What is the phone number of Wendy’s? 500 Domain- I’m looking for an Italian fine dining restaurant. Independent Give me the best Italian restaurant. Templates Find me the best restaurant with 500 or more reviews Show me some restaurant with less than 10 reviews What is the <prop> of <subject>? What is the <subject>’s <prop>?

  7. The Web Has a Schema!

  8. The Web Has a Schema! • Schema.org • Structure data to mark up web pages • Mainly used by search engines • It covers many domains, including restaurants, hotels, people, recipes, products, news …

  9. The Web Has a Schema! • Schema.org <script type="application/ld+json"> • Structure data to mark up web pages { • Mainly used by search engines @type: "restaurant", name: "The French Laundry", • It covers many domains, including servesCuisine: “ French", restaurants, hotels, people, recipes, aggregateRating: { products, news … @type: "AggregateRating", reviewCount: 2527, ratingValue: 4.5 } ... Schema.org markup on Yelp }

  10. The Web Has a Schema! • Schema.org <script type="application/ld+json"> • Structure data to mark up web pages { • Mainly used by search engines @type: "restaurant", name: "The French Laundry", • It covers many domains, including servesCuisine: “ French", restaurants, hotels, people, recipes, aggregateRating: { products, news … @type: "AggregateRating", reviewCount: 2527, ratingValue: 4.5 } 40% of the websites use it! ... Schema.org markup on Yelp }

  11. Outline • Introduction to Schema.org • Represent Questions in ThingTalk • LUINet: NL to ThingTalk • Training data generation • Experimental results • Work in progress: automate everything!

  12. Introduction to Schema.org

  13. Graph Data Model of Schema.org

  14. Graph Data Model of Schema.org Organization legalName: Text slogan: Text aggregateRating: AggregateRating ...

  15. Graph Data Model of Schema.org class Organization legalName: Text slogan: Text aggregateRating: AggregateRating ...

  16. Graph Data Model of Schema.org class Organization properties legalName: Text slogan: Text aggregateRating: AggregateRating ...

  17. Graph Data Model of Schema.org class Organization types – primitive or class properties legalName: Text slogan: Text aggregateRating: AggregateRating ...

  18. Graph Data Model of Schema.org Organization AggregateRating legalName: Text ratingCount: Integer slogan: Text ratingValue: Integer aggregateRating: AggregateRating ... ...

  19. Schema.org Hierarchy Thing name: Text url: URL ... Organization ( Thing ) AggregateRating legalName: Text ratingCount: Integer slogan: Text ratingValue: Integer aggregateRating: AggregateRating ... ...

  20. Schema.org Hierarchy Thing name: Text url: URL ... Organization ( Thing ) AggregateRating legalName: Text ratingCount: Integer slogan: Text ratingValue: Integer aggregateRating: AggregateRating ... ... LocalBusiness ( Place , Organization ) openingHours: Text priceRange: Text ...

  21. Some useful tools • Google Structured Data Testing Tool • Show schema.org markups in a web page • Google Custom Search • Search for pages that contain certain schema.org domains

  22. ThingTalk for Questions

  23. ThingTalk for QA

  24. ThingTalk for QA @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => => notify Show me restaurants in Stanford

  25. ThingTalk for QA @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” => notify Show me Chinese restaurants in Stanford

  26. ThingTalk for QA @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” => notify Show me Chinese restaurants in Stanford

  27. ThingTalk for QA sort aggregateRating.ratingValue desc of ( @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” ) => notify Show me top-rated Chinese restaurants in Stanford

  28. ThingTalk for QA sort aggregateRating.ratingValue desc of ( @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” ) => notify Show me top-rated Chinese restaurants in Stanford

  29. ThingTalk for QA sort aggregateRating.ratingValue desc of ( @QA.restaurant(), geo == makeLocation (“ Stanford ”) now => && servesCuisine =~ “ Chinese ” ) => notify join ( @QA.Review(), in_array(id, review) && author = “ bob ” ) Show me top-rated Chinese restaurants in Stanford reviewed by Bob

  30. ThingTalk for QA

  31. ThingTalk for QA

  32. ThingTalk for QA …

  33. LUINet: NL to ThingTalk

  34. Natural Language Programming LUINet Natural language ThingTalk What is the top-rated Chinese sort aggregateRating.ratingValue desc of ( @ QA.restaurant(), restaurant in Palo Alto? geo == new MakeLocation (“ Stanford ”) && servesCuisine =~ “ Chinese ” )

  35. Genie Pipeline LUINet Natural language ThingTalk

  36. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant LUINet Natural language ThingTalk

  37. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Domain-independent Templates What is the <prop> of <table>? What is the <table >’s <prop>? LUINet Natural language ThingTalk

  38. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates What is the <prop> of <table>? What is the <table >’s <prop>? LUINet Natural language ThingTalk

  39. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates Paraphrase What is the <prop> of <table>? What is the <table >’s <prop>? LUINet Natural language ThingTalk

  40. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates Paraphrase What is the <prop> of <table>? What is the <table >’s <prop>? Parameter & data augmentation LUINet Natural language ThingTalk

  41. Genie Pipeline Thingpedia Manifest Schema Natural Language Annotations Name Price Cuisine … cuisine of the restaurant restaurant’s cuisine cuisine served by the restaurant ThingTalk Grammar Synthesize sentence/code pairs Domain-independent Templates Paraphrase What is the <prop> of <table>? What is the <table >’s <prop>? Parameter & data augmentation Training Data LUINet Natural language ThingTalk

Recommend

Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on

Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on

Question Answering What is Ques+on Answering? Dan Jurafsky Ques%on Answering One of the oldest NLP tasks (punched card systems in 1961) Simmons, Klein,

856 views • 73 slides
A Multilingual Hybrid Question-Answering System Cross-Lingual Open-Domain Question Answering

A Multilingual Hybrid Question-Answering System Cross-Lingual Open-Domain Question Answering

A Multilingual Hybrid Question-Answering System Cross-Lingual Open-Domain Question Answering Gnter Neumann, Bogdan Sacaleanu 30th DFKI SAB MEETING 04/04/2006 German Research Center for Artificial Intelligence Inference Linguistic World

428 views • 20 slides
Semantic Question Answering on Big Data Tatiana Erekhinskaya July, 2016 The Goal Challenge:

Semantic Question Answering on Big Data Tatiana Erekhinskaya July, 2016 The Goal Challenge:

Semantic Question Answering on Big Data Tatiana Erekhinskaya July, 2016 The Goal Challenge: Find answers to complex questions in large structured and unstructured data resources Sample question: List Chinese researchers who

240 views • 20 slides
Question Answering and AnswerFinder Diego Moll a Centre for Language Technology Department of

Question Answering and AnswerFinder Diego Moll a Centre for Language Technology Department of

AnswerFinder Stages in Question Answering Learning Question Answering Rules Question Answering and AnswerFinder Diego Moll a Centre for Language Technology Department of Computer Science Macquarie University Athens, 4 July 2007 Diego

1.03k views • 64 slides
CS345a Data Mining Project A Web Based Question Answering System Vincenzo Di Nicola Jyotika

CS345a Data Mining Project A Web Based Question Answering System Vincenzo Di Nicola Jyotika

CS345a Data Mining Project A Web Based Question Answering System Vincenzo Di Nicola Jyotika Prasad The Ultimate question answering system What is the meaning of life? Who are we? Why are we doing CS? Or, less philosophically,

162 views • 13 slides
Statistical NLP Spring 2011 Lecture 26: Question Answering Dan Klein UC Berkeley Question

Statistical NLP Spring 2011 Lecture 26: Question Answering Dan Klein UC Berkeley Question

Statistical NLP Spring 2011 Lecture 26: Question Answering Dan Klein UC Berkeley Question Answering Following largely from Chris Mannings slides, which includes slides originally borrowed from Sanda Harabagiu, ISI, Nicholas

414 views • 20 slides
Question Answering Statistical NLP Following largely from Chris Mannings slides, which

Question Answering Statistical NLP Following largely from Chris Mannings slides, which

Question Answering Statistical NLP Following largely from Chris Mannings slides, which includes Spring 2011 slides originally borrowed from Sanda Harabagiu, ISI, Nicholas Kushmerick. Lecture 26: Question Answering Dan Klein UC

531 views • 7 slides
Designing deep architectures for Visual Question Answering Matthieu Cord Sorbonne University

Designing deep architectures for Visual Question Answering Matthieu Cord Sorbonne University

Designing deep architectures for Visual Question Answering Matthieu Cord Sorbonne University valeo.ai research lab. Paris Thanks to H. Ben-younes, R. Cadne Visual Question Answering Question Answering: + What does Claudia do? Visual

831 views • 68 slides
Question Answering and Reading Comprehension Kevin Duh Fall 2019, Intro to HLT, Johns Hopkins

Question Answering and Reading Comprehension Kevin Duh Fall 2019, Intro to HLT, Johns Hopkins

Question Answering and Reading Comprehension Kevin Duh Fall 2019, Intro to HLT, Johns Hopkins University What is Question Answering? Its a field concerned with building systems that answer questions posed in natural language Question

484 views • 36 slides
Towards End-to-End Reasoning for Question Answering Minjoon Seo Department of Computer Science

Towards End-to-End Reasoning for Question Answering Minjoon Seo Department of Computer Science

Towards End-to-End Reasoning for Question Answering Minjoon Seo Department of Computer Science &amp; Engineering University of Washington September 29, 2016 @ Samsung AI Lab What is reasoning? Simple Question Answering Model What is

727 views • 60 slides
Question-Answering: Shallow &amp; Deep Techniques for NLP Deep Processing Techniques for NLP

Question-Answering: Shallow &amp; Deep Techniques for NLP Deep Processing Techniques for NLP

Question-Answering: Shallow &amp; Deep Techniques for NLP Deep Processing Techniques for NLP Ling 571 March 12, 2014 (Examples from Dan Jurafsky) Roadmap Question-Answering: Definitions &amp; Motivation Basic pipeline:

426 views • 32 slides
Factoid Question Answering Roy Aslan (ra2752@Columbia.edu) A Neural Network for Factoid

Factoid Question Answering Roy Aslan (ra2752@Columbia.edu) A Neural Network for Factoid

Factoid Question Answering Roy Aslan (ra2752@Columbia.edu) A Neural Network for Factoid Question Answering over Paragraphs Mohit Iyyer, Jordan Boyd-Graber, Leonardo Claudino, Richard Socher, and Hal Daum III Task and Setting Factoid

504 views • 21 slides
Question-Answering: Shallow &amp; Deep Techniques for NLP Ling571 Deep Processing Techniques

Question-Answering: Shallow &amp; Deep Techniques for NLP Ling571 Deep Processing Techniques

Question-Answering: Shallow &amp; Deep Techniques for NLP Ling571 Deep Processing Techniques for NLP March 9, 2011 Examples from Dan Jurafsky) Roadmap Question-Answering: Definitions &amp; Motivation Basic pipeline:

993 views • 84 slides
Phrase-Indexed Question Answering : A New Challenge for Scalable Document Comprehension Minjoon

Phrase-Indexed Question Answering : A New Challenge for Scalable Document Comprehension Minjoon

Phrase-Indexed Question Answering : A New Challenge for Scalable Document Comprehension Minjoon Seo, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, Hannaneh Hajishirzi Question Answering? 1961 Model Barack Obama (1961-present) was the 44 th

460 views • 32 slides
Question Classification in English-Chinese Cross-Language Question Answering: An Integrated

Question Classification in English-Chinese Cross-Language Question Answering: An Integrated

Question Classification in English-Chinese Cross-Language Question Answering: An Integrated Genetic Algorithm and Machine Learning Approach Min-Yuh Day 1, 2 , Chorng-Shyong Ong 2 , and Wen-Lian Hsu 1,* , Fellow, IEEE 1 Institute of Information

608 views • 21 slides
Self-Critical Reasoning for Robust Visual Question Answering Jialin Wu and Raymond J. Mooney

Self-Critical Reasoning for Robust Visual Question Answering Jialin Wu and Raymond J. Mooney

Self-Critical Reasoning for Robust Visual Question Answering Jialin Wu and Raymond J. Mooney Visual Question Answering (VQA) Common VQA system What utensil is pictured? Knife (0.72) Answer Prediction Fork Visual feature set

513 views • 11 slides
Question Answering Spring 2020 2020-04-02 Adapted from slides from Danqi Chen and Karthik

Question Answering Spring 2020 2020-04-02 Adapted from slides from Danqi Chen and Karthik

SFU NatLangLab CMPT 825: Natural Language Processing Question Answering Spring 2020 2020-04-02 Adapted from slides from Danqi Chen and Karthik Narasimhan (with some content from slides from Chris Manning) Question Answering Goal: build

921 views • 42 slides
Chess Q&amp;A : Question Answering on Chess Games Reasoning, Attention, Memory NIPS Workshop 12

Chess Q&amp;A : Question Answering on Chess Games Reasoning, Attention, Memory NIPS Workshop 12

Chess Q&amp;A : Question Answering on Chess Games Reasoning, Attention, Memory NIPS Workshop 12 December 2015 Volkan Cirik Louis-Philippe Morency Eduard Hovy 1 Visual Question Answering Which city is pictured? Malinowski et. al. 2015, Gao

466 views • 13 slides
Embeddings for KB and text representation, extraction and question answering. Jason Weston

Embeddings for KB and text representation, extraction and question answering. Jason Weston

Embeddings for multi-relational data Pros and cons of embedding models Embeddings for KB and text representation, extraction and question answering. Jason Weston &amp; Antoine Bordes &amp; Sumit Chopra Facebook AI Research External

595 views • 24 slides
CQARank:Jointly Model Topics and Expertise in Community Question Answering Liu Yang, Minghui Qiu,

CQARank:Jointly Model Topics and Expertise in Community Question Answering Liu Yang, Minghui Qiu,

CQARank:Jointly Model Topics and Expertise in Community Question Answering Liu Yang, Minghui Qiu, Swapna Gottipati, Feida Zhu, Jing Jiang, Huiping Sun, Zhong Chen Peking University Singapore Management University Community Question Answering

456 views • 33 slides
An Question Recommendation System for Question Answer Community (Stackoverflow) Presenter: Haoyu

An Question Recommendation System for Question Answer Community (Stackoverflow) Presenter: Haoyu

CS 6501 Text Mining: An Question Recommendation System for Question Answer Community (Stackoverflow) Presenter: Haoyu Chen Haoran Hou What is Question Answering Community: Community question answering (cQA) provides a platform for

250 views • 24 slides
Question Answering Alexander Solovyev Bauman Moscow Sate Technical University a-soloviev@mail.ru

Question Answering Alexander Solovyev Bauman Moscow Sate Technical University a-soloviev@mail.ru

Syntactic and semantic models and algorithms in Question Answering Alexander Solovyev Bauman Moscow Sate Technical University a-soloviev@mail.ru 20.10.2011 RCDL. Voronezh. 1 Agenda Question Answering and Answer Validation task Answer

704 views • 28 slides
Neural Question Answering at BioASQ 5B Georg Wiese, Dirk Weissenborn, Mariana Neves Motivation

Neural Question Answering at BioASQ 5B Georg Wiese, Dirk Weissenborn, Mariana Neves Motivation

Neural Question Answering at BioASQ 5B Georg Wiese, Dirk Weissenborn, Mariana Neves Motivation Neural question answering (QA) systems are end-to-end trainable machine learning models which achieve top performance in domains with large

145 views • 13 slides
Question Answering &amp; the Semantic Web Gnter Neumann Language Technology-Lab DFKI,

Question Answering &amp; the Semantic Web Gnter Neumann Language Technology-Lab DFKI,

Question Answering &amp; the Semantic Web Gnter Neumann Language Technology-Lab DFKI, Saarbrcken Overview Hybrid Question Answering Language Technology and the Semantic Web 2004 G. Neumann Motivation: From Search Engines to

874 views • 53 slides

More recommend