low dimensional embeddings of logic
play

Low-dimensional Embeddings of Logic aschel, 1 Matko Bosnjak, 1 Sameer - PowerPoint PPT Presentation

Sep 13, 2023 •333 likes •903 views

Low-dimensional Embeddings of Logic aschel, 1 Matko Bosnjak, 1 Sameer Singh 2 and Sebastian Riedel 1 Tim Rockt 1 University College London 2 University of Washington Machine Reading ACL 2014 Workshop on Semantic Parsing 26th June 2014

  1. Low-dimensional Embeddings of Logic aschel, 1 Matko Bosnjak, 1 Sameer Singh 2 and Sebastian Riedel 1 Tim Rockt¨ 1 University College London 2 University of Washington Machine Reading ACL 2014 Workshop on Semantic Parsing 26th June 2014

  2. Motivation Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  3. Motivation Machine Reading and Reasoning Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  4. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  5. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  6. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  7. Motivation Machine Reading and Reasoning “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  8. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding mammals that are found in Southeast Asia.” mammal ( Colugo ) “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  9. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding Endothermic mammals that are found in Colugo Mammal Southeast Asia.” Kagu Ectothermic mammal ( Colugo ) HasFeathers “All mammals are endothermic.” ∀ x : mammal ( x ) ⇒ endothermic ( x ) Reasoning... endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  10. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding Endothermic mammals that are found in Colugo Mammal Southeast Asia.” Kagu Ectothermic mammal ( Colugo ) HasFeathers “All mammals are endothermic.” Wrong Predictions ∀ x : mammal ( x ) ⇒ endothermic ( x ) mammal ( Kagu ) Reasoning... ectothermic ( Colugo ) endothermic ( Colugo ) I wish I had a distributed model... Mammal Endothermic Colugo Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  11. Motivation Machine Reading and Reasoning Debugging Distributed Representations “Colugos are arboreal gliding Endothermic mammals that are found in Colugo Mammal Southeast Asia.” Kagu Ectothermic mammal ( Colugo ) HasFeathers “All mammals are endothermic.” Wrong Predictions ∀ x : mammal ( x ) ⇒ endothermic ( x ) mammal ( Kagu ) Reasoning... ectothermic ( Colugo ) endothermic ( Colugo ) I wish I had a distributed model... I wish I could fix this with... ∀ x : hasFeathers ( x ) ⇒ ¬ mammal ( x ) Mammal Endothermic ∀ x : animal ( x ) ⇒ Colugo endothermic ( x ) ⊕ ectothermic ( x ) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 1/7

  12. Information Extraction Evidence DB Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  13. Information Extraction Logic Evidence Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  14. Information Extraction Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  15. Information Extraction Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Background Knowledge Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  16. Logical Inference Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  17. Logical Inference Logic Evidence Discourse Representation Theory or Semantic Parsing Relation Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) endothermic(Colugo), weight 0.53 DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  18. Inference via Distributed Representations Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013) Relation Tapir Extraction mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo endothermic DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  19. Inference via Distributed Representations Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) OpenIE Tapir mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) endothermic(Tapir) Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  20. Inference via Distributed Representations Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) OpenIE Tapir mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  21. Distributed Representations that Simulate First-order Logical Reasoning Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) This Work OpenIE Tapir mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal mammal(Tapir), endothermic( x ) endothermic(Colugo), Colguo “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  22. Distributed Representations that Simulate First-order Logical Reasoning Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) This Work OpenIE mammal mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) Colugo endothermic(Colugo), Tapir “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  23. Distributed Representations that Simulate First-order Logical Reasoning Logic Distributed Representations Evidence Bordes et al. (2011), Nickel et al. (2012), Socher et al. (2013), Riedel et al. (2013) This Work OpenIE mammal mammal(Colugo), ∀ x : mammal( x ) ⇒ mammal(Tapir), endothermic( x ) Colugo endothermic(Colugo), Tapir “Colugos, gliding endothermic mammals. . . “ DB Logical Inference Background Algebra · · · Knowledge Baader et al. (2007) Bos (2008) · · · Garrette et al. (2011) Beltagy et al. (2013) True endothermic(Tapir) endothermic(Tapir) False Rockt¨ aschel et al. Low-dimensional Embeddings of Logic 2/7

  24. Propositional Logic Logic Logical Tensor Calculus (Grefenstette, 2013) � 1 � 0 � � [true]; [false] ; 0 1

Recommend

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings Agenda Traditional NLP Word Embeddings-1 Word Embeddings-2 Text preprocessing Topic Modeling ELMo Bag-of-words model Neural

900 views • 64 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

1.25k views • 75 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

2k views • 24 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

422 views • 17 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

828 views • 15 slides
Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme: Index dun mot: w i Table dembeddings (lookup matrix): V Embedding: e i e i = V( w i ) Reprsentation dun mot

780 views • 47 slides
Categories, Logic, Physics Higher Dimensional Birmingham Group Theory? September 21,

Categories, Logic, Physics Higher Dimensional Birmingham Group Theory? September 21,

What is and what should be Categories, Logic, Physics Higher Dimensional Birmingham Group Theory? September 21, 2010 Ronnie Brown What it should be Dreams shattered! Back to basics Groupoids to the rescue Homotopy What is

575 views • 38 slides
Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds University of California, San Diego Southern California Machine Learning Symposium, Caltech, 11/18/2018 Word Embeddings Language models which learn

537 views • 32 slides
What is best for spoken langage understanding: small but task-dependent embeddings or huge but

What is best for spoken langage understanding: small but task-dependent embeddings or huge but

What is best for spoken langage understanding: small but task-dependent embeddings or huge but out-of-domain embeddings Sahar Ghannay, Antoine Neuraz, Sophie Rosset 1 Goal Focus on semantic evaluation of common word embeddings

360 views • 16 slides
Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff

Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff

Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff and Qin Zhang IBM Research Almaden COLT13, June 12, 2013 1-1 Subspace embeddings Subspace embeddings: A distribution over linear maps : R n

810 views • 44 slides
Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an

Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an

Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an Embedding ? 3 Why Embeddings ? 4 Embeddings Pipeline 5 Whats Next Agenda 1 Team 2 Whats an Embedding ? 3 Why Embeddings ? 4

732 views • 40 slides
Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview Word types and tokens Training contextual embeddings Embeddings from Language Models (ELMo) 1 Overview Word types and tokens Training

670 views • 40 slides
Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments

Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments

Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl June 12, 2018 Drawings of Graphs Drawings of Graphs G = ( V, E ) is a simple graph . The set of vertices V is finite

596 views • 56 slides
What can we say about high- dimensional objects from a low-dimensional representation? 2

What can we say about high- dimensional objects from a low-dimensional representation? 2

Dimension Reduction with Certainty Rasmus Pagh IT University of Copenhagen S CALABLE S IMILARITY ECML-PKDD S EARCH September 21, 2016 Slides: goo.gl/hZoRWo 1 What can we say about high- dimensional objects from a low-dimensional

940 views • 75 slides
CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu

CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu

CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu Research Engineer, Facebook AI ledell@fb.com 1 Outline Word Embeddings word2vec Graph Embeddings Applications world2vec

769 views • 41 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

537 views • 35 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

569 views • 38 slides
Embeddings for KB and text representation, extraction and question answering. Jason Weston

Embeddings for KB and text representation, extraction and question answering. Jason Weston

Embeddings for multi-relational data Pros and cons of embedding models Embeddings for KB and text representation, extraction and question answering. Jason Weston & Antoine Bordes & Sumit Chopra Facebook AI Research External

595 views • 24 slides
Name Embeddings and Online News Analysis Speaker: Junting Ye Department: Computer

Name Embeddings and Online News Analysis Speaker: Junting Ye Department: Computer

Name Embeddings and Online News Analysis Speaker: Junting Ye Department: Computer Science Advisor: Prof. Steven Skiena Outline Overview Name Embeddings Nationality Classification Ethnicity & Gender

509 views • 15 slides
A Logic for Social Influence through Communication Zo e Christoff Institute for Logic,

A Logic for Social Influence through Communication Zo e Christoff Institute for Logic,

Introduction: the two approaches to combine A two dimensional social network plausibility framework Social influence through communication Further research A Logic for Social Influence through Communication Zo e Christoff Institute for

688 views • 41 slides
Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic

Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic

Putting the Pieces Together Markov Logic Markov Logic Probability First-Order Logic Propositional Logic Markov Logic Definition A Markov Logic Network (MLN) is a set of A logical KB is a set of hard constraints pairs (F, w) where on

350 views • 6 slides
Extensions of Embeddings in the 0 2 Turing Degrees. Antonio Montalb an. U. of Chicago

Extensions of Embeddings in the 0 2 Turing Degrees. Antonio Montalb an. U. of Chicago

Decidability results Our results Extensions of Embeddings in the 0 2 Turing Degrees. Antonio Montalb an. U. of Chicago Nanjing, May 2008 Joint work with Rod Downey, Noam Greenberg and Andy Lewis. Extensions of Embeddings in the 0

404 views • 24 slides
Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY

Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY

Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY 5/3/18 Outline NLP Intro Word representations and word embeddings Word2vec models Visualizing word embeddings Word2vec in

1.18k views • 63 slides
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko

Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko

Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko and Chong Cher Chia and Hady W. Lauw Singapore Management University Word Embeddings Dense vectors of words Unsupervised training: GloVe,

522 views • 19 slides

More recommend