Plausible reasoning based on qualitative entity embeddings Steven - PowerPoint PPT Presentation

Plausible reasoning based on qualitative entity embeddings Steven Schockaert (joint work with Shoaib Jameel) School of Computer Science & Informatics Cardi ff University, Cardi ff , UK SchockaertS1@cardi ff .ac.uk http://users.cs.cf.ac.uk/S.Schockaert

Plausible reasoning Geometric representations of meaning Learning conceptual spaces Qualitative representations of meaning

Category based induction Chardonnay contains polyphenols Malbec contains polyphenols Most wines contain polyphenols

Category based induction Chardonnay contains polyphenols Malbec contains polyphenols Most wines contain polyphenols ? Pinot Blanc contains polyphenols Pinot Gris contains polyphenols Most wines contain polyphenols

Category based induction Chardonnay contains polyphenols Malbec contains polyphenols Most wines contain polyphenols ? Pinot Blanc contains polyphenols Pinot Gris contains polyphenols Most wines contain polyphenols ? Champagne contains polyphenols Port wine contains polyphenols Most wines contain polyphenols

Interpolation Restaurants in Wales are required to display food hygiene ratings Ice cream shops in Wales are required to display food hygiene ratings Sandwich shops in Wales are required to display food hygiene ratings

Similarity based reasoning Restaurants in Wales are required to display food hygiene ratings ? Sandwich shops in Wales are required to display food hygiene ratings

A fortiori inference University sta ff are not permitted to travel in business class University sta ff are not permitted to travel in first class

A fortiori inference University sta ff are not permitted to travel in business class The problem with business class is that it is expensive First class is more expensive than business class University sta ff are not permitted to travel in first class

Motivation unstructured domain domain domain domain data theory theory theory theory domain Completed domain theory domain theory theory Interpretable machine learning models Ontology based data access Recognising textual entailment

Plausible reasoning Geometric representations of meaning Learning conceptual spaces Qualitative representations of meaning

Word embeddings chardonnay wine (Mikolov 2013, Pennington 2014)

Word embeddings Paris London France UK (Mikolov 2013, Pennington 2014)

Knowledge graph embeddings Steven Spielberg y b d lives in e t c e r i d USA Jurassic park Danny Boyle y b d e t c lives in e r i d UK Trainspotting (Bordes 2013, Wang 2014, Zhong 2015)

Conceptual spaces black bear lion mammal large vertebrate wild boar sheep rattlesnake spider cardinal spider scary black widow spider dangerous (Gärdenfors 2000)

Plausible reasoning Geometric representations of meaning Learning conceptual spaces Qualitative representations of meaning

Learning conceptual spaces textual description of entities

Betweenness aliens between star trek and cloverfield cast away between titanic and into the wild lord of the rings between harry potter and troy mission impossible between the rock and skyfall star wars between lord of the rings and star trek troy between braveheart and thor wall-e between monsters inc and 2001: a space odyssey good will hunting between dead poets society and rain man unbreakable between sin city and the sixth sense scarface between sin city and the godfather forest gump between million dollar baby and stand by me shrek 2 between wedding crashers and the lion king

Betweenness abbey between castle and chapel bistro between restaurant and tea room butcher shop between marketplace and slaughterhouse conservatory between greenhouse and playhouse duplex between detached house and triplex flower shop between garden center and gift shop grocery store between convenience store and farmers market manor between castle and mansion house rice paddy between bamboo forest and cropland sushi restaurant between Japanese restaurant and tapas restaurant veterinarian between animal shelter and emergency room wine shop between gourmet shop and liquor store

Learning interpretable directions Direction towards more “violent” films films whose associated text contains the word “violent” films whose associated text does not contain the word “violent”

Learning interpretable directions

Commonsense classifiers measure. Foursquare GeoNames OpenCYC Algorithm Acc. F1 Acc. F1 Acc. F1 n Col 0.947 0.717 0.881 0.401 0.383 0.956 interpolation Btw A 0.949 0.717 0.883 0.395 0.373 0.956 Btw B 0.943 0.617 0.881 0.349 0.954 0.295 analogical Analog A 0.921 0.636 0.822 0.330 0.933 0.375 Analog B 0.940 0.707 0.853 0.347 0.945 0.382 classifiers Analog C 0.925 0.686 0.859 0.411 0.942 0.391 FOIL 0 0.926 0.564 0.876 0.201 0.950 0.267 a fortiori FOIL 1 50 0.925 0.596 0.860 0.272 0.943 0.329 inference FOIL 2 0.926 0.627 0.861 0.285 0.946 0.335 FOIL 3 0.928 0.594 0.876 0.300 0.949 0.268 1-NN 0.939 0.710 0.853 0.357 0.945 0.380 C4.5 MDS 0.925 0.534 0.849 0.178 0.941 0.245 C4.5 dir 0.918 0.382 0.849 0.374 0.939 0.262 SVM MDS 0.932 0.656 0.859 0.343 0.912 0.328 SVM BoW 0.913 0.358 0.874 0.172 0.946 0.205

Entity embeddings with conceptual subspaces

Entity embeddings with conceptual subspaces Ranking Induction Analogy MAP P@5 MRR Acc. ρ Skip-gram 0.155 0.176 0.356 0.505 0.184 CBOW 0.159 0.182 0.350 0.500 0.213 RESCAL 0.081 0.020 0.189 0.423 0.371 TransE 0.110 0.060 0.200 0.451 0.382 TransH 0.142 0.072 0.210 0.415 0.382 TransR 0.100 0.102 0.302 0.489 0.378 CTransR 0.122 0.132 0.323 0.499 0.402 pTransE anch 0.099 0.101 0.301 0.488 0.476 pTransE art 0.202 0.218 0.475 0.751 0.512 pTransE full 0.213 0.224 0.490 0.756 0.532 EECS full 0.319 0.231 0.609 0.883 0.591 EECS no rel 0.301 0.229 0.588 0.868 0.552 EECS no type 0.266 0.225 0.585 0.854 0.549 EECS no NN 0.258 0.220 0.581 0.843 0.545 EECS text 0.254 0.218 0.579 0.831 0.540 EECS type-comb 0.312 0.231 0.601 0.883 0.595 EECS type-dist 0.295 0.231 0.585 0.858 0.550 EECS rel-dim 0.309 0.225 0.585 0.859 0.551 EECS rel-dist 0.299 0.225 0.585 0.855 0.549

Plausible reasoning Geometric representations of meaning Learning conceptual spaces Qualitative representations of meaning

Limitations of learned semantic spaces black bear lion mammal mammal large vertebrate vertebrate wild boar sheep rattlesnake cardinal spider scary spider spider black widow scary dangerous

Qualitative semantic representations venue restaurant shop italian dessert restaurant restaurant ice cream shop pizzeria Easy to use in a variety of KR and NLP Is-a relationships often not su ffi cient tasks Often too shallow to allow for reliable Easy to understand/correct/extend by inductive inferences human domain experts Often many ways to structure the terms of Can be automatically learned/refined from a given domain text collections

One-dimensional representations d 2 D ↓ d 2 A B ↓ d 2 D C E B A ↓ d 1 C ↓ d 1 D ↓ d 1 d 1

Directions ice cream shop Michelin star restaurant formal cheap trendy healthy Compact and easy-to-understand representation Possible to encode (and e ffi cient to reason with) incomplete information (e.g. from text) Betweenness, analogy and adjacency can only be approximately represented Similarity estimates possibly less accurate

Directions Michelin star restaurant formal cheap trendy healthy Modelling vagueness/typicality e ff ects

Directions formal cheap trendy healthy Modelling context e ff ects

Region connection calculus externally connected partially overlapping disconnected a b a b a b a EC b a PO b a DC b non-tangential PP tangential proper part equal a a b b a b a NTPP b a TPP b a EQ b b NTPP -1 a b TPP -1 a

Betweenness A D C A ⋈ B E B

Conclusions Plausible reasoning, based on semantic background knowledge, can be used to fill in the gaps or resolve inconsistencies in imperfect knowledge bases This semantic background knowledge can be obtained by learning vector-space representations, where: ‣ Entities correspond to points ‣ Concepts correspond to regions ‣ Relative features correspond to directions By deriving qualitative representations from these vector spaces we can: ‣ Model relationships between concepts more precisely ‣ Improve the representations based based on relation extraction methods or by interacting with experts ‣ Evaluate semantic relationships in a context-dependent manner