Some of the Team
Facebook AI Intelligent Dialogue tests Tests 5 99.5% represents an error in the test result due to an error in the input file. We did not attempt to recode to produce the wrong result to pass the test
Demo 1: Meaning Matcher • “The shark bit the surfer at the beach yesterday because it was hungry.” • “The shark bit the surfer the dog kissed at the beach yesterday because it was hungry.” • “The shark bit no ate the surfer the dog kissed at the beach yesterday because it was hungry.” • “Can you push the door open?” • “Can’t you push the door open?”
Deep Learning / ANN / Computational Text In Text Out Natural Language Processing (NLP) • NLU was abandoned as a goal in the 1950s and 1960s
Science / Ockham’s Razor Select the hypothesis with the fewest assumptions Aristotle (384–322 BC) “Nature operates in the shortest way possible” John Duns Scotus (1265–1308) “Pluralitas non est ponenda sine necessitate (Plurality is not to be posited without necessity)” Bertrand Russell (1872-1970) “Whenever possible, substitute constructions out of known entities for inferences to unknown entities” Albert Einstein (1879-1955) “ A scientific theory should be as simple as possible, but no simpler”
Role and Reference Grammar (RRG) Syntactic representation Discourse-pragmatics Linking Algorithm Semantic representation
Scientific Models for Linguistics Chomsky J.R. Firth Van Valin 1957 1957 1980s Study Study Syntax Language & “Rules- Social / Meaning in based” Context Context You shall know a word by the company it keeps (Firth, 1957)
Many NLP Tools – eg. Stanford • Tokenizer • Temporal tagger • Parser • Named Entity Recognition (eg. PERSON, ORGANIZATION, LOCATION) Stanford’s Dan Jurafsky c.2011
This science doesn’t support engineering Noam Chomsky, Syntactic Structures, 1957 “The fundamental aim in the linguistic analysis of a language L is to separate the grammatical sequences which are the sentences of L from the ungrammatical sequences which are not sentences of L and to study the structure of the grammatical sequences.” … “Sentences (1) and (2) are equally nonsensical, but any speaker of English will recognize that only the former is grammatical. ● (1) Colorless green ideas sleep furiously. ● (2) Furiously sleep ideas green colorless.”
Structure is both grammatical and ungrammatical Colorless green ideas sleep furiously JJ JJ NNS VBP RB verb �3 sg pres adj adj noun pl adverb Running running runnings run furiously Red little ridinghoods run furiously Little red ridinghoods run furiously Small round birds eat hungrily
Better to look at meaning – predicates resolve Predicate/ Referent/ Modifier (NOT noun, verb, adverb, adjective, etc) The running men (the men are running/the running of men) do' (men, [ run' (men)]) The running water (the water is running/the running of water) do’ (water, [ flow' (water)]) The running men are tired The running water is cold The tired running men ran the cold running water
Category Squishes
Inferring mathematics • Language understanding (NLU) with patterns (not templates or symbols) • The cat sadly ate the rat on the mat slowly yesterday • (cat) – (eat) – (rat) + where (on the mat) + when (yesterday) • 1+1=10 • (number) + (number) = (number) • 1+1=2 • (number) + (number) = (number) • “Can you start a song?” “Can you park in the back?”
e.g. I went because I was hungry to the burger place RRG Layered Structure Sentence Promise Sandy to wash the car (core juncture) Clause Persuade? Tomorrow (when) Nucleus Push the door open (nuclear juncture) Core Evidently (evidential) In an hour In the car (where) (when)
Demo 2/3: bAbI (benchmark) & conversation • Facebook’s AI Research Team (FAIR) directed by Yann LeCunn (Deep Learning scientist). Our report published at arXive (per FAIR): https://arxiv.org/ • HAVE’ • Results improve on keyword capacity of world’s best (distributional hypothesis) • The fish is off. The light is off. The party is off. • Call Beth, no John. Who do you call? • Translation: “The dog bit the mother”
Questions?
Recommend
More recommend
