Machine Reading and Reasoning with Neural Program Interpreters Sebastian Riedel @riedelcastro Bloomsbury AI Machine Reading
Collaborators Pontus Stenetorp Matko Bosnjak Johannes Welbl (UCL) (UCL) (UCL) Tim Rocktäschel Jason Naradowsky (now at Oxford) (Johns Hopkins University) 2
“Should we separate meaning from language?” [Maybe not?] Chris Manning @AKBC 2013 ! ? [Language] [Meaning] [Information Need] 3
End-to-End Reading and Comprehension (Hermann et. al 2015, Seo et al., 2016, Rajpurkar et al., 2016, Weissenborn 2016 … ) ! ? [Language] [Information Need] 4
Machine Reading What city did Tesla move to in 1880? Nicola Tesla Nicola Tesla Nicola Tesla Prague … … … In January 1880, two of Tesla's uncles put together In January 1880, two of Tesla's uncles put together In January 1880, two of Tesla's uncles put together enough money to help him leave Gospi ć for Prague enough money to help him leave Gospi ć for Prague enough money to help him leave Gospi ć for Prague where he was to study. Unfortunately, he arrived too where he was to study. Unfortunately, he arrived too where he was to study. Unfortunately, he arrived too Why was he unable to enroll at the university? late to enroll at Charles-Ferdinand University; he late to enroll at Charles-Ferdinand University; he late to enroll at Charles-Ferdinand University; he never studied Greek, a required subject; and he was never studied Greek, a required subject; and he was never studied Greek, a required subject; and he was illiterate in Czech, another required subject. Tesla did, illiterate in Czech, another required subject. Tesla did, illiterate in Czech, another required subject. Tesla did, however, attend lectures at the university, although, as however, attend lectures at the university, although, as however, attend lectures at the university, although, as arrived too late to enroll an auditor, he did not receive grades for the courses. an auditor, he did not receive grades for the courses. an auditor, he did not receive grades for the courses. 5
How to read and reason ? 6
Machine Reading and Reasoning Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. 7
Machine Reading and Reasoning Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. 2 8
Can we learn this end-to-end ? 9
Pontus Stenetorp Johannes Welbl Part 1: A Read & Reason Dataset 10
11
A Single Instance Jamie Burnett (born 16 September 1975 ) is a What is the nationality of Jamie Burnett? professional snooker player from Hamilton, South Lanarkshire … He began the 2014/2015 season with a quarter-final showing at the Yixing Open … Candidates: Scotland (correct) … Hamilton is a town in South Lanarkshire, in the China (incorrect) central Lowlands of Scotland … … The Yixing Open was a professional minor- ranking snooker tournament that took place at the Yixing Sports Centre in Yixing, China. 12
Dataset Construction Method Unlabelled Text Dataset Construction Method Multihop Dataset Knowledge Base [Conditionally Accepted to TACL] 13
Dataset Construction Method KB Triple Documents Instance Entities described in What is the nationality of Jamie Burnett? Jamie Burnett mentions Hamilton Jamie Burnett, citizenship, Scotland Scotland Scotland described in China Yixing Open mentions China 14
Dataset Construction Method Unlabelled Text Dataset Construction Method Multihop Dataset Knowledge Base [Conditionally Accepted to TACL] 15
Dataset Construction Method Unlabelled Text Dataset Construction Method WikiHop Knowledge Base 16
Dataset Construction Method Unlabelled Text Dataset Construction Method MedHop Knowledge Base 17
Baseline Results 50 Random 47.8 Max-Mention TF-IDF BiDAF 42.9 41.667 33.333 Accuracy [%] 25 25.6 16.667 13.9 11.5 10.6 9.5 8.333 9 0 WikiHop MedHop 18
Reduction to Traditional Machine Comprehension Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. 19
Reduction to Traditional Machine Comprehension Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. 20
Does BiDaf Aggregate? Which medical specialty deals with pituitary ACTH Pituitary ACTH hypersecretion ... is a form of hypersecretion? hyperpituitarism characterized by an abnormally high level of ACTH produced by the anterior pituitary … Endocrinology A major organ of the endocrine system , the anterior pituitary is the glandular, anterior lobe that ... The endocrine system is ... ... The field of study dealing with the endocrine system and its disorders is endocrinology , a branch of internal medicine. 21
Removing Relevant Documents, Keep Answer Documents 55 54.5 BIDAF 10% BIDAF doc-rem 50 45 44.6 Accuracy [%] 40 35 33.7 30 30.4 25 WikiHop MedHop 22
Matko Bosnjak Tim Rocktäschel Jason Naradowsky Part 2: Learning to Read and Calculate 23
Machine Reading and Reasoning: Math Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. 2 24
Differentiable Program Interpreters Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. Model Program Interpreter Reader 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d Stack Heap Code 3 4 2 Bosnjak et al. ICML 2017 25
Differentiable Program Interpreters Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. Model Program Interpreter Reader 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d Stack Code Stack Heap Code 3 4 4 2 2 3 Bosnjak et al. ICML 2017 26
Differentiable Program Interpreters Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. Model Program Interpreter Reader 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d Stack Code Stack Code Stack Heap Code 3 4 4 2 6 2 3 3 Bosnjak et al. ICML 2017 27
Differentiable Program Interpreters Isabel uploaded 2 pictures from her phone and 4 How many pictures were in each of the albums? from her camera to facebook. She sorted the pics into 3 different albums with the same amount of pics in each album. Model Program Interpreter Reader 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 p d Stack Code Stack Code Stack Code Stack Heap Code 3 4 4 2 6 3 2 3 3 6 Bosnjak et al. ICML 2017 28
Recommend
More recommend
