Overview of neuro-symbolic processing in Neural Blackboard - PowerPoint PPT Presentation

Overview of neuro-symbolic processing in Neural Blackboard Architectures Frank van der Velde University of Twente, The Netherlands f.vandervelde@utwente.nl Dagstuhl Seminar 17192: Human-Like Neural-Symbolic Computing 8-12 May 2017

Aims of Neural Blackboard Architectures (NBAs) Ø Combinatorial structures and processing in neuronal manner Ø Comparing with human behavior Ø Satisfying constraints on cognition and brain: • Grounded, ‘in situ’ representations, also in combinatorial structures • Connection paths as basis for behaviour • Content addressable combinatorial structure Ø Modeling of neuronal activity (Marc de Kamps) Ø Incremental sentence processing Ø Competition in NBA as tool (e.g., ambiguity resolution) Ø Architecture for parallel computing 2

Assumptions concept representation in brain Neural circuits & populations (interacting exc and inh). u Associative connections u Conditional connections (labeled connections) Content addressable • Grounded, ‘in situ’, develop over time • Hebbian-like assemblies, with (long-term) relations (combinations of distributed and more local representations) FvdV (2015). Neural Networks 3

Assumptions concept representation in brain Grounded representations: Not copied and pasted (e.g., like symbols) Without grounding: Central population has no meaning (no ‘neural code’). • Grounded, ‘in situ’, develop over time • Hebbian-like assemblies, with (long-term) relations (combinations of distributed and more local representations) FvdV (2015). Neural Networks 4

Combinatorial structures (e.g., short-term) • Relations between concepts in specified ‘neural blackboards’. • Interaction blackboards via ‘in situ’ concepts. FvdV (2015). Neural Networks 5

Neural blackboard architecture for sentence structures S y Sentence Phonology Blackboard Blackboard N x V z / ca / N u / do / cat / t / sees dog / g / Role blackboard (e.g., sentence structure): • Allow productive (novel) combinations • Provide co connect ctio ion path between sensory and motor activation, needed for behaviour FvdV (2015). Neural Networks 6

Importance connection paths Cognition: • ‘Delayed reflex’ • But not just reflexive behaviour • Increased complexity in evolution • But all behaviour depends on some connection path FvdV (2015). Neural Networks 7

Importance connection paths Argument against Neural Blackboards with ‘fixed’ connection structures for novel bindings: Feldman ( 2013, Cogn Neurodyn ): ‘‘if I tell you that my granddaughter Sonnet is brilliant, you have a new person to consider as a possible filler for variable roles and also a number of new facts for use in inference.’’ 8

Importance connection paths Not: If If ‘ ‘fixed’ connection structures for novel bindings. But: what kin kind ? Dedicated sentence representations: Content addressable. But: No novel bindings FvdV & MdK (2015). Cognitive Neurodynamics 9

Importance connection paths Not: If If ‘ ‘fixed’ connection structures for novel bindings. But: what kin kind ? Universal machine: Maximal novel binding. But: Not content addressable FvdV & MdK (2015). Cognitive Neurodynamics 10

Importance connection paths ‘Small-world’ like connection structure for binding: Content addressable • Forms of novel binding • Language: • Two tier structure (at least) • Phonology neural blackboard: new words • Sentence neural blackboards: novel sentences • But with constraints: familiar language; based on development FvdV & MdK (2015). Cognitive Neurodynamics 11

NBA: structure neural blackboard S 1 Conditional connections: Structure v n assemblies N 1 X n n v V 1 N 1 i t sees cat di t N 2 dog WM Control circuit circuit ‘in situ’ word assemblies Binding Syntax Structure assemblies: S 1 = Main assemblies v = Sub assemblies FvdV & MdK (2006). Behavioral and Brain Sciences 12

NBA: structure neural blackboard S 1 Conditional connections: Structure v n assemblies N 1 X n n v V 1 N 1 i t sees cat di t N 2 dog WM Control circuit circuit ‘in situ’ word assemblies Binding Syntax Structure assemblies: S 1 = Main assemblies v = Sub assemblies FvdV & MdK (2006). Behavioral and Brain Sciences 13

NBA: duplicating words, multiple sentences S 1 S 2 Structure v n n v assemblies n v v n V 1 N 1 V 2 N 3 t t sees likes cat t t N 2 N 4 dog bird • Same ‘in situ’ word assembly • Different structure assemblies Structure assemblies: S 1 = Main assemblies v = Sub assemblies FvdV & MdK (2006). Behavioral and Brain Sciences 14

NBA: Content addressable S 1 S 2 cat sees? Structure v n n v assemblies n v v n V 1 N 1 V 2 N 3 t t sees likes cat t t N 2 N 4 dog bird Illustrated with question: • Structure of question • Effect of question in blackboard • Evolutionary pressure : fast (direct) and informative Structure assemblies: S 1 = Main assemblies v = Sub assemblies FvdV & MdK (2006). Behavioral and Brain Sciences 15

NBA: Content addressable S 1 S 2 cat sees? v n n v n v v n V 1 N 1 V 2 N 3 t t sees likes cat t t N 2 N 4 dog bird Illustrated with question: • Structure of question • Effect of question in blackboard • Evolutionary pressure : fast (direct) and informative Structure assemblies: S 1 = Main assemblies v = Sub assemblies FvdV & MdK (2006). Behavioral and Brain Sciences 16

NBA: Binding • Based on binding competition in Connection Matrices (CMs) • CM: matrix of ‘connection nodes’ • Connection node: gating circuits and WM • WM: neuronal population with sustained activity Y j V 1 -t X i X in Y in i I = N 2 -t I WM i Connection Matrix (CM) Connection Node = inhibition: Internal CM competition (binding restriction) 17

NBA: Binding Pilot simulation of development of connection matrix (CM): • Initial random connections between assemblies and nodes in CM • Process of hebbian and anti-hebbian learning • Able to produce selective CMs Good (optimal) Not good: Good (not optimal) Confusion 18

Binding in the brain? NBA: A Connection Matrix NBA: for each specific binding: • Massively parallel architecture • Agent ‘ field’ • Extensive computation: • Theme ‘field’ More complex processing with more • Other ‘fields’ (but similar) neural ‘hardware’ Frankland & Greene, PNAS, 2015 lmSTC: left mid-superior temporal cortex 19

NBA: Incremental sentence processing Interaction control circuits and blackboard activity FvdV & MdK (2010). Cognitive Systems Research 20

NBA: Ambiguity resolution UPA1a. Bill knows John . UPA1b. Bill knows John likes fish. Lewis (1993) An Architecturally-based Theory of Human Sentence Comprehension Ø A collection of (31) unproblematic ambiguities (UPA) Ø A collection of (26) garden path (GP) constructions 21

NBA: Ambiguity resolution N 2 -t C 1 -c V 1 -t V 1 -c Connection Matrix Connection Matrix = inhibition: Between CM competition (Constraints on binding) FvdV & MdK (2010). Cognitive Systems Research 22

NBA: Ambiguity resolution UPA1a. Bill knows John . UPA1b. Bill knows John likes fish. 23

NBA: Ambiguity resolution UPA1a. Bill knows John . UPA1b. Bill knows John likes fish. UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 24

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 25

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 26

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 27

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 28

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 29

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 30

NBA: Ambiguity resolution UPA3a. Ron believes the linguistics professor. UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. 31

NBA: Ambiguity resolution UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. John believes Ron (who) believes the linguistics professor he had met the week before in Prague disliked him hated him. (Resembles Garden Path #2 in Lewis 1993) 32

NBA: Ambiguity resolution UPA3b. Ron believes the linguistics professor he had met the week before in Prague disliked him. John believes Ron (who) believes the linguistics professor he had met the week before in Prague disliked him hated him (Resembles Garden Path #2 in Lewis 1993) 33

NBA: Ambiguity resolution 34

NBA: Ambiguity resolution 35

NBA: Ambiguity resolution 36

NBA: Ambiguity resolution 37