reasoning with neural networks
play

Reasoning with Neural Networks Rodrigo Toro Icarte - PowerPoint PPT Presentation

Aug 14, 2022 •136 likes •750 views

University of Toronto Faculty of Arts and Science Department of Computer Science Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016 Introduction Reasoning with Neural Networks Questions References

  1. University of Toronto Faculty of Arts and Science Department of Computer Science Reasoning with Neural Networks Rodrigo Toro Icarte (rntoro@cs.toronto.edu) March 08, 2016

  2. Introduction Reasoning with Neural Networks Questions References Motivation Could a crocodile run a steeplechase? 1 1 The example was borrowed from Levesque (2014)

  3. Introduction Reasoning with Neural Networks Questions References Symbolic approach KB : ... ∀ x.Crocodile ( x ) ⊃ WeakLegs ( x ) ... ∀ x.WeakLegs ( x ) ⊃ ¬ CanJump ( x ) ... ∀ x. ¬ CanJump ( x ) ⊃ ¬ CanSteeplechase ( x ) ... Query : ¬∃ x.Crocodile ( x ) ∧ CanSteeplechase ( x )

  4. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a )

  5. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a )

  6. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a ) Observations : - This is SLD resolution.

  7. Introduction Reasoning with Neural Networks Questions References Symbolic approach Strategy : Contradiction proof. Assume query is false : Crocodile ( a ) ∧ CanSteeplechase ( a ) Observations : - This is SLD resolution. - Scalability problems.

  8. Introduction Reasoning with Neural Networks Questions References A Neural Conversational Model Vinyals & Le (2015) They trained a seq2seq: Credit : Vinyals & Le ... with movie subtitles.

  9. Introduction Reasoning with Neural Networks Questions References A Neural Conversational Model Vinyals & Le (2015) Human : who is skywalker ? Machine : he is a hero . Human : what is the color of the sky ? Machine : blue . Human : how many legs does a cat have ? Machine : four , i think . Human : what is the purpose of living ? Machine : to live forever

  10. Introduction Reasoning with Neural Networks Questions References Research question Can we use neural networks for common sense reasoning?

  11. Introduction Reasoning with Neural Networks Questions References Long term goal

  12. Introduction Reasoning with Neural Networks Questions References Reasoning with Neural Networks Two main branches: - Common Sense embeddings. - Neural Reasoners.

  13. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Hinton (1990)

  14. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Hinton (1990)

  15. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings McClelland & Rogers (2003)

  16. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings McClelland & Rogers (2003)

  17. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings McClelland & Rogers (2003)

  18. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Socher et al. (2013) Reasoning with neural tensor networks for knowledge base completion.

  19. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Socher et al. (2013) Reasoning with neural tensor networks for knowledge base completion.

  20. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Socher et al. (2013)

  21. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) Recursive neural networks can learn logical semantics.

  22. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) Recursive neural networks can learn logical semantics.

  23. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) � − → � x ( l ) � � + − → → − y TreeRNN = f M b → − x ( r ) − → − → y TreeRNN + f ( − → x ( l ) T T [1 ...n ] − → x ( r ) ) y TreeRNTN =

  24. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  25. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  26. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  27. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  28. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  29. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  30. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  31. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  32. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  33. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014) SICK textual entailment challenge

  34. Introduction Reasoning with Neural Networks Questions References Common Sense embeddings Bowman et al. (2014)

  35. Introduction Reasoning with Neural Networks Questions References Reasoning about facts

  36. Introduction Reasoning with Neural Networks Questions References Reasoning about facts The bAbI project (Weston et al. (2015)).

  37. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Three models have been proposed: - Dynamic Networks (Kumar et al. (2015)) - Memory Networks (Sukhbaatar et al. (2015)) - Neural Reasoner (Peng et al. (2015))

  38. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Sukhbaatar et al. (2015)

  39. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Kumar et al. (2015)

  40. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Peng et al. (2015)

  41. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Credit : Sukhbaatar et al. (2015)

  42. Introduction Reasoning with Neural Networks Questions References Reasoning about facts SLD resolution.

  43. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of?

  44. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of? A: cat (99.74%)

  45. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of? A: cat (99.74%) is emily afraid of gertrude?

  46. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts mice are afraid of sheep wolves are afraid of cats jessica is a wolf sheep are afraid of cats winona is a mouse cats are afraid of mice gertrude is a cat emily is a wolf Questions what is jessica afraid of? A: cat (99.74%) is emily afraid of gertrude? A: cat (71.79%)

  47. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts the triangle is to the left of the red square the pink rectangle is below the triangle Questions is the red square to the right of the pink rectangle?

  48. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts the triangle is to the left of the red square the pink rectangle is below the triangle Questions is the red square to the right of the pink rectangle? A: yes (87%)

  49. Introduction Reasoning with Neural Networks Questions References Reasoning about facts Testing Memory Networks Facts the triangle is to the left of the red square the pink rectangle is below the triangle Questions is the red square to the right of the pink rectangle? A: yes (87%) is the red square to the left of the pink rectangle?

Recommend

Neural Module Networks for Reasoning Over Text Nitish Gupta , Kevin Lin , Dan Roth , Sameer Singh

Neural Module Networks for Reasoning Over Text Nitish Gupta , Kevin Lin , Dan Roth , Sameer Singh

Neural Module Networks for Reasoning Over Text Nitish Gupta , Kevin Lin , Dan Roth , Sameer Singh & Matt Gardner Presented by: Jigyasa Gupta Neural Modules Introduced in the paper Deep Compositional Question Answering with Neural

780 views • 54 slides
Evolution of Connections in SHRUTI Networks Joe Townsend, Ed Keedwell and Antony Galton

Evolution of Connections in SHRUTI Networks Joe Townsend, Ed Keedwell and Antony Galton

Evolution of Connections in SHRUTI Networks Joe Townsend, Ed Keedwell and Antony Galton Motivation Neural-Symbolic Reasoning: Adaptable representations of logic programs in neural networks. Can help us understand how reasoning might be

1.05k views • 40 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides
Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides
Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What well cover ... f ( x ) computer vision architectures - convolutional networks - data augmentation 1 - residual networks ... ...

1.29k views • 75 slides
Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC

Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC

Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC www.kit.edu KIT The Research University in the Helmholtz Association Neural Network Concept (very) remotely brain inspired computational system

479 views • 25 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
7. Artificial neural networks Introduction to neural networks Despite struggling to understand

7. Artificial neural networks Introduction to neural networks Despite struggling to understand

7. Artificial neural networks Introduction to neural networks Despite struggling to understand intricacies of protein, cell, and network function within the brain, neuroscientists would agree on the following simplistic description of how the

157 views • 11 slides
Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons

Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons

Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms10ms cycle

804 views • 47 slides
Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A

Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A

Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Why Model Neural Networks? The nervous system is a

1.22k views • 23 slides
Modular Neural Networks CPSC 533 Franco Lee Ian Ko Modular Neural Networks What is it ? Dif

Modular Neural Networks CPSC 533 Franco Lee Ian Ko Modular Neural Networks What is it ? Dif

Modular Neural Networks CPSC 533 Franco Lee Ian Ko Modular Neural Networks What is it ? Dif f erent models of neural net w orks combined int o a single syst em. Each single net w ork is made int o a module t hat can be f reely int

289 views • 25 slides
Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0 L0 M*V Processor N N N N N N Project Brainwave Pretrained DNN Model Scalable DNN Hardware Neural Processing Unit Microservice Extract

525 views • 30 slides
Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural circuits David Sussillo with Valerio Mante and Bill Newsome Table of contents Introduc(on Training recurrent neural networks(RNNs) Understanding how

1.57k views • 105 slides

More recommend