Introduction to Artificial Intelligence Neural Networks - Deep - PowerPoint PPT Presentation

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016

Neural Networks 2/20

Neural Networks 3/20

Neural Networks Neural computing requires a number of neurons , to be connected together into a neural network . Neurons are arranged in layers. 4/20

Activation Functions ◮ The activation function is generally non-linear. ◮ Linear functions are limited because the output is simply proportional to the input. 5/20

Activation Functions 6/20

Network structures Feed-forward networks: ◮ Single-layer perceptrons ◮ Multi-layer perceptrons 7/20

Feed-forward example 8/20

Single-layer Perceptrons Output units all operate separately – no shared weights. Adjusting weights moves the location, orientation, and steepness of 9/20 cliff.

Multi-layer Perceptrons ◮ Layers are usually fully connected. ◮ Numbers of hidden units typically chosen by hand. 10/20

A neural network for learning word vector ◮ Idea : A word and its context is a posiGve training sample ◮ A random word in that same context gives a negative training sample: 11/20

A neural network for learning word vector 12/20

A neural network for learning word vector These are the word features we want to learn . 13/20

A neural network for learning word vector 14/20

Deep Learning ◮ Most current machine learning works well because of human-designed representations and input features . ◮ Machine learning becomes just optimizing weights to best make a final prediction. 15/20

Deep Learning ◮ Most current machine learning works well because of human-designed representations and input features . ◮ Machine learning becomes just optimizing weights to best make a final prediction. ◮ Deep learning algorithms attempt to learn multiple levels of representation of increasing complexity/abstraction. 15/20

A Deep Architecture 16/20

The Need for Distributed Representations Current NLP systems are incredibly fragile because of their atomic symbol representations 17/20

Handling the recursivity of human language 18/20

Recursive Deep Learning: Building on Word Vector Space Models 19/20

How should we map phrases into a vector space? 20/20