Neural Networks and Handwriting Recognition Background Neural - PowerPoint PPT Presentation

Neural Networks and Handwriting Recognition Steven Sloss Math 164 Neural Networks and Handwriting Recognition Background Neural Networks Neural Network Steven Sloss Structure Training Neural Networks Math 164 Motivation Problem Solution 26 April 2007 Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Presentation Outline Neural Networks and Background 1 Handwriting Recognition Neural Networks 2 Steven Sloss Math 164 Neural Network Structure Background Training Neural Networks Neural Networks Motivation 3 Neural Network Structure Training Neural Problem Networks 4 Motivation Solution 5 Problem Single-Character Recognition Solution Single-Character Multiple Character Recognition Recognition Multiple Character Recognizing Math Recognition Recognizing Math Outlook 6 Outlook

Artificial Intelligence Neural Networks and Handwriting Recognition Steven Sloss Today’s computers can perform many computations much, Math 164 much faster than a human being can. Background Example Neural Networks Neural Network Integrate Structure � 1 Training Neural Networks � 1 − x 2 dx Motivation 0 Problem My laptop: 0.3867... seconds. Solution Single-Character Me: ∼ 1.3 minutes Recognition Multiple Character Recognition Recognizing Math Outlook

Artificial Intelligence, Contd. Neural There are many areas where computers fall short, however. Networks and Handwriting Recognition Steven Sloss Math 164 Background Neural Networks Neural Network Structure Training Neural Networks Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Artificial Intelligence, Contd. Neural There are many areas where computers fall short, however. Networks and Handwriting Example Recognition Steven Sloss Find the swingset: Math 164 Background Neural Networks Neural Network Structure Training Neural Networks Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math My 2 year old neighbor: ∼ 1.2 seconds Outlook A computer: ???

Artificial Intelligence, Contd. Neural Networks and Handwriting Recognition Steven Sloss Math 164 Background Artificial Intelligence is the field of mathematics and Neural Networks computer science that tries to give computers human-like Neural Network cognitive abilities. Structure Training Neural Networks Neural Networks are an important way to do this. Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Presentation Outline Neural Networks and Background 1 Handwriting Recognition Neural Networks 2 Steven Sloss Math 164 Neural Network Structure Background Training Neural Networks Neural Networks Motivation 3 Neural Network Structure Training Neural Problem Networks 4 Motivation Solution 5 Problem Single-Character Recognition Solution Single-Character Multiple Character Recognition Recognition Multiple Character Recognizing Math Recognition Recognizing Math Outlook 6 Outlook

Neural Networks Neural Networks and Handwriting Recognition Steven Sloss Math 164 Background Neural Neural networks - teaching a computer to do pattern Networks Neural Network recognition like a human brain! Structure Training Neural Networks Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Biological Neural Networks versus Artificial Neural Networks Neural Networks and Handwriting Recognition Steven Sloss Math 164 Background Lots of parallels between artificial and biological neural Neural Networks networks. Neural Network Structure Both biological and artificial neural networks use neurons. Training Neural Networks Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Biological Neural Networks versus Artificial Neural Networks Neural Networks and Handwriting Recognition Steven Sloss Math 164 Background Neural Networks Neural Network Structure Training Neural Networks Motivation Problem Biological neurons : Solution Single-Character Accepts signal from Dentries. Recognition Multiple Upon accepting a signal, that neuron may fire Character Recognition If it fires, a signal is transmitted over the neuron’s axon, Recognizing Math leaving the neuron over the axon terminals Outlook This signal is then transmitted to other neurons or nerves

Biological Neural Networks versus Artificial Neural Networks Neural Networks and Handwriting Recognition Steven Sloss Math 164 Artificial neurons : Artificial neurons are based on digital Background systems (computers) rather than analogue systems Neural (dentries), Networks Neural Network Receives a number of inputs (from other neurons or the Structure Training Neural program itself) Networks Motivation Each input has a weight Problem Each neuron has a activation threshold Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Solving Problems with Neural Networks Neural Networks and Handwriting Recognition Steven Sloss Math 164 Problems not suited to neural networks Background Deterministic problems Neural Networks Programs that can be written with a flowchart Neural Network Structure Training Neural Where the logic of the program is likely to change Networks Motivation Where you must know how the solution was derived Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Solving Problems with Neural Networks Neural Networks and Handwriting Recognition Steven Sloss Math 164 Background Problems suited to neural networks Neural Problems that can’t be solved as a series of steps Networks Neural Network Structure Pattern recognition Training Neural Networks Classification Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

The Neuron Neural Networks and Handwriting Recognition Steven Sloss Math 164 The basic building block of the neural network. Background Neural Individual neurons are connected to one another Networks Neural Network Each connection is assigned a weight. Structure Training Neural Networks These connection weights determine the output of the Motivation neural network Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

The Neuron Input/Output Neural Networks and Receives input from other neurons or the user’s program Handwriting Recognition Sends output to other neurons or the user’s program Steven Sloss A neuron “fires” or “actives” when the sum of its inputs Math 164 �� � Background f ( x ) = K w i g i ( x ) Neural Networks i Neural Network Structure is high enough. We may use one of many activation functions, Training Neural Networks like Motivation Problem y = 1 Solution Single-Character Tanh: Recognition tanh( u ) = e u − e − u Multiple Character e u + e − u Recognition Recognizing Math Sigmoid: Outlook 1 y = 1 + e − x

Neural Network Structure Neural Networks and Handwriting Recognition Steven Sloss Math 164 Background Neural Networks Neural Network Structure Training Neural Networks Motivation Problem Solution Split into two parts, neurons and layers Single-Character Recognition Multiple Character Neurons - basic element. Interconnected, with each Recognition Recognizing connection having a weight. Math Outlook Layers - groups of neurons.

Neuron Connection Weights Neural Networks and Handwriting Recognition Steven Sloss Math 164 Neurons are connected together by weighted connections Background These weights allow the neural network to recognize Neural patterns Networks Neural Network If you adjust the weights, the neural network will recognize Structure Training Neural a different pattern. Networks Motivation Training a neural network is merely adjusting the weights Problem between the neurons until we get the desired output Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Layers of Neurons Neural Neurons are commonly grouped in layers Networks and Handwriting Recognition Layers - groups of neurons that perform similar functions Steven Sloss Three types Math 164 Input layer - receives input from the user 1 Background Output layer - sends data to user 2 Neural Hidden layer - neurons connected only to other neurons Networks 3 Neural Network Structure Training Neural Networks Motivation Problem Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook

Training Neural Networks and Handwriting Recognition Steven Sloss Math 164 Remember: neurons are connected via weighted Background connections, these weights determine the output of the Neural network Networks Neural Network Structure Training methodology: Training Neural Networks Assign random numbers to weights 1 Motivation Determine validity of neural network (see next slides) 2 Problem Adjust weights according to validation results 3 Solution Single-Character Recognition Multiple Character Recognition Recognizing Math Outlook