neural networks prediction i e the forward pass
play

Neural Networks: Prediction (i.e. the forward pass) Machine - PowerPoint PPT Presentation

Nov 14, 2022 •289 likes •470 views

Neural Networks: Prediction (i.e. the forward pass) Machine Learning Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, 1 Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others Neural Networks What is a

  1. Neural Networks: Prediction (i.e. the forward pass) Machine Learning Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, 1 Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others

  2. Neural Networks • What is a neural network? • Predicting with a neural network • Training neural networks • Practical concerns 2

  3. This lecture • What is a neural network? • Predicting with a neural network • Training neural networks • Practical concerns 3

  4. Let us consider an example network output We will use this example network as to introduce the general principle of how to make predictions with a neural network. 4

  5. Let us consider an example network output Naming conventions for this example Inputs: x • Hidden: z • 𝑧 Output: y • 1 𝑨 ! 𝑨 " 1 𝑦 ! 𝑦 " 5

  6. Let us consider an example network output Naming conventions for this example Inputs: x • Hidden: z • 𝑧 Output: y • 1 𝑨 ! 𝑨 " Bias feature, always 1 1 𝑦 ! 𝑦 " 6

  7. Let us consider an example network output Naming conventions for this example Inputs: x • Hidden: z • 𝑧 Output: y • Sigmoid activations 1 𝑨 ! 𝑨 " Bias feature, always 1 1 𝑦 ! 𝑦 " 7

  8. Let us consider an example network output Naming conventions for this example Inputs: x • Linear activation Hidden: z • 𝑧 Output: y • Sigmoid activations 1 𝑨 ! 𝑨 " Bias feature, always 1 1 𝑦 ! 𝑦 " 8

  9. Let us consider an example network output Naming Convention for Weights -.)/0- - 1.20) 𝑧 𝑥 ()*+,-* #$% 𝑥 "! #$% 𝑥 &! #$% 𝑥 !! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 9

  10. Let us consider an example network output Naming Convention for Weights -.)/0- - 1.20) 𝑧 𝑥 ()*+,-* #$% 𝑥 "! #$% 𝑥 &! #$% 𝑥 !! #$% 𝑥 &! 1 𝑨 ! 𝑨 " From neuron #0 ' 𝑥 &! to neuron #1 in ' 𝑥 "" output layer 1 𝑦 ! 𝑦 " 10

  11. How to predict: The forward pass Given an input x , how is the output predicted output 𝑧 #$% 𝑥 "! #$% 𝑥 &! #$% 𝑥 !! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 11

  12. The forward pass Given an input x , how is the output predicted output 𝑧 #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 12

  13. The forward pass Given an input x , how is the output predicted output 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 13

  14. The forward pass Given an input x , how is the output predicted output % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 14

  15. The forward pass Given an input x , how is the output predicted output % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " ' 𝑥 &! ' 𝑥 "" 1 𝑦 ! 𝑦 " 15

  16. The forward pass output Given an input x , how is the output predicted % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " In general, before visiting (i.e. computing) the value of a node, visit all nodes that serve as ' 𝑥 &! inputs to it. ' 𝑥 "" 1 𝑦 ! 𝑦 " 16

  17. The forward pass output Given an input x , how is the output predicted % + 𝑥 !! % 𝑨 ! + 𝑥 $! % 𝑨 $ output y = 𝑥 "! 𝑧 # + 𝑥 !$ # 𝑦 ! + 𝑥 $$ # 𝑦 $ ) 𝑨 $ = 𝜏(𝑥 "$ #$% 𝑥 "! #$% 𝑥 &! # + 𝑥 !! # 𝑦 ! + 𝑥 $! # 𝑦 $ ) #$% 𝑥 !! z ! = 𝜏(𝑥 "! 1 𝑨 ! 𝑨 " In general, before visiting (i.e. computing) the value of a node, visit all nodes that serve as inputs to it. ' 𝑥 &! ' 𝑥 "" Questions? 1 𝑦 ! 𝑦 " 17

Recommend

Neural Networks for Time Series Prediction 15-486/782: Artificial Neural Networks Fall 2006

Neural Networks for Time Series Prediction 15-486/782: Artificial Neural Networks Fall 2006

Neural Networks for Time Series Prediction 15-486/782: Artificial Neural Networks Fall 2006 (based on earlier slides by Dave Touretzky and Kornel Laskowski) What is a Time Series? A sequence of vectors (or scalars) which depend on time t . In

975 views • 42 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
CS4501: Introduction to Computer Vision Neural Networks (NNs) Artificial Neural Networks (ANNs)

CS4501: Introduction to Computer Vision Neural Networks (NNs) Artificial Neural Networks (ANNs)

CS4501: Introduction to Computer Vision Neural Networks (NNs) Artificial Neural Networks (ANNs) Multi-layer Perceptrons (MLPs Previous Neural Networks The Perceptron Model The Multi-layer Perceptron (MLP) Forward-pass in an MLP

1.54k views • 40 slides
Neural Networks for Time Series Prediction 15-496/782: Artificial Neural Networks Kornel

Neural Networks for Time Series Prediction 15-496/782: Artificial Neural Networks Kornel

Neural Networks for Time Series Prediction 15-496/782: Artificial Neural Networks Kornel Laskowski Spring 2004 (based on slides by D. Touretzky, Spring 2002) What is a Time Series? A sequence of vectors (or scalars) which depend on time t . In

719 views • 42 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Prediction of genetic Values using Neural Networks Paulino Perez 1 Daniel Gianola 2 Jose Crossa 1 1

Prediction of genetic Values using Neural Networks Paulino Perez 1 Daniel Gianola 2 Jose Crossa 1 1

Prediction of genetic Values using Neural Networks Paulino Perez 1 Daniel Gianola 2 Jose Crossa 1 1 CIMMyT-Mexico 2 University of Wisconsin, Madison. September, 2014 SLU,Sweden Prediction of genetic Values using Neural Networks 1/26 Contents

597 views • 26 slides
Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

21-23 20 Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on Prediction Window Tushar D. Gaikwad , Zachary D. Asher, Aaron Rabinowitz, Farhang Motallebiaraghi, Thomas Bradley, Alvis Fong, Rick Meyer

1.19k views • 27 slides
Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on

Vehicle Velocity Prediction Using Artificial Neural Networks and Effect of Real-World Signals on Prediction Window by, Tushar D. Gaikwad Committee Dr. Zachary Asher, Chair Dr. Richard Meyer Dr. Alvis Fong Western Michigan University 1

906 views • 65 slides
Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and

Closed Loop Neural-Symbolic Learning via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning Motivation NS-RL Neural Symbolic Input Prediction Error Network Reasoning Ground Truth Forward pass

1.13k views • 27 slides
Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton,

Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton,

Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others This lecture What is a neural network? Training

1.28k views • 93 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Neural Networks: Computation + Gradient Descent LING572 Advanced Statistical Methods in NLP

Neural Networks: Computation + Gradient Descent LING572 Advanced Statistical Methods in NLP

Neural Networks: Computation + Gradient Descent LING572 Advanced Statistical Methods in NLP February 27 2020 1 Todays Outline Computation: the forward pass Functional form / matrix notation Parameters and Hyperparameters

832 views • 40 slides
Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are the commonest type of neural

534 views • 32 slides
NAXOS 2018 Prediction of wastewater N2O emissions using artificial Neural Networks. Vasilaki V.,

NAXOS 2018 Prediction of wastewater N2O emissions using artificial Neural Networks. Vasilaki V.,

NAXOS 2018 Prediction of wastewater N2O emissions using artificial Neural Networks. Vasilaki V., Mattias T., Angadi V. C., Sousa P., Mousavi A., Katsou E. 15/5/2018 Overview 21 June 2018 Introduction Aim and objectives Methodology

266 views • 22 slides
Link Prediction Based on Graph Neural Networks Muhan Zhang and Yixin Chen, NeurIPS 2018 Link

Link Prediction Based on Graph Neural Networks Muhan Zhang and Yixin Chen, NeurIPS 2018 Link

Link Prediction Based on Graph Neural Networks Muhan Zhang and Yixin Chen, NeurIPS 2018 Link Prediction (LP) Problem Given an incomplete network, predict whether two nodes are likely to have a link. Applications: Friend recommendation in

987 views • 11 slides
2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1

2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1

2nd Session Machine learning: feed-forward neural networks and self-organizing maps 1 Recommended reading J. Zupan, J. Gasteiger, Neural Networks in Chemistry and Drug Design: An Introduction, Wiley-VCH, Weinheim, 1999.

959 views • 62 slides
Continuous Fundamental Frequency Prediction with Deep Neural Networks Blint Pl Tth, Tams

Continuous Fundamental Frequency Prediction with Deep Neural Networks Blint Pl Tth, Tams

Continuous Fundamental Frequency Prediction with Deep Neural Networks Blint Pl Tth, Tams Gbor Csap csapot@tmit.bme.hu Budapest University of Technology and Economics http://smartlab.tmit.bme.hu Introduction Deep Learning: New era

510 views • 22 slides
Natural Language Processing with Deep Learning Neural Networks a Walkthrough Navid Rekab-Saz

Natural Language Processing with Deep Learning Neural Networks a Walkthrough Navid Rekab-Saz

Natural Language Processing with Deep Learning Neural Networks a Walkthrough Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Agenda Introduction Non-linearities Forward pass &

670 views • 47 slides
Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction SI

Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction SI

Addressing Delayed Feedback for Continuous Training with Neural Networks in CTR prediction SI Ktena, A Tejani, L Theis, P Kumar Myana, D Dilipkumar, F Huszr, S Yoo, W Shi RecSys 2019 Background Why continuous training? 2 Background Why

488 views • 36 slides
Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap from last time Feed-Forward Neural Network: Multilayer Perceptron + 0 ) = (

1.04k views • 93 slides

More recommend