neural networks deep learning
play

Neural Networks - Deep Learning Artificial Intelligence @ Allegheny - PowerPoint PPT Presentation

Sep 24, 2023 •213 likes •709 views

Neural Networks - Deep Learning Artificial Intelligence @ Allegheny College Janyl Jumadinova March 4-6, 2020 Credit: Google Workshop Janyl Jumadinova March 4-6, 2020 1 / 35 Neural Networks - Deep Learning Neural Networks Janyl Jumadinova

  1. Neural Networks - Deep Learning Artificial Intelligence @ Allegheny College Janyl Jumadinova March 4-6, 2020 Credit: Google Workshop Janyl Jumadinova March 4-6, 2020 1 / 35 Neural Networks - Deep Learning

  2. Neural Networks Janyl Jumadinova March 4-6, 2020 2 / 35 Neural Networks - Deep Learning

  3. Neural Networks Janyl Jumadinova March 4-6, 2020 3 / 35 Neural Networks - Deep Learning

  4. Neural Networks Neural computing requires a number of neurons , to be connected together into a neural network . Neurons are arranged in layers. Janyl Jumadinova March 4-6, 2020 4 / 35 Neural Networks - Deep Learning

  5. Neural Networks Neural computing requires a number of neurons , to be connected together into a neural network . Neurons are arranged in layers. Two main hyperparameters that control the architecture or topology of the network: 1) the number of layers, and 2) the number of nodes in each hidden layer. Janyl Jumadinova March 4-6, 2020 4 / 35 Neural Networks - Deep Learning

  6. Activation Functions The activation function is generally non-linear. Linear functions are limited because the output is simply proportional to the input. Janyl Jumadinova March 4-6, 2020 5 / 35 Neural Networks - Deep Learning

  7. Activation Functions Janyl Jumadinova March 4-6, 2020 6 / 35 Neural Networks - Deep Learning

  8. Network structures Two phases in each iteration: 1 Calculating the predicted output y , known as feed-forward 2 Updating the weights and biases, known as backpropagation Janyl Jumadinova March 4-6, 2020 7 / 35 Neural Networks - Deep Learning

  9. Feed-forward example Feed-forward networks: Single-layer perceptrons Multi-layer perceptrons Janyl Jumadinova March 4-6, 2020 8 / 35 Neural Networks - Deep Learning

  10. Feed-forward example Feed-forward networks: Single-layer perceptrons Multi-layer perceptrons Janyl Jumadinova March 4-6, 2020 8 / 35 Neural Networks - Deep Learning

  11. Single-layer Perceptrons Output units all operate separately – no shared weights. Adjusting weights moves the location, orientation, and steepness of cliff. Janyl Jumadinova March 4-6, 2020 9 / 35 Neural Networks - Deep Learning

  12. Multi-layer Perceptrons Layers are usually fully connected. Numbers of hidden units typically chosen by hand. Janyl Jumadinova March 4-6, 2020 10 / 35 Neural Networks - Deep Learning

  13. Neural Networks Janyl Jumadinova March 4-6, 2020 11 / 35 Neural Networks - Deep Learning

  14. Neural Networks Janyl Jumadinova March 4-6, 2020 12 / 35 Neural Networks - Deep Learning

  15. Neural Networks A fully connected NN layer Janyl Jumadinova March 4-6, 2020 13 / 35 Neural Networks - Deep Learning

  16. Implementation as Matrix Multiplication Janyl Jumadinova March 4-6, 2020 14 / 35 Neural Networks - Deep Learning

  17. Non-Linear Data Distributions Janyl Jumadinova March 4-6, 2020 15 / 35 Neural Networks - Deep Learning

  18. Janyl Jumadinova March 4-6, 2020 16 / 35 Neural Networks - Deep Learning

  19. Deep Learning Most current machine learning works well because of human-designed representations and input features. Machine learning becomes just optimizing weights to best make a final prediction. Janyl Jumadinova March 4-6, 2020 17 / 35 Neural Networks - Deep Learning

  20. Deep Learning Most current machine learning works well because of human-designed representations and input features. Machine learning becomes just optimizing weights to best make a final prediction. Deep learning algorithms attempt to learn multiple levels of representation of increasing complexity/abstraction. Janyl Jumadinova March 4-6, 2020 17 / 35 Neural Networks - Deep Learning

  21. Deep Learning Each neuron implements a relatively simple mathematical function. y = g ( w · x + b ) Janyl Jumadinova March 4-6, 2020 18 / 35 Neural Networks - Deep Learning

  22. Deep Learning Each neuron implements a relatively simple mathematical function. y = g ( w · x + b ) The composition of 10 6 − 10 9 such functions is powerful. Janyl Jumadinova March 4-6, 2020 18 / 35 Neural Networks - Deep Learning

  23. Deep Learning Book: http://www.deeplearningbook.org/ Chapter 5 “A core idea in deep learning is that we assume that the data was generated by the composition of factors or features, potentially at multiple levels in a hierarchy.” Janyl Jumadinova March 4-6, 2020 19 / 35 Neural Networks - Deep Learning

  24. Results get better (to a degree) with: more data bigger models more computation Janyl Jumadinova March 4-6, 2020 20 / 35 Neural Networks - Deep Learning

  25. Results get better (to a degree) with: more data bigger models more computation Better algorithms, new insights and improved methods help, too! Janyl Jumadinova March 4-6, 2020 20 / 35 Neural Networks - Deep Learning

  26. TensorFlow Janyl Jumadinova March 4-6, 2020 21 / 35 Neural Networks - Deep Learning

  27. Adoption of Deep Learning Tools on GitHub Janyl Jumadinova March 4-6, 2020 22 / 35 Neural Networks - Deep Learning

  28. TensorFlow Epoch : a training iteration (one pass through the dataset). Batch : Portion of the dataset (number of samples after dataset has been divided). Regularization : a set of techniques that helps learning models to converge ( http://www.godeep.ml/regularization-using-tensorflow/ ). Janyl Jumadinova March 4-6, 2020 23 / 35 Neural Networks - Deep Learning

  29. TensorFlow Operates over tensors : n-dimensional arrays Janyl Jumadinova March 4-6, 2020 24 / 35 Neural Networks - Deep Learning

  30. TensorFlow Operates over tensors : n-dimensional arrays Using a flow graph : data flow computation framework Janyl Jumadinova March 4-6, 2020 24 / 35 Neural Networks - Deep Learning

  31. TensorFlow Operates over tensors : n-dimensional arrays Using a flow graph : data flow computation framework Janyl Jumadinova March 4-6, 2020 24 / 35 Neural Networks - Deep Learning

  32. TensorFlow 5.7 ← Scalar Number, Float, etc. Janyl Jumadinova March 4-6, 2020 25 / 35 Neural Networks - Deep Learning

  33. TensorFlow Janyl Jumadinova March 4-6, 2020 26 / 35 Neural Networks - Deep Learning

  34. TensorFlow Janyl Jumadinova March 4-6, 2020 27 / 35 Neural Networks - Deep Learning

  35. TensorFlow Tensors have a Shape that is described with a vector Janyl Jumadinova March 4-6, 2020 28 / 35 Neural Networks - Deep Learning

  36. TensorFlow Tensors have a Shape that is described with a vector [1000 , 256 , 256 , 3] 10000 Images Each Image has 256 Rows Each Row has 256 Pixels Each Pixel has 3 values (RGB) Janyl Jumadinova March 4-6, 2020 28 / 35 Neural Networks - Deep Learning

  37. TensorFlow Computation is a dataflow graph Janyl Jumadinova March 4-6, 2020 29 / 35 Neural Networks - Deep Learning

  38. TensorFlow Computation is a dataflow graph with tensors Janyl Jumadinova March 4-6, 2020 30 / 35 Neural Networks - Deep Learning

  39. TensorFlow Computation is a dataflow graph with state Janyl Jumadinova March 4-6, 2020 31 / 35 Neural Networks - Deep Learning

  40. Core TensorFlow data structures and concepts Graph : A TensorFlow computation, represented as a dataflow graph: - collection of ops that may be executed together as a group. Janyl Jumadinova March 4-6, 2020 32 / 35 Neural Networks - Deep Learning

  41. Core TensorFlow data structures and concepts Graph : A TensorFlow computation, represented as a dataflow graph: - collection of ops that may be executed together as a group. Operation : a graph node that performs computation on tensors Janyl Jumadinova March 4-6, 2020 32 / 35 Neural Networks - Deep Learning

  42. Core TensorFlow data structures and concepts Graph : A TensorFlow computation, represented as a dataflow graph: - collection of ops that may be executed together as a group. Operation : a graph node that performs computation on tensors Tensor : a handle to one of the outputs of an Operation: - provides a means of computing the value in a TensorFlow Session. Janyl Jumadinova March 4-6, 2020 32 / 35 Neural Networks - Deep Learning

  43. TensorFlow Constants Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  44. TensorFlow Constants Placeholders : must be fed with data on execution. Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  45. TensorFlow Constants Placeholders : must be fed with data on execution. Variables : a modifiable tensor that lives in TensorFlow’s graph of interacting operations. Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  46. TensorFlow Constants Placeholders : must be fed with data on execution. Variables : a modifiable tensor that lives in TensorFlow’s graph of interacting operations. Session : encapsulates the environment in which Operation objects are executed, and Tensor objects are evaluated. Janyl Jumadinova March 4-6, 2020 33 / 35 Neural Networks - Deep Learning

  47. TensorFlow Janyl Jumadinova March 4-6, 2020 34 / 35 Neural Networks - Deep Learning

  48. TensorFlow https://playground.tensorflow.org Janyl Jumadinova March 4-6, 2020 35 / 35 Neural Networks - Deep Learning

Recommend

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

LRP Ariyan Zarei Motivation Having More interpretable Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks to have more interpretable Papers and Demo Introduction Machine Learning models

373 views • 35 slides
Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning and AI y t-1 y t y t+1 Computer vision h t-1 h t h t+1 Language translation Speech recognition h t-1 h t h t+1 Question answering x t-1 x t x t+1

454 views • 43 slides
PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 5: Neural Networks and Deep Learning November

PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 5: Neural Networks and Deep Learning November

PATTERN RECOGNITION AND MACHINE LEARNING Slide Set 5: Neural Networks and Deep Learning November 2019 Heikki Huttunen heikki.huttunen@tuni.fi Signal Processing Tampere University default Traditional Neural Networks Neural networks have

571 views • 21 slides
6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Monday (2/5) 30 minutes Topics: Optimization Basic neural networks Neural Network

619 views • 41 slides
Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning Tasks that were once assumed to be purely in the human domain

1.92k views • 150 slides
Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit: Vlad Morariu Training (Deep) Neural Networks Computational graphs Improvements to gradient descent Stochastic gradient descent Momentum

814 views • 21 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning 2 Recap : NNets and the brain In their basic form, NNets

2.14k views • 145 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Neural Networks: Optimization Part 1 Intro to Deep Learning, Fall 2017 Story so far Neural

Neural Networks: Optimization Part 1 Intro to Deep Learning, Fall 2017 Story so far Neural

Neural Networks: Optimization Part 1 Intro to Deep Learning, Fall 2017 Story so far Neural networks are universal approximators Can model any odd thing Provided they have the right architecture We must train them to approximate

1.37k views • 134 slides
DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology

DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology

DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology October 2016 DEEP LEARNING Artificial neural networks Many layers of abstractions Outperforms traditional methods in: Image

824 views • 61 slides
Unsupervised Learning and Inverse Problems with Deep Neural Networks J oan Bruna, Stphane

Unsupervised Learning and Inverse Problems with Deep Neural Networks J oan Bruna, Stphane

Unsupervised Learning and Inverse Problems with Deep Neural Networks J oan Bruna, Stphane Mallat, Ivan Dokmanic, Martin de Hoop cole Normale Suprieure www.di.ens.fr/data Deep Convolutional Networks ( x ) x ( u ) L j L 1 f M (

520 views • 47 slides
Data Mining II Neural Networks and Deep Learning Heiko Paulheim Deep Learning A recent

Data Mining II Neural Networks and Deep Learning Heiko Paulheim Deep Learning A recent

Data Mining II Neural Networks and Deep Learning Heiko Paulheim Deep Learning A recent hype topic 03/26/19 Heiko Paulheim 2 Deep Learning Just the same as artificial neural networks with a new buzzword? 03/26/19 Heiko Paulheim

895 views • 76 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics Activation Functions Stochastic Gradient Descent (SGD) Regularization (Dropout) Training Tips and Tricks Neural Network (NN)

828 views • 45 slides
AI and Predictive Analytics in Data-Center Environments Neural Networks and Deep Learning Josep

AI and Predictive Analytics in Data-Center Environments Neural Networks and Deep Learning Josep

AI and Predictive Analytics in Data-Center Environments Neural Networks and Deep Learning Josep Ll. Berral @BSC Intel Academic Education Mindshare Initiative for AI Introduction Neural networks attempt to imitate the brain neuron

339 views • 21 slides
Deep Learning - Theory and Practice Deep Neural Networks 12-03-2020

Deep Learning - Theory and Practice Deep Neural Networks 12-03-2020

Deep Learning - Theory and Practice Deep Neural Networks 12-03-2020 http://leap.ee.iisc.ac.in/sriram/teaching/DL20/ deeplearning.cce2020@gmail.com Logistic Regression 2- class logistic regression Maximum likelihood solution K-class

558 views • 24 slides

More recommend