ece 6504 deep learning for perception
play

ECE 6504: Deep Learning for Perception Topics: LSTMs (intuition - PowerPoint PPT Presentation

Feb 27, 2023 •225 likes •410 views

ECE 6504: Deep Learning for Perception Topics: LSTMs (intuition and variants) [Abhishek:] Lua / Torch Tutorial Dhruv Batra Virginia Tech Administrativia HW3 Out today Due in 2 weeks Please please please please please

  1. ECE 6504: Deep Learning for Perception Topics: – LSTMs (intuition and variants) – [Abhishek:] Lua / Torch Tutorial Dhruv Batra Virginia Tech

  2. Administrativia • HW3 – Out today – Due in 2 weeks – Please please please please please start early – https://computing.ece.vt.edu/~f15ece6504/homework3/ (C) Dhruv Batra 2

  3. RNN • Basic block diagram (C) Dhruv Batra 3 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  4. Key Problem • Learning long-term dependencies is hard (C) Dhruv Batra 4 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  5. Meet LSTMs • How about we explicitly encode memory? (C) Dhruv Batra 5 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  6. LSTMs Intuition: Memory • Cell State / Memory (C) Dhruv Batra 6 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  7. LSTMs Intuition: Forget Gate • Should we continue to remember this “bit” of information or not? (C) Dhruv Batra 7 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  8. LSTMs Intuition: Input Gate • Should we update this “bit” of information or not? – If so, with what? (C) Dhruv Batra 8 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  9. LSTMs Intuition: Memory Update • Forget that + memorize this (C) Dhruv Batra 9 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  10. LSTMs Intuition: Output Gate • Should we output this “bit” of information to “deeper” layers? (C) Dhruv Batra 10 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  11. LSTMs Intuition: Output Gate • Should we output this “bit” of information to “deeper” layers? (C) Dhruv Batra 11 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  12. LSTMs • A pretty sophisticated cell (C) Dhruv Batra 12 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  13. LSTM Variants #1: Peephole Connections • Let gates see the cell state / memory (C) Dhruv Batra 13 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  14. LSTM Variants #2: Coupled Gates • Only memorize new if forgetting old (C) Dhruv Batra 14 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  15. LSTM Variants #3: Gated Recurrent Units • Changes: – No explicit memory; memory = hidden output – Z = memorize new and forget old (C) Dhruv Batra 15 Image Credit: Christopher Olah (http://colah.github.io/posts/2015-08-Understanding-LSTMs/)

  16. RMSProp Intuition • Gradients ≠ Direction to Opt – Gradients point in the direction of steepest ascent locally – Not where we want to go long term • Mismatch gradient magnitudes – magnitude large = we should travel a small distance – magnitude small = we should travel a large distance (C) Dhruv Batra 16 Image Credit: Geoffrey Hinton

  17. RMSProp Intuition • Keep track of previous gradients to get an idea of magnitudes over batch • Divide by this accumulate (C) Dhruv Batra 17

Recommend

ECE 6504: Deep Learning for Perception Topics: (Finish) Backprop Convolutional Neural

ECE 6504: Deep Learning for Perception Topics: (Finish) Backprop Convolutional Neural

ECE 6504: Deep Learning for Perception Topics: (Finish) Backprop Convolutional Neural Nets Dhruv Batra Virginia Tech Administrativia Presentation Assignments https://docs.google.com/spreadsheets/d/

925 views • 67 slides
ECE 6504: Deep Learning for Perception Topics: Recurrent Neural Networks (RNNs) BackProp

ECE 6504: Deep Learning for Perception Topics: Recurrent Neural Networks (RNNs) BackProp

ECE 6504: Deep Learning for Perception Topics: Recurrent Neural Networks (RNNs) BackProp Through Time (BPTT) Vanishing / Exploding Gradients [Abhishek:] Lua / Torch Tutorial Dhruv Batra Virginia Tech Administrativia HW3

458 views • 25 slides
(Deep) Learning for Robot Perception and Navigation Wolfram Burgard Deep Learning for Robot

(Deep) Learning for Robot Perception and Navigation Wolfram Burgard Deep Learning for Robot

(Deep) Learning for Robot Perception and Navigation Wolfram Burgard Deep Learning for Robot Perception (and Navigation) Lifeng Bo, Claas Bollen, Thomas Brox, Andreas Eitel, Dieter Fox, Gabriel L. Oliveira, Luciano Spinello, Jost Tobias

509 views • 50 slides
Deep Learning for Perception Robert Platt Northeastern University Perception problems We will

Deep Learning for Perception Robert Platt Northeastern University Perception problems We will

Deep Learning for Perception Robert Platt Northeastern University Perception problems We will focus on these applications We will ignore these applications image segmentation speech-to-text natural language processing

933 views • 60 slides
Convolutional Neural Networks (Application in Object and Scene

Convolutional Neural Networks (Application in Object and Scene

Convolutional Neural Networks (Application in Object and Scene Recognition) Harsh Agrawal (Sept 8 th , 2015) ECE: 6504, Deep Learning For Perception Contents Y. LeCun, L.

732 views • 54 slides
PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING ARJUN

PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING ARJUN

PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING ARJUN CHANDRASEKARAN DEEP LEARNING AND PERCEPTION (ECE 6504) PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING

1.07k views • 62 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Parameter Learning Structure Learning Readings: KF 18.1, 18.3; Barber 9.5, 10.4 Dhruv Batra Virginia

351 views • 32 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Structure Learning Readings: KF 18.4; Barber 9.5, 10.4 Dhruv Batra Virginia Tech Administrativia HW1

596 views • 30 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Dhruv Batra Virginia Tech What is this class about? Some of the most exciting developments in Machine Learning, AI, Statistics &

925 views • 61 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets: Inference Marginals, MPE, MAP Variable Elimination Readings: KF 9.1,9.2; Barber 5.1 Dhruv Batra Virginia Tech

667 views • 22 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics: Bayes Nets: Representation/Semantics v-structures Probabilistic influence, Active Trails Readings: Barber 3.3; KF

322 views • 16 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics: Bayes Nets: Representation/Semantics d-separation, Local Markov Assumption Markov Blanket I-equivalence, (Minimal)

569 views • 28 slides
Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and

Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and

Natural Language Processing with Deep Learning LSTM, GRU, and applications in summarization and contextualized word embeddings Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception

1.11k views • 51 slides
Towards Weakly-Supervised Visual Understanding Zhiding Yu Learning & Perception Research,

Towards Weakly-Supervised Visual Understanding Zhiding Yu Learning & Perception Research,

Towards Weakly-Supervised Visual Understanding Zhiding Yu Learning & Perception Research, NVIDIA zhidingy@nvidia.com Introduction The Benefit of Big Data and Computation Power Figure credit: Kaiming He et al., Deep Residual Learning for

884 views • 39 slides
Natural Language Processing with Deep Learning Footprint of Societal Biases in NLP Navid

Natural Language Processing with Deep Learning Footprint of Societal Biases in NLP Navid

Natural Language Processing with Deep Learning Footprint of Societal Biases in NLP Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception Agenda Motivation Bias in word

822 views • 41 slides
ECE6504 Deep Learning for Perception Introduction to CAFFE Ashwin Kalyan V (C) Dhruv Batra

ECE6504 Deep Learning for Perception Introduction to CAFFE Ashwin Kalyan V (C) Dhruv Batra

ECE6504 Deep Learning for Perception Introduction to CAFFE Ashwin Kalyan V (C) Dhruv Batra 2 Logistic Regression as a Cascade (C) Dhruv Batra 3 Slide Credit: Marc'Aurelio Ranzato, Yann LeCun Logistic Regression as a Cascade (C) Dhruv

1.39k views • 54 slides
Scaling-Up Deep Learning For Autonomous Vehicles JOSE M. ALVAREZ | | San Jose 2019 1 NVIDIA

Scaling-Up Deep Learning For Autonomous Vehicles JOSE M. ALVAREZ | | San Jose 2019 1 NVIDIA

Scaling-Up Deep Learning For Autonomous Vehicles JOSE M. ALVAREZ | | San Jose 2019 1 NVIDIA AI-Infra 2 AI-Infra Team One of our top Goals Industry grade Deep Learning to take AV Perception DNN into production, tested in multiple

1.25k views • 108 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides
ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image

314 views • 26 slides
Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice Scaling up DL 2 What is Deep Learning? 3 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY SECURITY & DEFENSE MEDIA &

1.45k views • 39 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
Natural Language Processing with Deep Learning Sequence-to-sequence Models with Attention Navid

Natural Language Processing with Deep Learning Sequence-to-sequence Models with Attention Navid

Natural Language Processing with Deep Learning Sequence-to-sequence Models with Attention Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception Agenda Sequence-to-sequence

900 views • 64 slides
Natural Language Processing with Deep Learning Neural Information Retrieval Navid Rekab-Saz

Natural Language Processing with Deep Learning Neural Information Retrieval Navid Rekab-Saz

Natural Language Processing with Deep Learning Neural Information Retrieval Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Institute of Computational Perception Agenda Information Retrieval Crash course

666 views • 56 slides
Natural Language Processing with Deep Learning Sentiment Analysis with Machine Learning Navid

Natural Language Processing with Deep Learning Sentiment Analysis with Machine Learning Navid

Natural Language Processing with Deep Learning Sentiment Analysis with Machine Learning Navid Rekab-Saz navid.rekabsaz@jku.at Institute of Computational Perception Agenda Introduction to Machine Learning Sentiment Analysis

844 views • 53 slides

More recommend