deep learning state of the art 2020 deep learning lecture
play

Deep Learning: State of the Art (2020) Deep Learning Lecture Series - PowerPoint PPT Presentation

Nov 16, 2022 •217 likes •1.1k views

Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020 Outline Deep Learning Growth, Celebrations, and Limitations

  1. Deep Learning: State of the Art (2020)

  2. Deep Learning Lecture Series https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  3. Outline • Deep Learning Growth, Celebrations, and Limitations • Deep Learning and Deep RL Frameworks • Natural Language Processing • Deep RL and Self-Play • Science of Deep Learning and Interesting Directions • Autonomous Vehicles and AI-Assisted Driving • Government, Politics, Policy • Courses, Tutorials, Books • General Hopes for 2020 https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  4. “AI began with an ancient wish to forge the gods.” - Pamela McCorduck, Machines Who Think, 1979 Frankenstein (1818) Ex Machina (2015) Visualized here are 3% of the neurons and 0.0001% of the synapses in the brain. Thalamocortical system visualization via DigiCortex Engine. https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  5. Deep Learning & AI in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created 4.54 billion years ago • Modern humans 300,000 years ago 1700s and beyond: Industrial revolution, steam engine, mechanized factory systems, machine tools • Civilization 12,000 years ago • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  6. Artificial Intelligence in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created 4.54 billion years ago • Modern humans Dreams, mathematical foundations, and engineering in reality. 300,000 years ago Alan Turing, 1951: “It seems probable that once the machine • Civilization thinking method had started, it would not take long to outstrip 12,000 years ago our feeble powers. They would be able to converse with each other to sharpen their wits. At some stage therefore, we should • Written record have to expect the machines to take control." 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  7. Artificial Intelligence in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created 4.54 billion years ago • Modern humans Dreams, mathematical foundations, and engineering in reality. 300,000 years ago Frank Rosenblatt, Perceptron (1957, 1962): Early description and • Civilization engineering of single-layer and multi-layer artificial neural 12,000 years ago networks. • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  8. Artificial Intelligence in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created 4.54 billion years ago • Modern humans Kasparov vs Deep Blue, 1997 300,000 years ago • Civilization 12,000 years ago • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  9. Artificial Intelligence in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created 4.54 billion years ago • Modern humans Lee Sedol vs AlphaGo, 2016 300,000 years ago • Civilization 12,000 years ago • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  10. Artificial Intelligence in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created Robots on four wheels. 4.54 billion years ago • Modern humans 300,000 years ago • Civilization 12,000 years ago • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  11. Artificial Intelligence in Context of Human History We are here Perspective: • Universe created 13.8 billion years ago • Earth created 4.54 billion years ago • Modern humans Robots on two legs. 300,000 years ago • Civilization 12,000 years ago • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  12. History of Deep Learning Ideas and Milestones* • 1943: Neural networks • 1957-62: Perceptron We are here • 1970-86: Backpropagation, RBM, RNN • 1979-98: CNN, MNIST, LSTM, Bidirectional RNN • 2006: “Deep Learning”, DBN • 2009: ImageNet + AlexNet Perspective: • 2014: GANs • Universe created • 2016-17: AlphaGo, AlphaZero 13.8 billion years ago • Earth created • 2017: 2017-19: Transformers 4.54 billion years ago * Dates are for perspective and not as definitive historical • Modern humans record of invention or credit 300,000 years ago • Civilization 12,000 years ago • Written record 5,000 years ago https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  13. Turing Award for Deep Learning • Yann LeCun • Geoffrey Hinton • Yoshua Bengio Turing Award given for: • “The conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.” • (Also, for popularization in the face of skepticism.) https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  14. Early Key Figures in Deep Learning (Not a Complete List by Any Means) • 1943: Walter Pitts and Warren McCulloch Computational models for neural nets • 1957, 1962: Frank Rosenblatt Perceptron (Single-Layer & Multi-Layer) • 1965: Alexey Ivakhnenko and V. G. Lapa Learning algorithm for MLP • 1970: Seppo Linnainmaa Backpropagation and automatic differentiation • 1979: Kunihiko Fukushima Convolutional neural networks • 1982: John Hopfield Hopfield networks (recurrent neural networks) https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  15. People of Deep Learning and Artificial Intelligence • History of science is a story of both people and ideas . • Many brilliant people contributed to the development of AI. Schmidhuber, Jürgen. "Deep learning in neural networks: An overview." Neural networks 61 (2015): 85-117 https://arxiv.org/pdf/1404.7828.pdf My (Lex) hope for the community: • More respect, open-mindedness, collaboration, credit sharing. • Less derision, jealousy, stubbornness, academic silos. https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  16. Limitations of Deep Learning • 2019 is the year it became cool to say that “deep learning” has limitations. • Books, articles, lectures, debates, videos were released that learning-based methods cannot do commonsense reasoning. Prediction from Rodney Brooks: “By 2020, the popular press starts having stories that the era of Deep Learning is over.” http://rodneybrooks.com/predictions-scorecard-2019-january-01/ https://deeplearning.mit.edu 2020 For the full list of references visit: [3, 4] http://bit.ly/deeplearn-sota-2020

  17. Deep Learning Research Community is Growing https://deeplearning.mit.edu 2020 For the full list of references visit: [2] http://bit.ly/deeplearn-sota-2020

  18. Deep Learning Growth, Celebrations, and Limitations Hopes for 2020 • Less Hype & Less Anti-Hype: Less tweets on how there is too much hype in AI and more solid research in AI. • Hybrid Research: Less contentious, counter- productive debates, more open-minded interdisciplinary collaboration. • Research topics: • Reasoning • Active learning and life-long learning • Multi-modal and multi-task learning • Open-domain conversation • Applications: medical, autonomous vehicles • Algorithmic ethics • Robotics https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  19. Outline • Deep Learning Growth, Celebrations, and Limitations • Deep Learning and Deep RL Frameworks • Natural Language Processing • Deep RL and Self-Play • Science of Deep Learning and Interesting Directions • Autonomous Vehicles and AI-Assisted Driving • Government, Politics, Policy • Courses, Tutorials, Books • General Hopes for 2020 https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

  20. Competition and Convergence of Deep Learning Libraries TensorFlow 2.0 and PyTorch 1.3 • Eager execution by default • TorchScript (imperative programming) (graph representation) • Keras integration + promotion • Quantization • Cleanup (API, etc.) • PyTorch Mobile (experimental) • TensorFlow.js • TPU support • TensorFlow Lite • TensorFlow Serving Python 2 support ended on Jan 1, 2020. >>> print “Goodbye World” https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020

Recommend

Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning

Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning

Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning www.deeplearningbook.org Ian Goodfellow 2016-09-27 Definition Regularization is any modification we make to a learning algorithm that is intended to reduce

179 views • 13 slides
Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Tofu: Parallelizing Deep Learning Systems with Automatic Tiling Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL system is based on dataflow GPU#0 w1 w2 data g1 g2 Forward propagation

405 views • 38 slides
Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Numerical Computation for Deep Learning Lecture slides for Chapter 4 of Deep Learning

Numerical Computation for Deep Learning Lecture slides for Chapter 4 of Deep Learning

Numerical Computation for Deep Learning Lecture slides for Chapter 4 of Deep Learning www.deeplearningbook.org Ian Goodfellow Last modified 2017-10-14 Thanks to Justin Gilmer and Jacob Buckman for helpful discussions Numerical concerns for

511 views • 39 slides
Structured Probabilistic Models for Deep Learning Lecture slides for Chapter 16 of Deep Learning

Structured Probabilistic Models for Deep Learning Lecture slides for Chapter 16 of Deep Learning

Structured Probabilistic Models for Deep Learning Lecture slides for Chapter 16 of Deep Learning www.deeplearningbook.org Ian Goodfellow 2016-10-04 Roadmap Challenges of Unstructured Modeling Using Graphs to Describe Model Structure

974 views • 34 slides
CS7015 (Deep Learning) : Lecture 1 (Partial/Brief) History of Deep Learning Mitesh M. Khapra

CS7015 (Deep Learning) : Lecture 1 (Partial/Brief) History of Deep Learning Mitesh M. Khapra

CS7015 (Deep Learning) : Lecture 1 (Partial/Brief) History of Deep Learning Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute of Technology Madras 1/49 Acknowledgements Most of this material is based on the

1.55k views • 98 slides
CS7015 (Deep Learning) : Lecture 1 (Partial/Brief) History of Deep Learning Mitesh M. Khapra

CS7015 (Deep Learning) : Lecture 1 (Partial/Brief) History of Deep Learning Mitesh M. Khapra

CS7015 (Deep Learning) : Lecture 1 (Partial/Brief) History of Deep Learning Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute of Technology Madras 1/81 Acknowledgements Most of this material is based on the

923 views • 81 slides
Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

541 views • 15 slides
Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org

Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org

Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org Ian Goodfellow Last updated 2016-10-04 Roadmap Example: Learning XOR Gradient-Based Learning Hidden Units Architecture Design

483 views • 30 slides
Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision : Image Classification,

1.53k views • 23 slides
CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe

CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe

DeepLoco by X. B. Peng, G. Berseth & M. van de Panne CMP784 DEEP LEARNING Lecture #12 Deep Reinforcement Learning Aykut Erdem // Hacettepe University // Spring 2018 Neural Face by Taehoon Kim Previously on CMP784 Generative

895 views • 71 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides
Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Lecture 6: CNNs and Deep Q Learning 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2019 1 With many slides for DQN from David Silver and Ruslan Salakhutdinov and some vision slides from Gianni Di Caro and images from Stanford CS231n,

1.17k views • 68 slides
Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Lecture 6: CNNs and Deep Q Learning 2 Emma Brunskill CS234 Reinforcement Learning. Winter 2018 2 With many slides for DQN from David Silver and Ruslan Salakhutdinov and some vision slides from Gianni Di Caro and images from Stanford CS231n,

850 views • 67 slides
ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image

308 views • 26 slides
Learning State of the Art 1 19.11.2019 What is Deep Learning? https://youtu.be/Kfe5hKNwrCU

Learning State of the Art 1 19.11.2019 What is Deep Learning? https://youtu.be/Kfe5hKNwrCU

Applications and Deep Learning State of the Art 1 19.11.2019 What is Deep Learning? https://youtu.be/Kfe5hKNwrCU Long pipeline of processing operations Designed by showing examples Example: TUT Age Estimation Image Recognition

556 views • 52 slides
Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice Scaling up DL 2 What is Deep Learning? 3 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY SECURITY & DEFENSE MEDIA &

1.45k views • 39 slides
Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490 HST.506 Guest Lecturer: Brandon Carter Prof. David Gifford Lecture 5 February 20, 2020 Deep Learning Model Interpretation http://mit6874.github.io 1

826 views • 53 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI Mission Helping people solve challenging problems using AI and deep learning. Developers, data scientists and engineers Self-driving cars,

1.44k views • 72 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin Clark Deep Learning for NLP 5 years ago No Seq2Seq No Attention No large-scale QA/reading comprehension datasets No TensorFlow or

1.41k views • 92 slides
An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning Microsoft Research Asia (MSRA) Kaiming He, Xiangyu Zhang, Shaoqing Ren, & Jian Sun. Deep Residual Learning for Image Recognition. arXiv

1.32k views • 54 slides
CMP784 DEEP LEARNING Lecture #12 Self-Supervised Learning Aykut Erdem // Hacettepe

CMP784 DEEP LEARNING Lecture #12 Self-Supervised Learning Aykut Erdem // Hacettepe

photo by unsplash user @tuvaloland CMP784 DEEP LEARNING Lecture #12 Self-Supervised Learning Aykut Erdem // Hacettepe University // Spring 2020 latent by Tom White Previously on CMP784 Motivation for Variational Autoencoders (VAEs)

2.02k views • 56 slides

More recommend