accelerate deep learning
play

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | - PowerPoint PPT Presentation

Aug 26, 2022 •47 likes •308 views

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image

  1. ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC’16

  2. DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image Classification Cancer Cell Detection Face Detection Pedestrian Detection Video Captioning Speech Recognition Diabetic Grading Video Surveillance Lane Tracking Video Search Language Translation Drug Discovery Satellite Imagery Recognize Traffic Sign Real Time Translation Language Processing Sentiment Analysis Recommendation 2

  3. DEEP LEARNING REVOLUTIONIZING COMPUTING Solves Problems Previously Unsolvable 3

  4. A NEW COMPUTING MODEL Traditional Computer Vision Deep Learning Domain experts design feature detectors DNN learn features from large data Quality = patchwork of algorithms Quality = data & training method Need CV experts and time Needs lots of data and compute 4

  5. DEEP LEARNING The Next Innovation Network? MACHINE INTELLIGENCE DEEP LEARNING SMARTPHONES INTERNET PERSONAL COMPUTERS INTEGRATED CIRCUITS MICROWAVES TEST EQUIPMENT VACUUM TUBES 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 5 (Source: Deep Learning Gold Rush of 2015, Tomasz Malisiewicz, November 07, 2015 (adapted from http://steveblank.com/secret-history/)

  6. THE AI RACE IS ON IMAGENET Accuracy Rate 100% Traditional CV Deep Learning 90% 80% 70% Baidu Deep Speech 2 IBM Watson Achieves Breakthrough Facebook Beats Humans in Natural Language Processing Launches Big Sur 60% 50% 40% 30% 20% 10% Google Toyota Invests $1B Microsoft & U. Science & Tech, China Launches TensorFlow Beat Humans on IQ in AI Labs 0% 2009 2010 2011 2012 2013 2014 2015 2016 6

  7. $500B DEEP LEARNING OPPORTUNITY Deep Learning Total Revenue by Segment Deep Learning Software Revenue by Industry World Markets: 2015-2024 World Markets: 2015-2024 $120,000 $100,000 Other $80,000 Ad Service ($ Millions) Retail Technology $60,000 Manufacturing $40,000 $20,000 Oil and Gas $- 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 Investment Media $5,000 DL-driven $4,000 ($ Millions) GPU Chip Revenue $2,000 $1,000 “ The current cutting edge of deep learning processing $- platforms seems to be massively parallel GPU systems. ” 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 — Tractica 7 SOURCE: “Deep Learning for Enterprise Applications, 4Q 2015, Tractica ” NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

  8. THE ENGINE OF MODERN AI EDUCATION BIG SUR TENSORFLOW WATSON CNTK TORCH CAFFE THEANO MATCONVNET START-UPS MOCHA.JL PURINE CHAINER DL4J KERAS OPENDEEP MINERVA MXNET* SCHULTS VITRUVIAN LABORATORIES NVIDIA DEEP LEARNING PLATFORM 8 * U. Washington, CMU, Stanford, TuSimple, NYU, Microsoft, U. Alberta, MIT, NYU Shanghai

  9. NVIDIA DEEP LEARNING PLATFORM DL FRAMEWORK (CAFFE, CNTK,TENSORFLOW, THEANO, TORCH …) DEEP LEARNING SDK TITAN X - DEVELOPERS TESLA - DEPLOYMENT AUTOMOTIVE - DRIVEPX EMBEDDED - JETSON 9 NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.

  10. NVIDIA Deep Learning SDK High Performance GPU-Acceleration for Deep Learning Recommendation Sentiment Analysis Image Classification Voice Recognition Language Translation Object Detection Engines SPEECH BEHAVIOR VISION Mocha.jl DEEP LEARNING FRAMEWORKS cuFFT cuBLAS cuSPARSE cuDNN NCCL DEEP LEARNING MULTI-GPU MATH LIBRARIES 10

  11. NVIDIA Deep Learning SDK Powerful developer tools and libraries for designing and deploying GPU-accelerated deep learning applications High Performance Deep Learning for NVIDIA GPUs Industry Vetted Deep Learning Algorithms Easily integrated into deep learning applications developer.nvidia.com/ deep-learning 11

  12. Pascal NVIDIA cuDNN 12.0x (cuDNN v5) 10.0x 8.0x M40 (cuDNN v3) 6.0x Building blocks for accelerating deep 4.0x K40 neural networks on GPUs (cuDNN v1) 2.0x High performance deep neural 0.0x network training 2014 2015 2016 AlexNet training throughput based on 20 iterations, CPU: 1x E5-2680v3 12 Core 2.5GHz. Accelerates Deep Learning: Caffe, CNTK, Tensorflow, Theano, Torch “ NVIDIA has improved the speed of cuDNN with each release while extending the Performance continues to improve interface to more operations and devices over time at the same time. ” — Evan Shelhamer, Lead Caffe Developer, UC Berkeley developer.nvidia.com/ cudnn 12

  13. cuBLAS Accelerated Linear Algebra for Deep Learning GPU-accelerated Basic Linear Algebra Subroutines that delivers 6x to 17x faster performance than the latest MKL BLAS Accelerated Level 3 BLAS: SGEMM, SYMM, TRSM, SYRK Up to 7 TFlops Single Precision on a single M40 Multi-GPU BLAS support available in cuBLAS-XT developer.nvidia.com/cublas 13

  14. NCCL Accelerating Multi-GPU Communications A topology-aware library of accelerated collectives to improve the scalability of multi-GPU applications Patterned after MPI’s collectives: includes all-reduce, all-gather, reduce-scatter, reduce, broadcast Optimized intra-node communication Supports multi-threaded and multi- process applications github.com/NVIDIA/nccl 14

  15. NVIDIA DIGITS Making Deep Learning Accessible An interactive development environment for training deep neural networks Prepare data quickly and easily for training Visualize network behavior Maximize training speed developer.nvidia.com/digits 15

  16. What’s new in DIGITS 3? Improves Deep Learning Training Productivity Train neural network models with Screenshot of new DIGITS Torch support (preview) Save time by quickly iterating to identify the best model Manage multiple jobs easily to optimize use of system resources Active open source project with New Results Browser! valuable community contributions developer.nvidia.com/digits 16

  17. Preview DIGITS Future Object Detection Workflow Object Detection Workflows for Automotive and Defense Targeted at Autonomous Vehicles, Remote Sensing Come see a live demo in the GTC Exhibit Hall! developer.nvidia.com/digits 17

  18. Deep Learning at GTC 18

  19. Deep Learning at GTC Deep Learning at NVIDIA, Monday 4/4 11:00am: Accelerate Deep Learning with NVIDIA's Deep Learning Platform 12:00pm: Hangout -- The DIGITS Roadmap 1:00pm: From Workstation to Embedded: Accelerated Deep Learning on NVIDIA Jetson TX1 3:00pm: A Tutorial on More Ways to Use DIGITS 4:00pm: Hangout – cuDNN -- Features, Roadmap and Q&A 19

  20. Deep Learning at GTC Frameworks Track, Wednesday 4/6 9:00am: Caffe: an Open Framework for Deep Learning 10:00am: TensorFlow: Scaling Up Machine Learning 2:00pm: Torch: A Flexible Platform for Deep Learning Research 3:00pm: Chainer: A Powerful, Flexible, and Intuitive Deep Learning Framework 4:00pm: Theano at a Glance: A Framework for Machine Learning 4:30pm: Deep Learning in Microsoft with CNTK Monday, 4/4 at 3:00pm: MXNet: Flexible Deep Learning Framework from Distributed GPU Clusters to Embedded Systems 20

  21. Deep Learning at GTC Frameworks Hands-on Labs Wednesday 4/6 Thursday 4/7 1:00pm: Introduction to CNTK 9:30am: Chainer Hands-on: Introduction To Train Deep Learning Model in Python 2:00pm: Machine Learning Using TensorFlow 9:30am: Deep Learning With the Theano 2:00pm: BIDMach Machine Learning Toolkit Python Library 3:30pm: Applied Deep Learning for Vision 1:00pm: IBM Watson Developers Lab and Natural Language with Torch7 Monday 4/4 3:30pm: Caffe Hands-on Lab 1:00pm: Train and Deploy Deep Learning for Vision, Natural Language and Speech Using MXNet 21

  22. Deep Learning at GTC Over 50 sessions on Deep Learning, highlights include -- Tuesday, 4/5 1:00pm: Distributed Deep Learning at Scale, Soumith Chintala , Research Engineer, Facebook AI Research 2:00pm: Generative Adversarial Networks, Ian Goodfellow , Senior Research Scientist, Google 3:00pm: Video Classification of Live Streams on Twitter's Periscope, Nicolas Koumchatzky , Engineer, Twitter 4:00pm: Training and Deploying Deep Neural Networks for Speech Recognition, Bryan Catanzaro , Senior Researcher, Baidu Research Wednesday, 4/6, 9:30am: Deep Reinforcement Learning, Pieter Abbeel , Professor, UC Berkeley 22

  23. Deep Learning at GTC Hangouts Monday 4/4 Wednesday 4/6 12:00pm: The DIGITS Roadmap 10:00am: Deep Learning in Image and Video 4:00pm: cuDNN--Features, Roadmap and Q&A 1:00pm: Deep Learning Exploits Petabytes of DigitalGlobe GIS Tuesday 4/5 1:00pm: Reinforcement Learning 12:00pm: Dreaming Big: Scaling Up Deep Dream Thursday 4/7 to Operate on Multi-Hundred Megapixel Images 1:00pm: Large Vocabulary Speech Recognition with GPUs 1:00pm: NVIDIA Deep Learning Software 23

  24. Deep Learning at GTC NLP/NLU S6515 - Listen, Attend and Spell, William Chan , PhD Candidate, Carnegie Mellon University S6745 - VQA: Visual Question Answering, Aishwarya Agrawal , PhD Student, Virginia Tech S6781 - Deep Neural Networks for Conversational Language Understanding, Kaheer Suleman , CTO, Maluuba Inc. S6321 - How Deep Learning Works for Automated Customer Service, Chenghua (Kevin) Li , Chief Scientist of DNN Lab, JD.COM H6127 - Hangout: Large Vocabulary Speech Recognition with GPUs, Yifang Xu , Senior Deep Learning Software Engineer, NVIDIA 24

Recommend

Accelerate Deep Learning Training at Scale on GPUs Maggie Zhang ( ) maggiez@nvidia.com

Accelerate Deep Learning Training at Scale on GPUs Maggie Zhang ( ) maggiez@nvidia.com

Accelerate Deep Learning Training at Scale on GPUs Maggie Zhang ( ) maggiez@nvidia.com Introduction Why do we need to scale training AGENDA How to achieve scaling DL Training: from single GPU to multi-node 1.33 Minutes on

894 views • 47 slides
Accelerate Innovation in the Enterprise Solutions and Reference with Distributed ML / DL

Accelerate Innovation in the Enterprise Solutions and Reference with Distributed ML / DL

Accelerate Innovation in the Enterprise Solutions and Reference with Distributed ML / DL architecture Nanda Vijaydev BlueData (recently acquired by HPE) The AI Conference, New York Agenda AI, Machine Learning (ML), and Deep Learning

803 views • 35 slides
Can we accelerate the adoption of AI in pharma by learning from experiences in other

Can we accelerate the adoption of AI in pharma by learning from experiences in other

Can we accelerate the adoption of AI in pharma by learning from experiences in other industries? Dr Sam Genway 9 th October 2018 Tessella, Altran's World Class Center for Analytics We use data science to accelerate evidence-based decision

513 views • 26 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides
Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice Scaling up DL 2 What is Deep Learning? 3 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY SECURITY & DEFENSE MEDIA &

1.45k views • 39 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI Mission Helping people solve challenging problems using AI and deep learning. Developers, data scientists and engineers Self-driving cars,

1.44k views • 72 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin Clark Deep Learning for NLP 5 years ago No Seq2Seq No Attention No large-scale QA/reading comprehension datasets No TensorFlow or

1.41k views • 92 slides
An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning Microsoft Research Asia (MSRA) Kaiming He, Xiangyu Zhang, Shaoqing Ren, & Jian Sun. Deep Residual Learning for Image Recognition. arXiv

1.32k views • 54 slides
Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

541 views • 15 slides
Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision : Image Classification,

1.53k views • 23 slides
Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Tofu: Parallelizing Deep Learning Systems with Automatic Tiling Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL system is based on dataflow GPU#0 w1 w2 data g1 g2 Forward propagation

405 views • 38 slides
Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu

Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu

Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu 2020 For the full list of references visit: http://bit.ly/deeplearn-sota-2020 Outline Deep Learning Growth, Celebrations, and Limitations

1.1k views • 87 slides
12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Wanli Ouyang, Zsolt Kira, Lawrence Neal, Raymond Yeh, Junting Lou and Teck-Yian Lim Unsupervised Learning in General Unsupervised learning is

1.48k views • 115 slides
Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning

Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning

Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning www.deeplearningbook.org Ian Goodfellow 2016-09-27 Definition Regularization is any modification we make to a learning algorithm that is intended to reduce

179 views • 13 slides
Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples RNNs Some slides borrowed from Fei-Fei Li & Justin Johnson &

1.1k views • 55 slides
Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical banquet, just so many strange dishes And yet, we seem to do fine We can understand and generate language effortlessly. Almost 1 Wouldnt it be

1.62k views • 52 slides
Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning for robots? slide from V. Vanhoucke End-to-end Deep Learning for robots? 2010 : Speech Recognition Audio Acoustic Model Phonetic Model

1.05k views • 65 slides
Bayesian Deep Learning Mohd Adnan Problems With Deep Learning What does a model not know?

Bayesian Deep Learning Mohd Adnan Problems With Deep Learning What does a model not know?

Bayesian Deep Learning Mohd Adnan Problems With Deep Learning What does a model not know? Uninterpretable black-boxes Easily fooled (AI safety) Lacks solid mathematical foundation Crucially relies on big dat Why

297 views • 15 slides
Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief Introduction to Deep Learning Artificial Intelligence Landscape Deep Learning 4 Example Deep Learning Algorithms Multilayer Recurrent

651 views • 44 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
DSSTNE: Deep Learning At Scale For Large Sparse Datasets

DSSTNE: Deep Learning At Scale For Large Sparse Datasets

DSSTNE: Deep Learning At Scale For Large Sparse Datasets https://github.com/amznlabs/amazon-dsstne Scott Le Grand Senior Scientist Teza Technologies varelse2005@gmail.com Outline What's Deep Learning? Why GPUs? Deep Learning for

1.55k views • 76 slides

More recommend