tensor layers for compression of deep learning networks
play

TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka - PowerPoint PPT Presentation

Jul 30, 2023 •287 likes •937 views

TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka Senior Research Scientist, NVIDIA GTC 2018 Tensors Computations and the GPU AGENDA Tensor Networks and Decompositions Tensor Layers in Deep Learning 2 TENSOR COMPUTATIONS

  1. TENSOR LAYERS FOR COMPRESSION OF DEEP LEARNING NETWORKS Cris Cecka Senior Research Scientist, NVIDIA GTC 2018

  2. Tensors Computations and the GPU AGENDA Tensor Networks and Decompositions Tensor Layers in Deep Learning 2

  3. TENSOR COMPUTATIONS AND THE GPU Modern data is inherently multi-dimensional. 3

  4. TENSOR CONTRACTIONS • Core primitive of multilinear algebra • BLAS level 3 — unbounded compute intensity. X

  5. TENSOR LIBRARIES Explicit permutation dominates. Y . Shi, U. N. Niranjan, A. Anandkumar and C. Cecka , "Tensor Contractions with Extended BLAS Kernels on CPU and GPU," 2016 IEEE X 23rd International Conference on High Performance Computing (HiPC), Hyderabad, 2016, pp. 193-202.

  6. CONTRACTIONS : Single GEMM (Provided compact layout) X

  7. BATCHED MATRIX-MATRIX MULTIPLY cublas<T>gemmStridedBatched X

  8. CONTRACTIONS : Single SB-GEMM (Any layout) X

  9. APPLICATION: FFT : Tensor/FFT — vendor optimized cublas<T>gemmStridedBatched : Custom kernel : FMM Communication StridedBatchedGEMM: 75%+ of the runtime 1.5x over cuFFT on 2xV100 2.6x over cuFFT on 8xV100 Cris Cecka. “Low communication FMM-accelerated FFT on GPUs." In Proceedings of the International Conference for High X Performance Computing, Networking, Storage and Analysis (SC '17). ACM, New York, NY, USA.

  10. WHY TENSORS? 10

  11. DENSITY AND SPARSITY H. Anzt, S. Tomov, J. Dongarra , “Energy Efficiency and Performance Frontiers for Sparse Computations on GPU 11 Supercomputers," PMAM 2015.

  12. DENSITY AND SPARSITY In general, need < 5% sparsity for a computational win. Solutions Block-sparse — Locally dense and globally sparse 12

  13. TENSOR DECOMPOSITIONS Decompositions for data sparse representations. 13

Recommend

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides
Introduction to Artificial Intelligence Deep Learning - Tensor Flow Janyl Jumadinova December 2,

Introduction to Artificial Intelligence Deep Learning - Tensor Flow Janyl Jumadinova December 2,

Introduction to Artificial Intelligence Deep Learning - Tensor Flow Janyl Jumadinova December 2, 2016 Credit: Google Workshop Neural Networks 2/24 Neural Networks 3/24 Neural Networks A fully connected NN layer 4/24 Implementation as

1.2k views • 34 slides
CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward autodifferentiation (Beginning of) Convolutional neural networks Zsolt Kira Georgia Tech Projects We will release a set of project ideas for those

1.04k views • 50 slides
CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward

CS 4803 / 7643: Deep Learning Topics: Specifying Layers Forward &amp; Backward autodifferentiation (Beginning of) Convolutional neural networks Zsolt Kira Georgia Tech Administrivia PS0 released mean of 20.7

1.39k views • 92 slides
Collaborative Channel Pruning for Deep Networks 11th June 2019 Background Model compression

Collaborative Channel Pruning for Deep Networks 11th June 2019 Background Model compression

Collaborative Channel Pruning for Deep Networks 11th June 2019 Background Model compression method Compact network design; Source:https://orbograph.com/ deep-learning-how-will-it-change-healthcare/ Network quantization; Channel or

396 views • 12 slides
Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem

Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem

Deep Learning for Predictive Maintenance Pawel Morkisz GTC 2017 Agenda Problem Introduction and notion of deep neural networks o Convolutional layers o Residual networks (ResNet) One dimensional convolutional networks in failure

276 views • 23 slides
DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology

DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology

DEEP LEARNING FFR135, Artificial Neural Networks Olof Mogren Chalmers University of Technology October 2016 DEEP LEARNING Artificial neural networks Many layers of abstractions Outperforms traditional methods in: Image

824 views • 61 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Tensor Methods for Signal Processing and Machine Learning Qibin Zhao Tensor Learning Unit RIKEN

Tensor Methods for Signal Processing and Machine Learning Qibin Zhao Tensor Learning Unit RIKEN

Tensor Methods for Signal Processing and Machine Learning Qibin Zhao Tensor Learning Unit RIKEN AIP 2018-6-9 @ Waseda University 1 Monographs Tensor networks for dimensionality reduction and large optimization Andrzej Cichocki, Namgil

1.34k views • 77 slides
Applications of Tensor Networks: Machine Learning &amp; Quantum Computing E.M. Stoudenmire Aug

Applications of Tensor Networks: Machine Learning &amp; Quantum Computing E.M. Stoudenmire Aug

Applications of Tensor Networks: Machine Learning &amp; Quantum Computing E.M. Stoudenmire Aug 2018 - ICTP Trieste Outline: Yesterday : intro to tensor networks, mainly matrix product states (MPS) computations with MPS intro to

1.78k views • 142 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Using Tensor Swapping and NVLink to Overcome GPU Memory Limits with TensorFlow Sam Matzek Deep

Using Tensor Swapping and NVLink to Overcome GPU Memory Limits with TensorFlow Sam Matzek Deep

Using Tensor Swapping and NVLink to Overcome GPU Memory Limits with TensorFlow Sam Matzek Deep learning is memory constrained GPUs have limited memory Neural networks are growing deeper and wider Amount and size of data to process is

397 views • 23 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Apr 7, 2016 1 Intro about me and my advisor Fourth year PhD with Prof. Bill Dally at Stanford.

2.06k views • 70 slides
Efficient Neural Network Compression Namhoon Lee University of Oxford 3 May 2019 A Challenge in

Efficient Neural Network Compression Namhoon Lee University of Oxford 3 May 2019 A Challenge in

Efficient Neural Network Compression Namhoon Lee University of Oxford 3 May 2019 A Challenge in Deep Learning: Overparameterization Large neural networks require: memory &amp; computations power consumption A Challenge in Deep Learning:

722 views • 49 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Jan 6, 2015 A few words about us Fourth year PhD with Prof. Bill Dally at Stanford.

966 views • 54 slides
Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels

563 views • 43 slides
Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels

1.45k views • 34 slides
Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org

Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org

Deep Feedforward Networks Lecture slides for Chapter 6 of Deep Learning www.deeplearningbook.org Ian Goodfellow Last updated 2016-10-04 Roadmap Example: Learning XOR Gradient-Based Learning Hidden Units Architecture Design

483 views • 30 slides
Compression Strategies &amp; Alternate Summarization Systems and Applications Ling 573 May 23,

Compression Strategies &amp; Alternate Summarization Systems and Applications Ling 573 May 23,

Compression Strategies &amp; Alternate Summarization Systems and Applications Ling 573 May 23, 3017 Roadmap Content Realization: Compression Deep, Heuristic Approaches Compression Integration Compression Learning

317 views • 31 slides
Deep learning-based parameter mapping for joint relaxation and diffusion tensor MR Fingerprinting

Deep learning-based parameter mapping for joint relaxation and diffusion tensor MR Fingerprinting

Deep learning-based parameter mapping for joint relaxation and diffusion tensor MR Fingerprinting Carolin M. Pirkl *1,2 , Pedro A. Gmez *1 , Ilona Lipp 3,4,5 , Guido Buonincontri 6,7 , Miguel Molina-Romero 1 , Anjany Sekuboyina 1,8 , Diana

361 views • 7 slides
E9 205 Machine Learning for Signal Processing Understanding Deep Networks 08-11-2019 Instructor

E9 205 Machine Learning for Signal Processing Understanding Deep Networks 08-11-2019 Instructor

E9 205 Machine Learning for Signal Processing Understanding Deep Networks 08-11-2019 Instructor - Sriram Ganapathy (sriramg@iisc.ac.in.in) Understanding Deep Networks Understanding Deep Networks Understanding Deep Networks SVHN dataset

366 views • 23 slides
Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine

Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine

Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Deep Convolutional Neural Nets 1 / 25 Outline 1 Why Neural Networks? 2 Circuits 3 Neurons, Layers, and Networks 4 Correlation and Convolution 5

287 views • 25 slides
CS 4803 / 7643: Deep Learning Topics: Backpropagation Vector/Matrix/Tensor math

CS 4803 / 7643: Deep Learning Topics: Backpropagation Vector/Matrix/Tensor math

CS 4803 / 7643: Deep Learning Topics: Backpropagation Vector/Matrix/Tensor math Deriving vectorized gradients for ReLU Zsolt Kira Georgia Tech Administrivia PS1/HW1 out Start thinking about project topics/teams (C)

482 views • 21 slides
Vathys: Petascale Deep Learning on a (Single) Chip Tapa Ghosh Vathys What is deep learning? In

Vathys: Petascale Deep Learning on a (Single) Chip Tapa Ghosh Vathys What is deep learning? In

Vathys: Petascale Deep Learning on a (Single) Chip Tapa Ghosh Vathys What is deep learning? In a sentence: Layers of parametric, differentiable operations whose parameters are changed using gradient information. Below is a Convolutional

473 views • 28 slides

More recommend