and inference for convolutional
play

and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 - PowerPoint PPT Presentation

Aug 12, 2023 •182 likes •648 views

Band-limited Training and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast Training of Convolutional Networks through FFTs Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x

  1. Band-limited Training and Inference for Convolutional Neural Networks 1

  2. 2

  3. FFT IFFT 3

  4. 4

  5. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  6. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  7. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) Out: o offt xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  8. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  9. Mathieu et al.: “Fast Training of Convolutional Networks through FFTs” Fast Convolutional Nets With fbfft: A GPU Performance Evaluation cuDNN cu DNN: Subs ubstantial tantial memor ory y wor orkspace space neede ded d for or intermed ermediate iate resul ults. ts. Data: x xfft FFT(x) offt Out: o xfft  yfft IFFT(offt) Filter: y FFT(y) yfft

  10. Band-limiting = masking out high frequencies xfft Data: x xCfft Band-limited FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) (yfft) yCfft

  11. xfft Data: x xCfft Band-limited Less memory used FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) (yfft) yCfft

  12. xfft Data: x xCfft Band-limited Less memory used FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) Faster computation (yfft) yCfft

  13. Preserve enough of the spectrum to retain high accuracy of models. xfft Data: x xCfft Band-limited Less memory used FFT(x) (xfft) offt Out: o xCfft  yCfft IFFT(offt) Filter: y yfft Band-limited FFT(y) Faster computation (yfft) yCfft

  14. 14

  15. 2. Conjugate symmetry 1-j 1+j 15

  16. 2. Conjugate symmetry 1+j 16

  17. 2. Conjugate symmetry DC 1+j 3. Real values 17

  18. 2. Conjugate symmetry DC 1+j 3. Real values 4. No constraints 18

  19. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 19

  20. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 20

  21. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 6. 2 nd compression 21

  22. 2. Conjugate symmetry DC 3. Real values 4. No constraints 5. 1 st compression 6. 2 nd compression 22

  23. Test Accuracy (%) 95 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Compression rate (%) 23

  24. 93.5% Test Accuracy (%) 95 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Compression rate (%) 24

  25. 93.5% Test Accuracy (%) 95 92% 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Compression rate (%) 25

  26. 93.5% Test Accuracy (%) 95 92% 90 ResNet-18 on CIFAR-10 85 0 20 40 60 80 Test Accuracy (%) 80 75.3% 71.2% 70 DenseNet-121 on CIFAR-100 60 0 20 40 60 80 Compression rate (%) 26

  27. ▪ ▪ ▪ ▪ ▪ ▪ ▪ 27

  30. ▪ ▪ ▪ ▪ ▪ ▪ ▪ ▪ 30

  31. Cross-correlate input data and filter: x ∗ c y F x ω = F x n F y ω = F y n x ∗ c y = F −1 (F x ω ʘ F y ω ) Spectrum of convolution: S ω = F x ω ʘ F y ω 𝐍 𝐝 𝛛 = ቊ 𝟐, 𝛛 ≤ 𝐝 𝐏, 𝛛 > 𝐝 x ∗ c y = F −1 [ F x ω ʘ M c ω ) ʘ (F y ω ʘ M c ω ] x ∗ c y = F −1 S ω ʘ M c ω 𝑂−1 |𝑦 𝑜 | 2 = σ 𝜕=0 Energy (Parseval’s theorem): σ 𝑜=0 2𝜌 𝑦 𝜕 | 2 |𝐺 31

  32. 32

  33. DenseNet-121 on CIFAR-100 80 70 Test accuracy (%) 60 50 40 30 20 10 C=50 C=75 0 0 20 40 60 80 Inference Compression Rate (%) 33

  34. DenseNet-121 on CIFAR-100 80 70 Test accuracy (%) 60 50 40 30 20 10 C=0 C=50 C=75 C=85 0 0 20 40 60 80 Inference Compression Rate (%) 34

  35. ResNet-18 on CIFAR-10 performance Normalized 100 (%) 50 GPU memory allocated 0 0 20 40 60 80 performance 100 Normalized 50 (%) Epoch time 0 0 20 40 60 80 Compression rate (%) 35

  36. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) 36

  37. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 C=85 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) 37

  38. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 C=85 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) Smooth degradation of accuracy during inference 38

  39. 100 ResNet-18 on CIFAR-10 80 Test accuracy (%) 60 40 Train Compression Rate (%): 20 C=0 C=30 C=50 C=85 0 0 10 20 30 40 50 60 70 80 Inference Compression Rate (%) Apply the same compression rate to training and inference 39

  40. Test Accuracy (%) 95 100 90 50 GPU memory allocated 85 0 0 50 0 20 40 60 80 100 Test Accuracy (%) 80 50 70 Epoch time 0 60 0 20 40 60 80 0 50 Compression rate (%) Compression rate (%) 40

  41. 41

  42. 42

  43. 43

  44. 44

  45. 45

  46. “Speaking of longer term, it would be nice if the community migrated to a fully open sourced implementation for all of this [convolution operations, etc.]. This stuff is just too important to the progress of the field for it to be locked away in proprietary implementations . The more people working together on this the better for everyone. There's plenty of room to compete on the hardware implementation side.” Scott Gray https://github.com/soumith/convnet-benchmarks/issues/93 46

Recommend

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN 17 August 2020 ICPP 2020 Sai Qian Zhang , Jieyu Lin, Qi Zhang Executing DNN Inference Tasks for End Users Option 2: cloud only Option 1: edge only

579 views • 17 slides
Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon

Inference Networks, Graph Convolutional Networks Greg Mori School of Computing Science Simon Fraser University Outline Scene Image annotation with label hierarchies outdoor outdoor indoor man-made natural house sports man-made

682 views • 47 slides
Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder

Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder

Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder Motivation: image to autoencoder ? Convolutional autoencoder extends the basic structure of the simple autoencoder by changing the fully

349 views • 15 slides
Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter (this lecture) and Ariel Procaccia Carnegie Mellon University Spring 2017 1 Outline Convolutional neural networks Applications of convolutional

1.32k views • 32 slides
Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Supun

Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Supun

Incremental and Approximate Inference for Faster Occlusion-based Deep CNN Explanations Supun Nakandala , Arun Kumar, and Yannis Papakonstantinou University of California, San Diego Introduction Deep Convolutional Neural Networks (CNNs) are

456 views • 30 slides
Exact Inference Inference Basic task for inference: Compute

Exact Inference Inference Basic task for inference: Compute

Exact Inference Inference Basic task for inference: Compute a posterior distribu8on for some query variables given some observed evidence Sum out

699 views • 30 slides
Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Lecture 8: Convolutional Neural Nets Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier) https://courses.grainger.illinois.edu/cs447/ 1 Convolutional Neural Nets (ConvNets, CNNs) [4 parameters, applied 3

615 views • 24 slides
Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference by enumeration Approximate inference by stochastic simulation Chapter 14.45 2 Inference tasks Simple queries: compute posterior marginal P (

450 views • 28 slides
Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides
Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1

Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1

Inference in First-Order Logic C H A P T E R 9 H A S S A N K H O S R A V I S P R I N G 2 0 1 1 Outline Reducing first-order inference to propositional inference Unification Generalized Modus Ponens Forward chaining Backward

856 views • 58 slides
Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural networks Problem Description Convolutional Neural Networks Convolutional Layer Max Pooling Transforming Classification networks to segmentation

157 views • 12 slides
Lecture 6: Convolutional NN Princeton University COS 495 Instructor: Yingyu Liang Review:

Lecture 6: Convolutional NN Princeton University COS 495 Instructor: Yingyu Liang Review:

Deep Learning Basics Lecture 6: Convolutional NN Princeton University COS 495 Instructor: Yingyu Liang Review: convolutional layers Convolution: two dimensional case Input Kernel/filter a b c d w x e f g h y z i j k l wa + bx

847 views • 36 slides
Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference

Inference in Bayesian networks Chapter 14.45 Chapter 14.45 1 Outline Exact inference by enumeration Exact inference by variable elimination Approximate inference by stochastic simulation Approximate inference by Markov

530 views • 38 slides
Type Inference 75 Definition Type Inference Type inference = Java compiler's ability

Type Inference 75 Definition Type Inference Type inference = Java compiler's ability

Type Inference 75 Definition Type Inference Type inference = Java compiler's ability To look at each method invocation + corresponding declaration In order to determine the type argument (or arguments) that make the

457 views • 11 slides
Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii s state flower ? ROOT Mingbo Ma Liang Huang Bing Xiang Bowen Zhou CUNY IBM T. J. Watson ACL 2015 Beijing Convolutional Neural Network

742 views • 50 slides
Causal Inference and Response Surface Modeling Inference and

Causal Inference and Response Surface Modeling Inference and

Causal Inference and Response Surface Modeling Inference and Representa6on DS-GA-1005 Fall 2015 Guest lecturer: Uri Shalit What is Causal Inference?

503 views • 48 slides
Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Hu, NIPS14 Irsoy, NIPS14 Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences. NIPS14 Convolutional Sentence Model Convolutional Matching Models Experiments Deep Recursive Neural

779 views • 35 slides
ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result

ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result

DIRECT CONVOLUTION FOR DEEP NEURAL NETWORK CLASSIFICATION ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result Convolutional Fully Connected Input layer layer Convolutional Layer An example: A E

621 views • 18 slides
Convolutional Neural Networks in R Mengli Xiao Division of Biostatistics University of Minnesota

Convolutional Neural Networks in R Mengli Xiao Division of Biostatistics University of Minnesota

Convolutional Neural Networks in R Mengli Xiao Division of Biostatistics University of Minnesota February 21, 2018 1/9 Outline Preparation Create Python environment Install R packages: keras, tensorflow(optional) Application Convolutional

121 views • 10 slides

More recommend