Tuning the Performance of Convolutional Neural Network for Image - PowerPoint PPT Presentation

Apr 15, 2023 •32 likes •182 views

Tuning the Performance of Convolutional Neural Network for Image Classification on GPU Agenda Adoptions of Image classification or image recognition at Alibaba Easy ways to improve performance of Caffe Further performance

Tuning the Performance of Convolutional Neural Network for Image Classification on GPU
Agenda • Adoptions of Image classification or image recognition at Alibaba • Easy ways to improve performance of Caffe • Further performance optimization of convolution layer • Ongoing works 2 Confidential & Proprietary
Image classification at Alibaba • Product Display Classification Model-Upper/Item-Bottom/Multi-Object • Fashion Style Classification • Buy-by-photo mobile Sweet / Street / Office app, search for visually similar products by images • Leverage Caffe framework Confidential & Proprietary
Profiling Caffe • Most expensive part Caffe spends more than 70% of time on Convolution layers ! 4 Confidential & Proprietary
Convolution layer • How does the convolution layer work in Caffe Image to Column SGemm Confidential & Proprietary
The gap • Is it really fast? Blue: Caffe(imagenet model) Red: Sgemm routine of Cublas Green: Peak of K20 ImageNet model, refer to the ILSVRC12 challenge Confidential & Proprietary
How does Cublas Sgemm perform 7 Confidential & Proprietary
Easiest way to narrow the gap • To Overcome the low efficient of SGEMM at small scale Processing one batch Processing one batch Image to Column Image to Column Single image Batch-coalesced images every every loop loop Gemm Gemm 8 Confidential & Proprietary
Performance of Fast mode • Titan black, mini-batch size is 256 9 Confidential & Proprietary
Moving forward • How is cublas sgemm implemented Confidential & Proprietary
Use high performance sgemm routines • Example: ImageNet convolution layer “conv5”: M = 96, N=3025, K=363 • cuBLAS use: sgemm_64x16x64x16x16, slow! • We use: sgemm_128x8x128x16x16 to get the same result, 1.54x faster on K20 ！ Confidential & Proprietary
Implement our own conv layer • Auto-gen gpu kernels for convolution layers • Kernels are implemented in PTX assembly Conv2 from Alex’s Net, Height = 16; Width = 16; Channel = 5; Stride = 1; Ksize = 5; Pad = 2; Neuron = 32 Confidential & Proprietary
Is PTXAS good enough? snippet of instru • Problem /* 0x 09 00 10 1c1c 10 1c1c */ ctions from sge FFMA R23, R83, R84, R23; mm kernel, sm_ – Register usage FFMA R33, R88, R84, R33; 35 FFMA R36, R88, R85, R36; – Manipulate “control code” on Kepler NOP; FFMA R45, R89, R84, R45; • Our own assembler for Kepler FFMA R32, R89, R85, R32; NOP; – Probe native ins /* 0x 08 80 10 14 10 14 10 14 */ FFMA R5, R80, R86, R5; – Probe control ins FFMA R2, R81, R86, R2; FFMA R14, R81, R87, R14; – Ongoing FFMA R7, R80, R92, R7; FFMA R3, R80, R87, R3; • Some users need a native assembler, please! FFMA R8, R81, R92, R8; NOP; Confidential & Proprietary
Other ongoing works • Convert model from Single-precision floating points to – half-precision (maxwell) – flexible fixed-points (FPGA) Confidential & Proprietary
Thank You • Download the mobile app at taobao.com and try out Buy-by- Photo 15 Confidential & Proprietary

Recommend

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement Learning Yan li, Kenneth Chang, Oceane Bel, Ethan L. Miller, Darrel D. E. Long Performance Tuning Tuning systems parameters for high

343 views • 15 slides

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides

ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result

DIRECT CONVOLUTION FOR DEEP NEURAL NETWORK CLASSIFICATION ON TEGRA X1 ALAN WANG, NVIDIA Convolutional Neural Network optimization target Result Convolutional Fully Connected Input layer layer Convolutional Layer An example: A E

621 views • 18 slides

Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture

Neural Network Part 3: Convolutional Neural Networks CS 760@UW-Madison Goals for the lecture you should understand the following concepts convolutional neural networks (CNN) convolution and its advantage pooling and its advantage 2

750 views • 50 slides

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides

Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences.

Outline Hu, NIPS14 Irsoy, NIPS14 Outline Convolutional Neural Network Architectures for Matching Natural Language Sentences. NIPS14 Convolutional Sentence Model Convolutional Matching Models Experiments Deep Recursive Neural

779 views • 35 slides

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last

CS4501: Introduction to Computer Vision Deeper Convolutional Neural Network Architectures Last Class Neural Networks multilayer perceptron model (MLP) Backpropagation Convolutional Neural Networks Todays Class More on

772 views • 41 slides

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii

Dependency-based Convolutional Neural Networks for Sentence Embedding What is Hawaii s state flower ? ROOT Mingbo Ma Liang Huang Bing Xiang Bowen Zhou CUNY IBM T. J. Watson ACL 2015 Beijing Convolutional Neural Network

742 views • 50 slides

Neural Network Part 3: Convolutional Neural Networks Yingyu Liang Computer Sciences 760 Fall

Neural Network Part 3: Convolutional Neural Networks Yingyu Liang Computer Sciences 760 Fall 2017 http://pages.cs.wisc.edu/~yliang/cs760/ Some of the slides in these lectures have been adapted/borrowed from materials developed by Mark Craven,

819 views • 48 slides

Neural Network Basics Part II Content Image-to-image Why fully convolutional?

Neural Network Basics Part II Content Image-to-image Why fully convolutional? Fully Convolutional Networks (FCN) Up-sampling Network architecture Recurrent Neural Networks Sequence data and representation

626 views • 45 slides

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan

11/21/2018 ImageNet Classification with Deep Convolutional Neural Networks Prepared by Faizaan Naveed Won Mo (Andy) Jung Lassonde School of Engineering, York University Canada Objective Overview of Convolutional Neural Network

567 views • 17 slides

and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast

Band-limited Training and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast Training of Convolutional Networks through FFTs Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x

648 views • 46 slides

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette Phil Blunsom Department of Computer Science, Oxford University Overview of Model Represent sentences by extracting more abstract features Input:

195 views • 16 slides

Convolutional Neural Networks Basics Praveen Krishnan Overview Paradigm Shift Simple

Convolutional Neural Networks Basics Praveen Krishnan Overview Paradigm Shift Simple Network Convolutional Network Layers Case Study1: Alex Net Training Generalization Visualizations Transfer Learning Case

1.02k views • 53 slides

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation

DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation EMNLP19 Deepanway Ghosal, Navonil Majumder, Soujanya Poria , Niyati Chhaya and Alexander Gelbukh Singapore University of Technology and Design, Singapore

995 views • 21 slides

Convolutional Neural Network to Model Articulation Impairments in Patients with Parkinsons

Convolutional Neural Network to Model Articulation Impairments in Patients with Parkinsons Disease asquez-Correa 1 , 2 Juan Camilo V Juan Rafael Orozco-Arroyave 1 , 2 , Elmar N oth 2 1 GITA research group, University of Antioquia UdeA. 2

621 views • 32 slides

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides

Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Lecture 8: Convolutional Neural Nets Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier) https://courses.grainger.illinois.edu/cs447/ 1 Convolutional Neural Nets (ConvNets, CNNs) [4 parameters, applied 3

615 views • 24 slides

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Monday (2/5) 30 minutes Topics: Optimization Basic neural networks Neural Network

619 views • 41 slides

Introduction to High-Performance Machine Learning: Convolutional Neural Networks Valeriu Codreanu

Introduction to High-Performance Machine Learning: Convolutional Neural Networks Valeriu Codreanu SURFsara 1 Introduction to High-Performance Machine Learning: www.prace-ri.eu Convolutional Neural Networks SURFsara History: 1971: Founded

1.12k views • 82 slides

Effective Use of f Word Order for Text xt Categorization wit ith Convolutional Neural Network

Effective Use of f Word Order for Text xt Categorization wit ith Convolutional Neural Network Presenter: Yi-Hsin Chen Text xt Categorization Automatically assign pre-defined categories to documents written in natural language

463 views • 14 slides

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides

Atlas registration and ensemble deep convolutional neural network-based prostate segmentation

Atlas registration and ensemble deep convolutional neural network-based prostate segmentation using magnetic resonance imaging Haozhe Jia, Yong Xia, Yang Song, Weidong Cai, Micheal Fulham, David Dagan Feng Paper link:

530 views • 36 slides