cDeepArch: A Compact Deep Neural Network Architecture for Mobile - PowerPoint PPT Presentation

Sep 16, 2022 •197 likes •437 views

cDeepArch: A Compact Deep Neural Network Architecture for Mobile Sensing Kang Yang 1 , Xiaoqing Gong 1 , Yang Liu 2 , Zhenjiang Li 2 , Tianzhang Xing 1 , Xiaojiang Chen 1 , Dingyi Fang 1 1 Northwest University, China 2 City University of Hong Kong

cDeepArch: A Compact Deep Neural Network Architecture for Mobile Sensing Kang Yang 1 , Xiaoqing Gong 1 , Yang Liu 2 , Zhenjiang Li 2 , Tianzhang Xing 1 , Xiaojiang Chen 1 , Dingyi Fang 1 1 Northwest University, China 2 City University of Hong Kong 1
Motivation … Camera + Learning Gyro. Technology Acc.
Application ? Cognitive decline
Application First-person view Cognitive aid system Recognizing pot cup open open close
Common design principle . . . Rich sensor data Recognized by learning Applications
Challenges Large targets . . .
Challenges • Deep Learning Too large Big deep neural network Resource-limited
Challenges • Deep Learning inaccurate Original Shrunk No quantitative measure on model model available resource conditions
Any countermeasure? 0101… Server • Long and uncontrollable latency • High Service cost Potential privacy leakage •
Our solution Context (office) Large targets . . .
Our solution compact network (Office) Context recognition + Object recognition Context-oriented (computer, large and deep network target recognition mouse…) compact network adequate storage computation resource
Our solution energy Context recognition computation + Available resource conditions Context-oriented target recognition • not based on designer’s experience • Formulation facilitated configuration
Convolutional Neural Network Image data Conv1 Pool1 Conv2 Pool2 FC1 • Convolutional layer (dominant) • Pooling layer • Full connected layer
Formulation facilitated configuration Selected W F W o *W o S P C (W+2P)*(W+2P)
From computation to resource cost Conv1:64 fc:5 Conv2:128 computation resource(energy) designed network Conv1:16 Conv2:32 fc:5 : computation : actual resource consumption a small scale network
Now… Context recognition + Object recognition Context-oriented target recognition Recognition task decomposition • • Formulation facilitated configuration From formulation to estimate the resource consumption •
Enhancement: Convolutional layer Original Conv1 Conv2 Conv3 model Separated Conv1a Conv1b Conv2 Conv3 model #$ % − ⁄ 1 ) ⁄ ! ≤ ) 2
Evaluation
Experiments setup Dataset: • o Context recognition : § MIT Place2 (related to the daily contexts ) Object recognition : o § Cifar10 § Cifar100 (20 classes associated contexts)
Evaluation results • Overall performance 10 targets 20 targets
Conclusion 1, 2, 3 1. Large targets Decompose recognition task 2. Systematic way to configure network Execution overhead formulation facilitated configuration 3. Enhancement techniques

Recommend

Agenda Overview of deep learning Building a FAQ model with DeepLearning4J Integrating

Agenda Overview of deep learning Building a FAQ model with DeepLearning4J Integrating with a chatbot application Overview of deep learning AI ML Deep Learning Neural network architecture Neural network architecture Neural

469 views • 46 slides

A Neural Network Architecture for Detec2ng Gramma2cal Errors in SMT A Neural Network Architecture

A Neural Network Architecture for Detec2ng Gramma2cal Errors in SMT A Neural Network Architecture for Detec2ng Gramma2cal Errors in SMT Morpho-Syntac-c features outperform word embeddings on this task Syntac-c n-grams improve the

1.18k views • 38 slides

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics Activation Functions Stochastic Gradient Descent (SGD) Regularization (Dropout) Training Tips and Tricks Neural Network (NN)

828 views • 45 slides

Neural Architecture Search Yu Cao What is Neural Architecture Search (NAS) Selecting the optimal

Neural Architecture Search Yu Cao What is Neural Architecture Search (NAS) Selecting the optimal network architecture automatically via machine instead of design it manually. It is an important aspect of AutoML. NAS search space 1.

3.43k views • 37 slides

Recurrent Neural Networks CS60010: Deep Learning Abir Das IIT Kharagpur Mar 11, 2020

Recurrent Neural Networks CS60010: Deep Learning Abir Das IIT Kharagpur Mar 11, 2020 Introduction Recurrent Neural Network LSTM Agenda Get introduced to different recurrent neural architecture e.g. , RNNs, LSTMs, GRUs etc. Get

743 views • 48 slides

Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo,

Linearly Augmented Deep Neural Network Pegah Ghahremani, Johns Hopkins University, Jasha Droppo, Michael L. Seltzer, Microsoft Research Typical DNN Architecture Softmax L4 Layers are a composition of an affine and a non- linear function.

199 views • 19 slides

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Apr 7, 2016 1 Intro about me and my advisor Fourth year PhD with Prof. Bill Dally at Stanford.

2.06k views • 70 slides

Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7

Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7 1 History of Neural Network Research Neural network Deep belief net Back propagation Science

1.27k views • 108 slides

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Jan 6, 2015 A few words about us Fourth year PhD with Prof. Bill Dally at Stanford.

966 views • 54 slides

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides

Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks

Bit Fu Bi Fusion on Bit-Level Dynamically Composable Architecture for Deep Neural Networks Hardik Sharma Georgia Institute of Technology Jongse Park Arm, Inc. Naveen Suda Liangzhen Lai University of California, San Diego

275 views • 24 slides

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning Tasks that were once assumed to be purely in the human domain

1.92k views • 150 slides

ImageNet Classification with Deep Convolutional Neural Networks Krizhevsky et. all Outline

ImageNet Classification with Deep Convolutional Neural Networks Krizhevsky et. all Outline Introduction DataSet Architecture of the Network Reducing overfitting Learning Results Discussion Introduction

1.3k views • 31 slides

ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever,

ImageNet Classification with Deep Convolutional Neural Networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton Presented by Tugce Tasci, Kyunghee Kim 05/18/2015 Outline Goal DataSet Architecture of the Network Reducing

709 views • 35 slides

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning 2 Recap : NNets and the brain In their basic form, NNets

2.14k views • 145 slides

IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK

IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK LABORATORY FOR NEURAL COMPUTING FOR MACHINE LEARNING DEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE YORK UNIVERSITY, TORONTO NOVEMBER

211 views • 20 slides

EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1

EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1 Todays agenda Introduction of deep neural network Basics of neural network 2 Short bio Research interests Automatic speech

1.71k views • 62 slides

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are the commonest type of neural

534 views • 32 slides

How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit)

How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit) Massachusetts Institute of Technology In collaboration with Yu-Hsin Chen, Joel Emer, Yannan Wu, Tien-Ju Yang, Google Mobile Vision Team Slides available at

692 views • 48 slides

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan,

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan, Laxmikant Kishor Mokadam, Xipeng Shen, Seung-Hwan Lim, Robert Patton 1 Build an image classifier? Deep Neural Network (DNN) CPU GPU Training

957 views • 41 slides

Global Optimality in Neural Network Training Benjamin D. Haeffele and Ren Vidal Johns Hopkins

Global Optimality in Neural Network Training Benjamin D. Haeffele and Ren Vidal Johns Hopkins University, Center for Imaging Science. Baltimore, USA Questions in Deep Learning Architecture Design Optimization Generalization Questions in

1.56k views • 91 slides