Running Deep Learning in less than 100KB on Microcontrollers Pete - PowerPoint PPT Presentation

Running Deep Learning in less than 100KB on Microcontrollers

Pete Warden Engineer, TensorFlow petewarden@google.com @petewarden

Why am I here?

150 Billion Devices! Growing faster than internet users or smaruphones

Why ML?

Energy! Many devices rely on batuery or energy harvesting Transmituing data takes a lot of power, and can’t improve fast enough Capturing and processing data locally very cheap

Energy! Most captured data is currently being wasted ML lets us turn it into something actionable

How is this done?

What are the challenges? Less than 100KB of RAM and storage Less than 10 million arithmetic ops per second Can’t rely on fmoating point hardware No operating system

Model Design We needed a 20KB model Happily common in speech world Learned a lot about quantization Actually just a tiny image CNN on spectrograms

Model Design Uses fewer than 400,000 arithmetic operations htups://www.tensorglow.org/tutorials/sequences/a udio_recognition

Sofuware Design TensorFlow Lite still > 100KB binary size Depends on Posix and standard C/C++ libraries Uses dynamic memory allocation

Sofuware Design But a lot we don’t want to lose from TensorFlow Lite: - Existing op implementations - Well-documented APIs and fjle format - Conversion tooling

Sofuware Design Modularized existing code Separated out API defjnitions from implementations Used reference code Added minimal new runtime layer for MCUs

Sofuware Design Focused on getuing one end to end example working, rather than going broad poruing whole framework at once

What does this mean in practice?

int32 acc = 0; for (int filter_y = 0; filter_y < filter_height; ++filter_y) { for (int filter_x = 0; filter_x < filter_width; ++filter_x) { const int in_x = in_x_origin + dilation_width_factor * filter_x; const int in_y = in_y_origin + dilation_height_factor * filter_y; // If the location is outside the bounds of the input image, // use zero as a default value. if ((in_x >= 0) && (in_x < input_width) && (in_y >= 0) && (in_y < input_height)) { int32 input_val = input_data[Offset(input_shape, b, in_y, in_x, ic)]; int32 filter_val = filter_data[Offset( filter_shape, 0, filter_y, filter_x, oc)]; acc += (filter_val + filter_offset) * (input_val + input_offset); } } }

Reference code is imporuant Most ML operations can be implemented simply Most frameworks only ship with optimized versions Understandable, but makes it very hard to extend or optimize for other platgorms

What’s the takeaway?

There is no killer app Voice intergaces are the closest Vision, accelerometer, audio sensors ofger a lot Need to connect with the right problems

Think about your domain What could you do if your model ran on a 50 cent chip that could be peeled and stuck anywhere, and run forever?

Get it. Try it. Code : github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro Docs : tensorglow.org/lite/guide/microcontroller Example : g.co/codelabs/sparkfunTF

Pete Warden Engineer, TensorFlow petewarden@google.com @petewarden

Running Deep Learning in less than 100KB on Microcontrollers Pete - PowerPoint PPT Presentation

Running Deep Learning in less than 100KB on Microcontrollers Pete Warden Engineer, TensorFlow petewarden@google.com @petewarden Why am I here? 150 Billion Devices! Growing faster than internet users or smaruphones Why ML? Energy! Many

Deep Water Running Richard Lucas What is it? Mimicking running action while in the water

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: State of the Art (2020) Deep Learning Lecture Series https://deeplearning.mit.edu

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

Regularization for Deep Learning Lecture slides for Chapter 7 of Deep Learning

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep Reinforcement Learning and Complex Environments Raia Hadsell End-to-end Deep Learning

Bayesian Deep Learning Mohd Adnan Problems With Deep Learning What does a model not know?

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

DSSTNE: Deep Learning At Scale For Large Sparse Datasets

Deep Learning With Differential Privacy Presenter: Xiaojun Xu Deep Learning Framework