Neural Network Overlay Using FPGA DSP Blocks Lenos Ioannou and - PowerPoint PPT Presentation

Nov 10, 2022 •26 likes •96 views

Neural Network Overlay Using FPGA DSP Blocks Lenos Ioannou and Suhaib A. Fahmy School of Engineering, University of Warwick, UK Introduction Long back-end tool compilation hinders rapid deployment of Neural Networks on FPGAs at the edge

Neural Network Overlay Using FPGA DSP Blocks Lenos Ioannou and Suhaib A. Fahmy School of Engineering, University of Warwick, UK
Introduction • Long back-end tool compilation hinders rapid deployment of Neural Networks on FPGAs at the edge • Use of overlays to build abstractions on top of the FPGA • Effectively enabling rapid deployment • Core NN operation, multiply-accumulate, maps well to DSP Blocks • Most FPGA NN implementations operate sub-max frequencies [1] • Can be solved by optimising the overlay around the DSP blocks [3]
Neural Network Test Cases • Trained 3 NNs using Tensorflow [2], each one comprises four layers • Use of ReLU in the intermediate layers • Considering the input bit-widths of the DSP48E2: • 18 bit weights • 27 bit inputs • 48 bit biases
Overlay • Each neuron is mapped to a single DSP block • DSP blocks alternate between two opmodes • Serial data flow • Needs to stall when # neurons > # inputs • Adjustable latency
Implementation Results • Implemented the overlay targeting the Zynq Ultrascale+ ZU7EV • Maintains low resource utilization • Feedforward serial data flow is highly efficient • High operating frequency • Near the DSP blocks’ theoretical maximum
Conclusion • Not offering peak performance in a particular NN implementation • Contribute to the more rapid deployment of NNs on FPGAs at the edge • Prioritise low resource utilization and energy efficiency Future work • Implement a mechanism to handle the data flow and stall accordingly • Expand the overlay for deeper topologies • Integration with a rapid compiler flow
References [1] E. Wu, X. Zhang, D. Berman, and I. Cho, “A high-throughput reconfigurable processing array for neural networks,” in Int. Conference on Field Programmable Logic and Applications (FPL), Sep. 2017. [2] Martin Abadi et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. [3] A. K. Jain, D. L. Maskell, and S. A. Fahmy, “Throughput oriented FPGA overlays using DSP blocks,” in 2016 Design, Automation Test in Europe Conference Exhibition (DATE), March 2016, pp. 1628–1633. [4] A. K. Jain, X. Li, P. Singhai, D. L. Maskell, and S. A. Fahmy, “DeCO: A DSP block based FPGA accelerator overlay with low overhead interconnect,” in Proc. Int. Symposium on Field-Programmable Custom Computing Machines (FCCM), 2016, pp. 1–8.

Recommend

An introduction to FPGA-based acceleration of neural networks Marco Pagani 1 What is an FPGA?

An introduction to FPGA-based acceleration of neural networks Marco Pagani 1 What is an FPGA? Field-programmable gate array (FPGA) are integrated circuits designed to be con fi gured after manufacturing for implementing arbitrary logic

429 views • 15 slides

CS5412: OVERLAY NETWORKS Lecture IV Ken Birman Overlay Networks 2 We use the term overlay

CS5412 Spring 2012 (Cloud Computing: Birman) 1 CS5412: OVERLAY NETWORKS Lecture IV Ken Birman Overlay Networks 2 We use the term overlay network when one network (or a network-like data structure) is superimposed upon an underlying

793 views • 57 slides

FPGA-based Training Accelerator Utilizing Sparseness of Convolutional Neural Network Hiroki

FPGA-based Training Accelerator Utilizing Sparseness of Convolutional Neural Network Hiroki Nakahara, Youki Sada, Masayuki Shimoda, Akira Jinguji, Shimpei Sato Tokyo Institute of Technology, JP FPL2019 @Barcelona Challenges in DL Training

369 views • 10 slides

De DeCO: : A DS DSP Block Based FPGA Accelerator Overlay Wi With Low Overhead Interconnect Ab

De DeCO: : A DS DSP Block Based FPGA Accelerator Overlay Wi With Low Overhead Interconnect Ab Abhishek Kumar Ja Jain, Xiangwei Li, Pranjul Singhai, Douglas L. Maskell School of Computer Science and Engineering Nanyang Technological

564 views • 39 slides

Network Intrusion Detection Using Neural Networks on FPGA SoCs Lenos Ioannou and Suhaib A. Fahmy

Network Intrusion Detection Using Neural Networks on FPGA SoCs Lenos Ioannou and Suhaib A. Fahmy School of Engineering, University of Warwick, UK Introduction Mainstream approaches in intrusion detection do not scale well to the embedded

936 views • 7 slides

FPGA-based Convolutional Neural Network Accelerator Ke Xu Xingyu Hou Manqi Yang Wenqi Jiang

FPGA-based Convolutional Neural Network Accelerator Ke Xu Xingyu Hou Manqi Yang Wenqi Jiang Outline Background Software Implementation Python / C implementation of VGG-16 Profiling and acceleration strategy

663 views • 20 slides

Contents Introduction Pipelined FPGA DNN accelerators Roof-line Model and optimizing

FPGA implementation of Quantized (Deep) Neural Network (QNN) Accelerators Biruk Seyoum Phd Student, Scoula Superiore SantAnna, Pisa Contents Introduction Pipelined FPGA DNN accelerators Roof-line Model and optimizing FPGA DNN

587 views • 23 slides

Tinsel: a manythread overlay for FPGA clusters POETS Project (EPSRC) Matthew Naylor, David

Tinsel: a manythread overlay for FPGA clusters POETS Project (EPSRC) Matthew Naylor, David Thomas, Simon Moore, Imperial College London University of Cambridge New compute devices allow ever-larger problems to be solved. But theres

782 views • 32 slides

A Time-Multiplexed FPGA Overlay with Linear Interconnect Xiangwei Li , Douglas L. Maskell

A Time-Multiplexed FPGA Overlay with Linear Interconnect Xiangwei Li , Douglas L. Maskell Abhishek K. Jain, Suhaib A. Fahmy, School of Computer Lawrence Livermore School of Engineering Science and Engineering National Laboratory University

531 views • 27 slides

CompSci 356: Computer Network Architectures Lecture 24: Overlay Networks Chap 9.4 Xiaowei Yang

CompSci 356: Computer Network Architectures Lecture 24: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview What is an overlay network? Examples of overlay networks End system multicast Unstructured Gnutella,

972 views • 61 slides

Processor Architecture Charles Eric LaForest J. Gregory Steffan ECE, University of Toronto FPGA

Octavo: An FPGA-Centric Processor Architecture Charles Eric LaForest J. Gregory Steffan ECE, University of Toronto FPGA 2012, February 24 Easier FPGA Programming We focus on overlay architectures Nios, MicroBlaze, Vector Processors

519 views • 38 slides

Peer-to-peer overlay networks Structured overlay network: Chord Basics Issue Large-scale

Peer-to-peer overlay networks Structured overlay network: Chord Basics Issue Large-scale distributed computer systems are spread across Each peer is assigned a unique m -bit identifier id . the Internet, yet their constituents need to

229 views • 3 slides

Feasibility of ILA as Network Virtualisation Overlay in multi-domain, multi-tenant Cloud Tako

Feasibility of ILA as Network Virtualisation Overlay in multi-domain, multi-tenant Cloud Tako Marks Bsc. 04-07-2017 System and Network Engineering University of Amsterdam 1 Introduction A Network Virtualisation Overlay (NVO) is a collection

614 views • 29 slides

L2 Overlay L2 Overlay

L2 Overlay L2 Overlay July 30, 2019 A. Gerrard, What Is Kubernetes?

536 views • 24 slides

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides

P4, FPGA, HLS Domain-specific language - network data forwarding Standardized and

P4, FPGA, HLS Domain-specific language - network data forwarding Standardized and maintained by P4 Language Consortium @ p4.org Independent of network protocols (VLAN, VxLAN, INT, ) and target architectures (ASIC, FPGA, NPU,

343 views • 9 slides

The Fundamentals of Deep Learning Building Blocks Theory with Applications Neural Units Neural

Deep Learning Antecedents Vision Case Study The Fundamentals of Deep Learning Building Blocks Theory with Applications Neural Units Neural Nets Deep Neural Nets Application ConvNets Jon Krohn LSTMs untapt jon@untapt.com

1.08k views • 106 slides

Tomography-based Overlay Network Monitoring Yan Chen, David Bindel, and Randy H. Katz UC

Tomography-based Overlay Network Monitoring Yan Chen, David Bindel, and Randy H. Katz UC Berkeley Motivation Infrastructure ossification led to thrust of overlay and P2P applications Such applications flexible on paths and targets,

624 views • 15 slides

Routing on Overlay Networks EECS 228 Abhay Parekh parekh@eecs.berkeley.edu October 28, 2002

Routing on Overlay Networks EECS 228 Abhay Parekh parekh@eecs.berkeley.edu October 28, 2002 What is an overlay network? A network defined over another set of networks The overlay addresses its own nodes A Links on one layer are

567 views • 38 slides

Andrew Clinton, Matt Liberty, Ian Kuon FPGA Routing (Interconnect) FPGA routing consists of a

Andrew Clinton, Matt Liberty, Ian Kuon FPGA Routing (Interconnect) FPGA routing consists of a network of wires and programmable switches Wire is modeled with a reduced RC network Drivers are modeled as a SPICE netlist 2-Level pass

673 views • 20 slides

Overlay-based IP Routing Richard Hartmann Chair for Network Architectures and Services

Overlay-based IP Routing Richard Hartmann Chair for Network Architectures and Services Department for Computer Science Technische Universit at M unchen November 10, 2010 Richard Hartmann: Overlay-based IP Routing 1 Outline Motivation

571 views • 19 slides

Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA Yufei Ma,

Scalable and Modularized RTL Compilation of Convolutional Neural Networks onto FPGA Yufei Ma, Naveen Suda, Yu Cao, Jae-sun Seo, Sarma Vrudhula School of Electrical, Computer and Energy Engineering School of Computing, Informatics,

1.01k views • 36 slides

CDA 4253/CIS 6930 FPGA System Design Sequential Circuit Building Blocks Hao Zheng Dept of Comp

CDA 4253/CIS 6930 FPGA System Design Sequential Circuit Building Blocks Hao Zheng Dept of Comp Sci & Eng USF Outline and Reading Introduction of behavioral modeling with process Sequential statements inside processes Modeling

1.58k views • 125 slides

FPGA What is a FPGA? How FPGAs work How do they work? Manufacturers

2/21/2012 Content FPGA What is a FPGA? How FPGAs work How do they work? Manufacturers Distributed RAM History FPGA vs ASIC FPGA and Microprocessors Alternatives to FPGAs ETI135, Advanced Digital IC Design

123 views • 10 slides