a 240x180 120db 10mw 12us latency sparse output vision
play

A 240x180 120dB 10mW 12us latency sparse output vision sensor for - PowerPoint PPT Presentation

Nov 23, 2022 •22 likes •202 views

A 240x180 120dB 10mW 12us latency sparse output vision sensor for mobile applications Raphael Berner, Christian Brandli, Minhao Yang, Shih Chii Liu and Tobi Delbruck Inst. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland

  1. A 240x180 120dB 10mW 12us ‐ latency sparse output vision sensor for mobile applications Raphael Berner, Christian Brandli, Minhao Yang, Shih ‐ Chii Liu and Tobi Delbruck Inst. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland 1

  2. Machine Vision for Mobile Applications • ENERGY matters because it is limited • LATENCY is crucial for real time interaction With conventional imager • Low frame rate: low energy / high latency • High frame rate: low latency / high energy Most of the time and energy spent on redundant information in and between frames WE NEED SMARTER WAYS OF SENSING & PROCESSING 2

  3. Simplified Dynamic Vision Sensor ( DVS ) pixel architecture Lichtsteiner et al. JSSC 2009 3

  4. Example DVS application ‐ RoboGoalie 4

  5. Example DVS application – RoboGoalie Achieves 550Hz update rate at 4% laptop CPU load Achieves 550fps (equivalent) at 4% CPU load 5 Delbruck et al., 2009

  6. The apsDVS chip adds an intensity readout to the DVS Intensity reset threshold I ON Intensity Change value events log I OFF Event comparators reset (ganglion cells) I change amplifier (bipolar cells) photoreceptor 6

  7. apsDVS pixel (aps=Active Pixel Sensor) 7

  8. The apsDVS chip SBRET10 • 0.18um 1P6M CMOS • MiM capacitors • 240x180 pixels • 22% fill factor • 12mW power Row APS readout Column APS readout 8

  9. 9

  10. 10

  11. 11

  12. The apsDVS can enhance machine vision in several ways • Low APS frame rate  Low power • DVS output  Low latency, sparse data Combined advantages: 1. DVS delivers motion features for free 2. Continuous tracking solves correspondence problem 3. DVS can be used to control frame rate or ROI Slide 12

  13. The apsDVS in Comparison This work (apsDVS) ATIS (Posch et al.) Intensity readout APS Time domain Image Dynamic Range 57 dB 125 dB Image FPN 1 % <0.25 % Pixel size [um 2 ] 18.5 x 18.5 30 x 30 Fill factor 22% 15% Pixel complexity 44 FET, 2 C, 1 PD 77 FET, 2 C, 2 PD Fixed integration time YES NO Power Consumption ~10mW (240x180) ~100mW (304x240) 13

  14. Thank you Christian Dr. habil. Dr. Raphael Prof. Tobi Minhao Brandli Shih ‐ Chii Liu Berner Delbruck Yang sensors.ini.uzh.ch Funding Organizations University of Zurich ETH Zurich EU 7th framework program (SeeBetter) Swiss National Science Foundation (NCCR Robotics) 14

  15. SBRET10 pixel layout 22% fill factor 18.5um 15

  16. Technology UMC 180 6M 1P Chip size 5x5mm Design APS-DVS array size 240x180 Pixel size 18.5 x18.5 um 2 Supply voltage(s) 3.3V/1.8V Pixel architecture apsDVS, 44T+2C. Fill factor 22% Power consumption 7.4mW low activity 13.5mW high activity DVS event threshold limit 12% contrast Measured DVS illumination operating range 120dB (down to 0.1lux) DVS latency 12us APS readout frame rate 40 FPS APS FPN (midlevel) 1% of full signal swing Readout Noise <2mV Output Voltage Range 1.1V APS Dynamic Range ~55dB 16

  17. Simplified Dynamic Vision Sensor ( DVS ) pixel architecture A random variation  log I Lichtsteiner et al. JSSC 2009 17

  18. 18

Recommend

The Input/Output Complexity of Sparse Matrix Multiplication Rasmus Pagh, Morten St ockel IT

The Input/Output Complexity of Sparse Matrix Multiplication Rasmus Pagh, Morten St ockel IT

The Input/Output Complexity of Sparse Matrix Multiplication Rasmus Pagh, Morten St ockel IT University of Copenhagen September 9 2014 Pagh, St ockel September 9 2014 1 / 29 Sparse matrix multiplication Problem description Sparse

572 views • 32 slides
Output Perception Colour models Managing output 1 Human Elements of Graphical Output

Output Perception Colour models Managing output 1 Human Elements of Graphical Output

Output Perception Colour models Managing output 1 Human Elements of Graphical Output Psychophysics: out there vs. in here Relationship between external stimuli and internal sensations Temporal and spatial resolution Vision

702 views • 29 slides
Output Perception Colour models Managing output 1 Human Elements of Graphical Output

Output Perception Colour models Managing output 1 Human Elements of Graphical Output

Output Perception Colour models Managing output 1 Human Elements of Graphical Output Psychophysics: out there vs. in here Relationship between external stimuli and internal sensations Temporal and spatial resolution Vision

939 views • 28 slides
Output-sensitive algorithms for sumset and sparse polynomial multiplication Daniel S. Roche

Output-sensitive algorithms for sumset and sparse polynomial multiplication Daniel S. Roche

Sparsity and Arithmetic Informative Example Consequences and Conclusions Output-sensitive algorithms for sumset and sparse polynomial multiplication Daniel S. Roche Andrew Arnold Cheriton School of Computer Science Computer Science

989 views • 57 slides
Networks 1a. Define a latency procedure that takes one input, a list of lists of bit arrival times,

Networks 1a. Define a latency procedure that takes one input, a list of lists of bit arrival times,

Networks 1a. Define a latency procedure that takes one input, a list of lists of bit arrival times, and produces as output a number representing the latency of the communication. For example, imagine that we send three bits across four semaphore

440 views • 14 slides
The Input/Output Complexity of Sparse Matrix Multiplication Rasmus Pagh 1 , Morten St ockel 2 1

The Input/Output Complexity of Sparse Matrix Multiplication Rasmus Pagh 1 , Morten St ockel 2 1

The Input/Output Complexity of Sparse Matrix Multiplication Rasmus Pagh 1 , Morten St ockel 2 1 IT University of Copenhagen, 2 University of Copenhagen SIAM LA, October 26 2015 Pagh, St ockel ITU, DIKU October 26 2015 1 / 30 Sparse matrix

878 views • 35 slides
Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are

Sparse Matrices Example Of Sparse Matrices diagonal tridiagonal sparse many elements are zero lower triangular (?) dense few elements are zero These are structured sparse matrices. May be mapped into a 1D array so that a mapping

391 views • 6 slides
Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of

Sparse Matrices sparse many elements are zero dense few elements are zero Example Of Sparse Matrices diagonal tridiagonal lower triangular (?) These are structured sparse matrices. May be mapped into a 1D array so that a mapping

698 views • 11 slides
Sparse Flat Neighborhood Networks (SFNNs): Scalable Guaranteed Pairwise Bandwidth &amp; Unit

Sparse Flat Neighborhood Networks (SFNNs): Scalable Guaranteed Pairwise Bandwidth &amp; Unit

Sparse Flat Neighborhood Networks (SFNNs): Scalable Guaranteed Pairwise Bandwidth &amp; Unit Latency Timothy I. Mattox, Henry G. Dietz, &amp; William R. Dieter Electrical and Computer Engineering Department University of Kentucky Lexington, KY

353 views • 12 slides
Reconstructing Objects with Sparse Boundaries: Total Variation vs. Discrete Tomography Willem Jan

Reconstructing Objects with Sparse Boundaries: Total Variation vs. Discrete Tomography Willem Jan

Reconstructing Objects with Sparse Boundaries: Total Variation vs. Discrete Tomography Willem Jan Palenstijn iMinds-Vision Lab, Universiteit Antwerpen, Belgium 27 March 2014, Sparse Tomo Days, DTU, Denmark What is Discrete Tomography?

577 views • 28 slides
Enc Encoding ding, F , Fas ast and Slo t and Slow: w: Low-Latency Video Processing Using

Enc Encoding ding, F , Fas ast and Slo t and Slow: w: Low-Latency Video Processing Using

Enc Encoding ding, F , Fas ast and Slo t and Slow: w: Low-Latency Video Processing Using Thousands of Tiny Threads Presenter: Wen-Fu Lee Outline Vision &amp; Goals mu: Supercomputing as a Service Fine-grained Parallel Video

649 views • 50 slides
16. Recursion 2 Output: 103 Input: (3 + 5) * 20 Output: 160 Input: -(3 + 5) + 20 Output: 12

16. Recursion 2 Output: 103 Input: (3 + 5) * 20 Output: 160 Input: -(3 + 5) + 20 Output: 12

Motivation: Calculator Goal: we build a command line calculator Example Input: 3 + 5 Output: 8 Input: 3 / 5 Output: 0.6 Input: 3 + 5 * 20 16. Recursion 2 Output: 103 Input: (3 + 5) * 20 Output: 160 Input: -(3 + 5) + 20 Output: 12 Building a

387 views • 11 slides
Real-Time Structure and Object- Model Aware Sparse SLAM ARC Centre of Excellence for Robotic

Real-Time Structure and Object- Model Aware Sparse SLAM ARC Centre of Excellence for Robotic

Real-Time Structure and Object- Model Aware Sparse SLAM ARC Centre of Excellence for Robotic Vision Mehdi Hosseinzadeh Australian Centre for Robotic Vision The University of Adelaide Invited Talk, Second LPM Workshop, 23 May 2019 ICRA 2019

663 views • 32 slides
Fundamentals for Low Latency Communications H. Vincent Poor Princeton University Supported in

Fundamentals for Low Latency Communications H. Vincent Poor Princeton University Supported in

Fundamentals for Low Latency Communications H. Vincent Poor Princeton University Supported in part by NSF Grants CCF-093970 and CCF-1513915. The Internet-of-Things (IoT) Vision Interconnecting perhaps100s of billions of devices Key

873 views • 36 slides
Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games

Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games

12/6/2011 Is It Latency or Do You Just Suck? Latency Can Kill: Precision and Deadline in Online Games http://www.youtube.com/watch?v=r6PwHkhEAkU Mark Claypool and Kajal Claypool ACM MMSys Scottsdale, AZ, USA February 2010

412 views • 4 slides
Chapter 12 Overview Devices and Output Visual Output Dynamic Visualizations Sound

Chapter 12 Overview Devices and Output Visual Output Dynamic Visualizations Sound

Chapter 12 Overview Devices and Output Visual Output Dynamic Visualizations Sound Output Developments in Output Devices and Output Output-information in some perceptible form coming from an electronic, internal

170 views • 6 slides
Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong

Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong

Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs Jinkyu Jeong Sungkyunkwan University (SKKU) Source: Gyusun Lee et al., Asynchronous I/O Stack: A Low-latency Kernel I/O Stack for Ultra-Low Latency SSDs, USENIX

672 views • 48 slides
= + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit,

= + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit,

Sparse processing / Compressed sensing Model : y = Ax + n , x is sparse = + N 1 N M measurements M 1 sparse signal Problem : Solve for x Basis pursuit, LASSO (convex objective function) Matching pursuit (greedy method)

210 views • 17 slides
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation

Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup: Data/signal in n-dimensional

418 views • 25 slides
Sparse Representations The Sparse Representations (SRs) framework was inspired by studies of

Sparse Representations The Sparse Representations (SRs) framework was inspired by studies of

Manifold Based Sparse Representation for Robust Expression Recognition without Neutral Subtraction Raymond Ptucha, Grigorios Tsagkatakis, Andreas Savakis Department of Computer Engineering, Rochester Institute of Technology, Rochester, NY Nov

411 views • 11 slides
The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible

The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible

The Benefits and Burdens of Nuclear Latency by Mehta and Whitlark Andrew Malandra Possible Effects of Latency The Background Virtual Deterrence Latency Provocation Theory Theory You cant deter challengers Essentially its the

273 views • 24 slides
Struck: Structured Output Tracking with Kernels Sam Hare, Amir Saffari, And Philip H. S. Torr

Struck: Structured Output Tracking with Kernels Sam Hare, Amir Saffari, And Philip H. S. Torr

Struck: Structured Output Tracking with Kernels Sam Hare, Amir Saffari, And Philip H. S. Torr International Conference On Computer Vision (ICCV), 2011 Motivations Problem: tracking-by-detection Input: target Output: locations over times

832 views • 39 slides
Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency

Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency

Bibliography on Synchronous Elastic (aka Latency Insensitive) Systems July 20, 2009 Latency insensitive designs [CMSV01,CSV02,CSV03,CM04,BMdS06a,Sve04,VA09] SELF implementation and compilation to elastic designs [CKG06,CK07,HB08] Interlock

313 views • 5 slides
STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data

STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data

STORM AND LOW-LATENCY PROCESSING www.inf.ed.ac.uk Low latency processing Similar to data stream processing, but with a twist Data is streaming into the system (from a database, or a network stream, or an HDFS file, or ) We want to

573 views • 31 slides

More recommend