Low-power smart imagers for vision-enabled Low-power smart imagers - PowerPoint PPT Presentation

Low-power smart imagers for vision-enabled Low-power smart imagers for vision-enabled wireless sensor networks and a case study wireless sensor networks and a case study J. Fernández-Berni, R. Carmona-Galán, Á. Rodríguez-Vázquez Institute of Microelectronics of Seville (IMSE-CNM), CSIC - Universidad de Sevilla (Spain) First ACM/ I EEE I nternational Workshop On Architecture of Smart Camera Clermont Ferrand, France April 5-6, 2012

Low-level image processing Low-level image processing INSTRUCTION FLOW, COMPUTATIONAL LOAD Sobel Operators - Potential parallel operation - Moderate accuracy required WASC April 2012, Clermont-Ferrand (France)

Conventional approach Conventional approach  Analytic issues are mostly software issues Brute force pattern matching used by many system developers  Extremely inefficient in terms of speed and power  Imager Digital Signal Processor Low-level Mid-level High-level ADC tasks tasks tasks F F f f I nformation Flow: F F >> >> f > f > f ARRAY OF SENSORS WASC April 2012, Clermont-Ferrand (France)

Focal-plane array computing Focal-plane array computing  Content-aware sensing-processing  Progressive extraction of relevant information  Parallel and distributed processing  Distributed memory Smart Imager Digital Signal Processor Mid-level High-level ADC tasks tasks f f f I nformation Flow: f > f > f ARRAY OF SENSOR – PROCESSORS WASC April 2012, Clermont-Ferrand (France)

Focal-plane array computing Focal-plane array computing Sensor Memory P ER P IXEL : Optical sensing   Neighbor Connectivity  Local Processing Mixed ‐ signal processor Local Memory   Single Instruction Multiple Data (SIMD) Architecture WASC April 2012, Clermont-Ferrand (France)

Focal-plane array computing Focal-plane array computing  Several generation of chips designed FLIP-Q  With fully programmable features  Covering a large variety of functional targets  Image-to-Decision at >1,000F/s with 60nW per pixel  Spatio-temporal filtering with 22nJ/cycle  Content-aware HDR acquisition with >145dB intra-frame DR  Etc. WASC April 2012, Clermont-Ferrand (France)

J. Fernández Berni, R. Carmona Galán and L. FLI P-Q: floorplan FLI P-Q: floorplan Carranza González, “FLIP-Q: A QCIF Resolution Focal-Plane Array for Low-Power Image Processing,” in IEEE J. Solid-State Circuits, vol. 46, no. 3, pp. 669–680, March 2011 WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: elementary processing cell FLI P-Q: elementary processing cell • Reset transistor • n-well/p-substrate photodiode • Electronic global shutter • Programmable block-wise image filtering and averaging • Programmable block-wise image energy computation • Readout circuitry WASC April 2012, Clermont-Ferrand (France)

Physical design Physical design  Crucial aspect affecting the total area, the fill factor and the pixel pitch  The electrical design must be realized bearing in mind the 29µm subsequent physical design  Relevant issues:  Metal layers available  Full-custom routing  Make the most of the design rules 34µm WASC April 2012, Clermont-Ferrand (France)

Physical design Physical design 60% +2% per additional µm in the elementary cell WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: A prototype smart imager FLI P-Q: A prototype smart imager WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: on-chip early vision FLI P-Q: on-chip early vision Programmable Gaussian filtering chip ideal error WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: on-chip early vision FLI P-Q: on-chip early vision Fully-programmable multi-resolution scene representation On-chip images WASC April 2012, Clermont-Ferrand (France)

FLI P-Q: on-chip early vision FLI P-Q: on-chip early vision Image pre-distortion for reduced kernel filtering Original Reduced kernels kernels WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node Smart Imager Digital Signal Processor Mid-level High-level ADC tasks tasks f f f I nformation Flow: f > f > f ARRAY OF SENSOR – PROCESSORS WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a Wi-FLI P: a vision-enabled vision-enabled WSN node WSN node Imote2 (MEMSIC Inc.) WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node DoG-based edge detection WASC April 2012, Clermont-Ferrand (France)

Wi-FLI P: a vision-enabled WSN node Wi-FLI P: a vision-enabled WSN node Very low throughput due to slow GPIO ports and TinyOS latency WASC April 2012, Clermont-Ferrand (France)

Case study: early detection of forest fires Case study: early detection of forest fires  High economic cost  Short maintenance cycles  Coarse grain coverage  Exact location must be inferred 27

Case study: early detection of forest fires Case study: early detection of forest fires • Vision-enabled Wireless Sensor Network ADVANTAGES DRAWBACKS  Robustness  Ultra low power consumption required  Scalability  Reliability  Better temporal resolution  Simpler smoke location 28

Case study: early detection of forest fires Case study: early detection of forest fires • Reconfigurable focal plane • A power-efficient vision algorithm for smoke detection • Multiresolution scene representation Candidate regions • Clustering ratio • Growth rate • Propagation speed SMOKE! 29

Case study: early detection of forest fires Case study: early detection of forest fires • Preliminary field tests http://www.imse-cnm.csic.es/vmote/ Original sequence Motion detector Our algorithm 30

Case study: early detection of forest fires Case study: early detection of forest fires • On-site smoke detection with Eye-RI S v1.2 31

Case study: early detection of forest fires Case study: early detection of forest fires • Field tests with Wi-FLI P 32

Case study: early detection of forest fires Case study: early detection of forest fires • Prescribed burning of a 95m x 20m shrub plot Wi-FLIP monitored all the activity for over two hours  No false alarm triggered  Successful smoke detection for two of the three vegetation areas explored  Thin smoke generated from a very sparse vegetation area was not detected  33

CONCLUSI ONS CONCLUSI ONS • Early vision tasks represent a considerably heavy computational load . • SIMD-based massively parallel mixed-signal processing takes advantage of their intrinsic characteristics to achieve high power efficiency and computational power . • FLI P-Q : A prototype vision chip tailored for ultra low-power applications. Very competitive in the state of the art. • Wi-FLI P : A vision-enabled Wireless Sensor Network node supported by I mote2 . Current drawback: low throughput. • Case study: Early detection of forest fires , with very good results in terms of reliability . 34

Thank you very much Thank you very much for your attention for your attention berni@imse-cnm.csic.es berni@imse-cnm.csic.es Publication Date: May 31, 2012 35

Acknowledgments This work is financially supported by Andalusian Regional Government, through project 2006-TIC-2352, the Spanish Ministry of Economy and Competitiveness, through projects TEC 2009-11812 and IPT-2011-1625-430000, both co-funded by the EU-ERDF and by the Office of Naval Research (USA), through grant N000141110312. 36