Efficient Hardware Architectures and Algorithms for Embedded Vision - PowerPoint PPT Presentation

Efficient Hardware Architectures and Algorithms for Embedded Vision Systems Eva Dokladalova 1 , 2 1 ESIEE Paris, Computer Science Department, France 2 Paris-Est University, LIGM (Gaspard Monge Computer Science Laboratory), A3SI team, CNRS UMR 8049, France October 27, 2015 E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 1 / 40

Who I am? Ing. DEA Head of CS spec. PhD PostDoc/Perm. Research E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 2 / 40

Efficient Hardware Architectures and Algorithms for Embedded Vision Systems Where and why? Applicative context Very specific sensing systems 1 Multiple technologically different vision sensors 2 High perfomance computing ability 3 Low processing latency requirements 4 Low energy consumption constraints 5 E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 3 / 40

Where and why? a a credits: Sagem Defense E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 4 / 40

Where and why? a a credits: CEA LIST, MEDEA+ CarVision project E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 4 / 40

Where and why? Alpha + gamma + electron a alpha gamma electron a credits: UTEF Praha, ZCU Plzen E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 4 / 40

Efficiency bottlenecks Intensive memory accesses E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 5 / 40

Efficiency bottlenecks Intensive memory accesses High performance computing E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 5 / 40

Efficiency bottlenecks Intensive memory accesses High performance computing ! Low working frequency E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 5 / 40

Propositions Axioms on algorithm limitations Respect sequential data reading → sensor pixel stream Enable on-the-fly processing → eliminate intermediate storage Reduce algorithm complexity → O ( 1 ) per pixel Consider low and object extraction Remark: properties interesting for all type of computing platforms E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 6 / 40

Image processing approaches Geometric space Abstract space Linear Statistical Convolution Multivariate analysis Fourrier Neural networks Linear Wavelets Syntactic Morphologic Grammars Filtering Indexation Measures Nonlinear Structural pattern Segmentation E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 7 / 40

Contents Algorithms 1 Implementation 2 Applications 3 Conclusion and perspectives 4 E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 8 / 40

Mathematical morphology Born in 1964 at Ecole des Mines de Paris, France. Mathematical theory studing interactions between image and set called structuring element (SE) Various image processing techniques implemented by combining only a few simple operations: erosion/dilation, closing/opening E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 9 / 40

Binary dilation and erosion Definition Let F be a binary image and B be a set called structuring element (SE). δ B ( F ) = { z : B ( z ) ∩ F } ε B ( F ) = { z : B ( z ) ⊆ F } Input image F δ 5 × 5 δ 7 × 7 ε 5 × 5 ε 7 × 7 E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 10 / 40

Grayscale dilation and erosion Definition Let B be a flat structuring element (SE) [ δ B ( f )]( x ) = max b ∈ B [ f ( x + b )] [ ε B ( f )]( x ) = min b ∈ B [ f ( x − b )] Input δ 5 × 5 δ 11 × 11 ε 5 × 5 ε 11 × 11 E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 11 / 40

Compound operators Opening; Closing: γ B ( f ) = δ B [ ε B ( f )] ; ϕ B ( f ) = ε B [ δ B ( f )] Original Opening Closing Gradient: g ( f ) = δ B ( f ) − ε B ( f ) Top hat: th γ ( f ) = f − γ B ( f ) ; th ϕ ( f ) = ϕ B ( f ) − f E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 12 / 40

Compound operators Opening; Closing: γ B ( f ) = δ B [ ε B ( f )] ; ϕ B ( f ) = ε B [ δ B ( f )] Original Opening Closing Gradient: g ( f ) = δ B ( f ) − ε B ( f ) Top hat: th γ ( f ) = f − γ B ( f ) ; th ϕ ( f ) = ϕ B ( f ) − f E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 12 / 40

Practical use cases Image enhancing and nonlinear filtering 1 ASF λ = ϕ λ γ λ ϕ λ − 1 γ λ − 1 . . . ϕ 1 γ 1 Figure: Alternate sequential filters E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 13 / 40

Practical use cases Image enhancing and nonlinear filtering 1 ASF λ = ϕ λ γ λ ϕ λ − 1 γ λ − 1 . . . ϕ 1 γ 1 Directional size distributions (texture analysis) 2 � � � PS λ j B ( f ) = γ λ i B f − γ λ j B f D Figure: Texture analysis E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 13 / 40

Application Challenges Synthesis of application requirements Variety of sizes, shapes, orientation of Structuring Elements l l left l right l up l up up l up l left l left l right l right l down l α l down l down down Rectangle Octagon Horizontal Vertical Inclined Computational complexity Naively: n × n square SE has complexity O ( n 2 ) E.g., B = 11 × 11 needs 120 max () per pixel E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 14 / 40

Optimizing Computation of Dilation Structuring element decomposition Decomposition by means of the Minkowski set addition δ B 1 ⊕ B 2 ( f ) = δ B 1 δ B 2 ( f ) Reduce O ( n 2 ) to O ( n ) Dilation by square SE is equal to horizontal dilation followed by vertical dilation l up l left l right l up l left l right l down l down Rectangle Horizontal Vertical 1-D Efficient algorithms with O ( 1 ) Reduce O ( n ) to O ( 1 ) E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 15 / 40

Morphological operators in constant time Principle Data read from left to right Limited field of view Results obtained only from visible data E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 16 / 40

1-D Dilation in constant time SE Principle f ( t ) Field of view limited on SE Find maximum within scope of the SE Erase the most recent pixel t t within SE if its value is smaller than the current Figure: 1-D stream dilation pixel E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 17 / 40

1-D Dilation in constant time SE Principle f ( t ) Field of view limited on SE Find maximum within scope of the SE Erase the most recent pixel x - l x t t within SE if its value is smaller than the current Figure: 1-D stream dilation pixel E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 17 / 40

1-D Dilation in constant time SE Principle f ( t ) Field of view limited on SE Find maximum within scope of the SE Erase the most recent pixel x - l+1 x+1 t within SE if its value is smaller than the current Figure: 1-D stream dilation pixel E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 17 / 40

1-D Dilation in constant time SE Principle f ( t ) Field of view limited on SE Find maximum within scope of the SE Erase the most recent pixel x - l+1 x+1 t within SE if its value is smaller than the current Figure: 1-D stream dilation pixel Interesting feature Size of SE can be changed on the fly ! SE implemented as a simple queue (FIFO) E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 17 / 40

1-D Opening in general Let B be a flat SE of length l Gray-scale opening is defined as the union of all SEs that fit under the graph of a function f f ( x ) f ( x ) B B B B x x (a) Input signal f ( x ) (b) Output signal γ B f ( x ) Intuition Erase all signal peaks narrower than l pixels. E. Dokladalova (ESIEE Paris) Efficient Hardware Architectures and Algorithms for Embedded Vision Systems October 27, 2015 18 / 40