Image Processing Techniques Ali Abdallah University of Rome - Tor - PowerPoint PPT Presentation

Image Processing Techniques Ali Abdallah University of Rome - Tor Vergata 11 April 2016 Ali Abdallah Image Processing Techniques

Image Analysis Make computers detect objects of interest in images and video. Medical image analysis. Security and safety systems. Optical character recognition. Augmented reality. Confort and fun. Ali Abdallah Image Processing Techniques

Vision is Extreamly Hard Why is recognition so hard? Real world is made of a jumble of objects. Objects occlude one another and appear in different poses. Complex non-rigid articulation and extreme variations in shape and appearance. Vision is an amazing capability of natural intelligence. More human brain devoted to vision than anything else (Virtual Cortex) Ali Abdallah Image Processing Techniques

Digital Images The most common way to model colors in images is the RGB color model. The most common general used RGB density is 24-bit implementation, 8 bits per color. 256 × 256 × 256 ≈ 16 . 7 million colors. In image analysis grayscale is used usually, where every pixel 0 . 21 R + 0 . 72 G + 0 . 07 B (a) RGB (b) Grayscale Ali Abdallah Image Processing Techniques

Linear Correlation Detects linear correlation between variables. Important in computer vision in order to detect the outlines of rectangular objects. Figure: Detection of outlines objects Ali Abdallah Image Processing Techniques

Edges - Gradiant based The most common type of edge detection process uses a gradient operator. Given an image function as f ( x , y ), the magnitude of the = (∆ x 2 + ∆ y 2 ) gradient is approximated by g ( x , y ) ∼ 1 2 , where ∆ x = f ( x + n , y ) − f ( x − n , y ) and ∆ y = f ( x , y + n ) − f ( x , y − n ) Ali Abdallah Image Processing Techniques

Zero Cross based method 1.2 0.06 0.004 1.0 0.05 0.8 0.04 0.002 0.6 0.03 0.000 0.4 0.02 −0.002 0.2 0.01 0.0 0.00 −0.004 −0.2 −0.01 −6 −4 −2 0 2 4 6 −10 −5 0 5 10 −10 −5 0 5 10 Zero crossing occurs when the intensity of the image changes rapidly. Typically this happens on edges, but can also happens on some places that are not necessarily associated to edges. For this, usually a low pass filter is used to ”smooth” other small features. Typical implementation is to blur the image with a Gaussian filter first. Ali Abdallah Image Processing Techniques

Edges - Example Example of an edge detected image. Ali Abdallah Image Processing Techniques

Hough Transform Describe straight lines in polor coordinates, xcos ( θ ) + ysin ( θ ) = r . For any ”on” pixel ( x 0 , y 0 ), compute a set of possible ( θ i , r i ). (a) Input (b) Hough (c) Detected lines Ali Abdallah Image Processing Techniques

WRM - Weigthed Resistive Network Very fast pattern processor technology from high energy physics. By means of resistors, input data is propagated inside the circuit. Pattern in the data are then detected following “a maximum likelihood function”. No computations are performed in the WRM. ⊲ GOAL: Adapt the WRM device as linear pattern recognizer for computer vision. Ali Abdallah Image Processing Techniques

WRM One Layer Example (a) One layer of the WRM (b) Voltage diffusion The voltage diffusion inside the circuit is equivalent to a convolution of the binary image Img with a kernel K S = Img ∗ K , · · · (1 2) n · · · 1 2 1 1 2 · · · (1 2) n · · · � Where K = � . Ali Abdallah Image Processing Techniques

Resistive Layer to Pattern Connection of the nodes between different layers can be modelized as a bitmap masks. Below are some typical examples. 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 , , , , 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 The above bitmaps are equivalent to the looked for patterns. Ali Abdallah Image Processing Techniques

WRM Linear Segment Detection S = ( s i , j ) 1 � i � n , 1 � j � m the n × m matrix output of convolution. p k : M 8 , 8 ( ❘ ) → M 8 , 8 ( ❘ ) is the Hadamard product (element-wise product) by the k bitmap 8 × 8 matrix. S ∗ , i � k < i +8 denotes the 8 × 8 matrix from column number i to i + 7 , f k is a vector a sums defined as follows f k i = e T p k ( S ∗ , i � k < i +8 ) e where e = (1 , 1 , 1 , 1 , 1 , 1 , 1 , 1). � d 2 f k � The discrete derivatives are computed ( � � th ) i � � di 2 � Then, arbitrary long segments are constructed from small 8 bits onces using the best fit method. Ali Abdallah Image Processing Techniques

WRM - Example input Figure: Input data example Figure: Second derivatives of the WRM output sums Ali Abdallah Image Processing Techniques

EDUSAFE and Augmented Reality ( Wikipedia ) Augmented reality (AR) is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data. Develop augmented reality prototypes for safety systems − → Interventions in hazardous areas. Realtime processing and object detection is fundamental − → WRM for fast processing. Fast object’s outline detector Fast scoring computation Fast motion detector Ali Abdallah Image Processing Techniques

Convolution Product For real or complex functions: � + ∞ � + ∞ ( f ∗ g )( x ) = f ( x − t ) . g ( t ) dt = f ( t ) . g ( x − t ) dt −∞ −∞ And for series: + ∞ + ∞ � � ( f ∗ g )( n ) = f ( n − m ) . g ( m ) = f ( m ) . g ( n − m ) m = −∞ m = −∞ Discrete convolution in 2d: + ∞ + ∞ � � ( A ∗ B )( x , y ) = A ( x − m , y − m ) . B ( m , m ) m = −∞ m = −∞ Ali Abdallah Image Processing Techniques

Convolutions in Images 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 -1 0 0 0 -1 5 -1 0 0 1 1 1 0 0 -1 1 0 0 0 0 -1 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (a) Sharpen (b) Blur (c) Edges Ali Abdallah Image Processing Techniques

Convolution - Algorithmic Complexity Convolution between patches + ∞ + ∞ � � ( A ∗ B )( x , y ) = A ( x − m , y − m ) . B ( m , m ) m = −∞ m = −∞ for m do for m do for i do for j do · · · end for end for end for end for Algorithmic complexity of O ( m 4 ) − → very expensive. Ali Abdallah Image Processing Techniques

Discrete Fourrier Transform - DFT For { y k } k ∈ ❩ a N -periodic sequence, the DFT of y k , denoted by z = F N y , is the sequence { z k } k ∈ ❩ defined as follows: N − 1 z k = 1 y l e − 2 π i kl � N . N l =0 We have: F N ( y ∗ z ) = N · F N y · F N y ⇔ y ∗ z = N . F − 1 N ( F N y · F N z ) A direct computation of F N y requires O ( N 2 ). Ali Abdallah Image Processing Techniques

Fast Fourrier Transform - DFFT But there exists an algorithm to compute the the FFT due to Cooley and Tukey (1965), which is the DFFT of complexity O ( Nlog 2 N ) To reduce algorithmic complexity the DFFT is used to compute convolutions. O ( N 4 ) − → O ( Nlog 2 N ) Face detection patch based method using convolution. Patches can be eyes, noses, mouths, ... . Ali Abdallah Image Processing Techniques

Thanks for your attention! Ali Abdallah Image Processing Techniques