NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT - PowerPoint PPT Presentation

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Giorgio Gemignani Lucia Maddalena Alfredo Petrosino Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION • Outline: – Moving and Stopped Object detection. – Dual Background approach and experimental results. – Alghoritm parallelization on GPU. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION • Stopped Object : temporally static image regions indicating objects that do not constitute the original background, but were brought into the scene at a subsequent time. • Examples : abandoned luggage or illegally parked vehicles. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Dual Background Approach Construct 2 separate models : • Long term model B L for scene background. • Short term model B s for static background elements F L F S Compare each sequence frame I t with these models and calculate 2 foreground binary masks : • Long foreground mask F L containing stopped and moving objects. • Short foreground mask F s containing moving objects. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Dual Background Approach For each pixel an evidence score is computed by applying the set of hypothesis on the foreground masks:  τ + ∆ ∧ L S min( , E ( x ) t ) if ( F ( x ) ! F ( x ))  − t 1 = E ( x )  t  − L ∨ S max( 0 , E ( x ) k ) if ( F ( x ) F ( x ))  − t 1 τ stationary threshold : minimum number of consecutive frames after which a pixel is classified as static. k decay factor : determine how fast system should recognize that a stopped pixel has moved again. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Neural Self Organizing Background Model • The background model constructed and maintained in SOBS algorithm [ Maddalena & Petrosino, TIP’08 ], here adopted for both the long-term and the short-term backgrounds, is based on a self organizing neural network organized as a 2-D flat grid of neurons . Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Neural Self Organizing Backgroung Model 1. For each pixel x , build a neuronal map consisting of n x n weight b t i ( x ). ( ) ( ) = = i 2 b x I x , i 1, , n  0 0 2. At each subsequent time instant t, every pixel x of I t is compared to current pixel weight vectors (b t 1 ( x ), …, b t L ( x )) to determine the weight vector b t BM ( x ) that best matches it according to a metric d() : ( ) ( ) ( ) ( ) ( ) ( ) = BM i d b x , I x min d b x , I x t t t t 2 = i 1, , n  ( ) ( ) ( ) ( ) ( ) ( ) ) ( ) = = : I x h , s , v , I x h , s , v ( ( ) HSV colour space = − d I x , I x v s cos(h ), v s sin(h ), v v s cos(h ), v s sin(h ), v i i i i j j j j i j i i i i i i i j j j j j j j Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Neural Self Organizing Backgroung Model 3. Weight vectors are updated in a neighborhood of best matching neuron. Updating the model Bt in a neighborhood N z : ( ) ( ( ) ) ( ) ( ) ( ) = − + ∀ ∈ B y 1 α y , z B y α y , z I y , y N (1) − t t t 1 t t z β [ ] [ ] ( ) ( ) ( ) = ∈ ∈ γ t , β 0,1 s.t. α y , z 0 , 1 = γ − α y , z G y z { ( ) } t t t − t t max G y z 4. For the purpose of the double background approach to stopped object detection: •B t L is updated according to (1) in ( ) ( ) ( ) < ε BM d b x , I x a selective way , only if t t • B t S is updated according to (1) in a γ > > γ S L non selective way with t t Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Results PVEasy Computational compexity both in space and time is Pets2006 O(n 2 x M x N) where n 2 is the number of weight vectors used to model each pixel and N x M is the image size. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Time Processing Image Size Processing Time Frame Rate 720 x 480 431.68 ms 2.3 fps 960 x 720 862.94 ms 1.15 fps 1200 x 960 1430.30 ms 0.69 fps Well…..but Real-time video surveillance applications require a frame rate of 24 fps. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Parallel Implementation • M x N Pixels • th x x th y number of threads per block • G 1 ( G S ,G L ) grid of Blocks: • G S short-term background model . • G L long-term background model. • G 2 generate the evidence Image Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Parallel Performance Evaluation • Serial Implementation: Intel Core i7 CPU at 3.3 GHz. • Parallel implementation: Tesla C1060 ( 30 multiprocessors ). • Measurement of processing time for on-line phase . Image Size CPU Time GPU Time Speed Frame Rate Up 720 x 480 431.68 ms 20.55 ms 21 x 49 fps 960 x 720 862.94 ms 40.96 ms 21 x 25 fps 1200 x 960 1430.30 ms 65.41 ms 22 x 16 fps AB-Easy Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION • OpenMp parallelization with N cores. Thread 0 Thread i Thread N Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Test on Intel Core i7 CPU N. of Processing Speed Up Efficiency Frame at 3.33 GHz Threads Time Rate 2 216.50 ms 1.99 x 0.99 4.61 fps Image Size: 720 x 480 4 115.20 ms 3.74 x 0.93 8.68 fps Gpu processing time: 20.55 ms 6 96.70 ms 4.46 x 0.74 10.34 fps Seq. processing time: 431.68 ms 8 76.42 ms 5.64 x 0.70 13.08 fps Image Size: 960 x 720 2 436.76 ms 1.97 x 0.98 2.28 fps 4 224.64 ms 3.84 x 0.96 4.45 fps Gpu processing time: 40.96 ms 6 190.26 ms 4.53 x 0.75 5.25 fps Seq. processing time: 862.94 ms 8 152.08 ms 5.67 x 0.70 6.57 fps Image Size: 1200 x 960 2 719.29 ms 1.98 x 0.99 1.39 fps 4 373.58 ms 3.82 x 0.95 2.67 fps Gpu processing time: 65.41 ms 6 317.86 ms 4.49 x 0.74 3.14 fps Seq. processing time: 1430.3 ms 8 254.25 ms 5.62 x 0.70 3.9 fps Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION Parallel computing is an excellent solution for scientific research and unconventional hpc platforms as GPUs, due to their computing power and low cost represent an interesting alternative to conventional parallel shared memory architectures. Thank you……….. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope

NEURAL DUAL BACKGROUND MODELING FOR REAL-TIME STOPPED OBJECT DETECTION References • R.T. Collins et al., “A system for video surveillance and monitoring,” The Robotics Institute, Carnegie Mellon University, Tech. Rep. CMU-RI-TR-00-12, 2000. • J.M. Ferryman (Ed.): Proceedings of the 9th IEEE International Workshop on PETS, New York, June 18, 2006. • J.M. Ferryman (Ed.): Proceedings of the 10th IEEE International Workshop on PETS, Rio de Janeiro, Brazil, October 14, 2007. • E. Herrero-Jaraba et al., Detected motion classification with a double-background and a Neighborhood-based difference, Patt. Recogn. Lett. 24, 2079–2092, 2003. • L. Maddalena and A. Petrosino, A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications, IEEE Transactions on Image Processing, vol. 17, no. 7, pp. July, 2008. • NVIDIA, CUDA guide, http://www.nvidia.com/object/cuda home new.html • F. Porikli,Y. Ivanov, T. Haga, Robust Abandoned Object Detection Using Dual Foregrounds, EURASIP Journal on Advances in Signal Processing, 2008. • Proc. of Fourth IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS 2007), IEEE Computer Society, 2007. Giorgio Gemignani PhD Student University of Milan associated with University of Naples Parthenope