Detecting Foreground Huynh The Thien thienht@oslab.khu.ac.kr Dept. - PowerPoint PPT Presentation

KYUNG HEE UNIVERSITY Department of Computer Science & Engineering PhD Thesis Dissertation Presentation PhD Thesis Dissertation Presentation Ubiquitous Computing Lab NIC: A Novel Background Estimation Algorithm For Detecting Foreground Huynh The Thien thienht@oslab.khu.ac.kr Dept. of Computer Science & Engineering Kyung Hee University Advisor: Prof. Sungyoung Lee sylee@oslab.khu.ac.kr Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 1

Agenda Agenda  Introduction Background  Motivation  Problem statement  Taxonomy   Related work Related work  Technical review  Limitation   NIC – A background estimation algorithm Overview  Workflow  Formulation  Summary   Experiment & results Dataset  Experiment setup  Results & discussion   Conclusion Contribution & Uniqueness  Future work   Publication  References Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 2

Introduction Background Background  Foreground detection, one of the major issues in the field of image processing and computer vision, aims to detect the changes in a video.  Among current approaches, background subtraction is widely used in video-based realistic systems because of Simple implementation  Real-time processing capability   Background subtraction detects moving objects from the difference between an input frame and a background image (see Figure). Estimate/model the background image  Extract the foreground by gray-scale thresholding   Due to the significant importance, most background subtraction methods contributes on background estimation/modeling algorithms. Background estimation Subtraction operation Figure: Overview of background subtraction technique gray-scale thresholding Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 3

Introduction Motivation Motivation  The performance of computer vision systems, e.g., accuracy of video-based action recognition [1], can be improved based on the foreground detection results due to its preliminary task in these systems  the importance of foreground detection in most of computer vision systems.  A powerful foreground detection system should Estimate the background image/model efficiently.  Adaptively work with various background challenges (baseline, dynamic background, camera jitter, intermittent  object motion, and etc.). Maintain a high-speed processing   Motivate to research the background estimation for foreground detection. Video-based traffic activity recognizer Feature extraction Background Subtraction Activity label estimation operation Model learning Classification Figure: General workflow of a video-based activity recognition. Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 4

Introduction Problem statement Problem statement Problem statement Current approaches are unable to adapt to various background challenges in the real world due to a lack of an efficient background updating scheme while they cannot maintain a high-speed processing [2]. Goal Development of a background estimation algorithm which has an efficient background updating scheme Able to work with various background challenges. • Estimate the background image accurately. • Has a low computation cost in use • Challenge • How to deal with variety of background challenges in the real world ? • How to balance accuracy and computational cost ? Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 5

Introduction Taxonomy Taxonomy The taxonomy of background estimation approaches is drawn as bellows [2]. Background Basic model Running average estimation Histogram over time Statistical model Parametric Gaussian Mixture Model GMM variants Non-parametric Kernel Density Estimation Codebook construction Advanced GMM improvement statistical model Pixel-based adaptive segmenter Visual background extractor Neighbor-based Intensity Correction Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 6

Related work Related work Related work Review some highlight algorithms in group of statistical model Research Description Advantage Limitation KDE : Non-parametric • Model per-pixel background based on No parameter estimation Time-consuming for per-pixel • • Model for Background smoothing the histogram of recent Able to adapt to various background modeling • Subtraction [3] samples by a kernel function background models Huge memory requirement • • Update the background model by first-in first-out manner EGMM : Improved • Model background using Mixture of Slightly improve accuracy Require parameters estimation • • adaptive gaussian mixture Gaussian distribution of foreground detection  a fixed setting when model for background • Update by a recursive equation with Reduce the processing implement in realistic systems • subtraction [4] learning rate and adaptively select time number of Gaussian component ViBE : Vibe: A universal • Update background model with a lifespan Cheap computation  high Low foreground detection • • background subtraction policy to select background pixels fps (frame per second) accuracy algorithm for video randomly. Sensitive to dark background, • sequences [5] • Smooth background consistency by a shadows, and frequent sample propagation scheme background change PBAS : Background • Model background based on recently Adaptive to gradual and Lack of a shadow removal • • segmentation with observed pixels sudden change of scheme. feedback: The Pixel-Based • Update model with pixel-wise learning illumination Many parameters need to be • Adaptive Segmenter [7] parameters in consideration of neighbor set in the algorithm Simp-SOBS : Comparative • Initialize the background image as an Do not require a set of Highly expensive computation • • study of motion detection arbitrary frame in a sequence frame for modeling for updating weights in the methods for video • Update the background image by self- background network surveillance systems [11] organizing map, a simple type of ANN Able to detect shadow • Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 7

Related work Related work As Is – To Be As Is Feature Require N initial input frames for modeling  extract the foreground of (N+1) th frame • Update the background model by a learning rate and foreground result • Memory consumption for background modeling • To Be Feature Do not need initial frames for modeling  allow to • extract the foreground at 2 nd frame NIC directly update the background image over time • Less memory consumption • Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 8

Related work Theoretical comparison Theoretical comparison ViBE [5] NIC Background First frame image Background NIC algorithm for     n initialization   Input frame ln   t t  1 0 background updating    n 1 P t t ( , ) e o 1 Lifespan policy + sample propagation Euclidean color space Background measurement pixel model Updated   background             * P x y , D x y , 1 S x y , 0   i i i Model updating Pixel refining Foreground Input frame extraction Foreground extraction Foreground Neighbor-based intensity updating Foreground scheme Feature Feature First frame being used to initialize the background model Assume first frame as an initial background image • • Update the model over time with a lifespan policy and sample • NIC algorithm operated as a background updating scheme • propagation scheme those are based on random selection. NIC has a pixel refining to discard noise • Foreground extraction using Euclidean color space measurement • Directly update background image with an intensity updating • to decide whether a pixel belongs to the background or rule based on analysis of surrounding neighbor pixels. foreground. Foreground extraction using subtraction operation • Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 9

Related work Limitation Limitation In summary, some major limitations of existing works are  Require a set of initial frames for background modeling.  Need a parameter estimation stage.  Background updating scheme has several shortcomings Inaccurate updating  Expensive computation  Huge memory consumption  NIC algorithm ( N eighbor-based I ntensity C orrection) Steadiness factor Maintain accuracy in the dynamic background challenge. ꟷ Increase the processing speed ꟷ Intensity updating rule Estimate background accurately ꟷ Adapt to various background challenges ꟷ Has a cheap computation ꟷ Thien Huynh-The (UCLab-KHU) NIC Algorithm 14 December 2017 10