TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A - PowerPoint PPT Presentation

USNCCM IX, San Francisco, CA, USA, July 22-26 2007 TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares FEUP – Faculdade de Engenharia da Universidade do Porto INEGI – Instituto de Mecânica e Gestão Industrial PORTUGAL

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Contents: o Introduction; o Methodology Used: o Kalman Filter; o Matching: o Mahalanobis Distance; o Optimization Techniques; o Features’ Management Model; o Experimental Results; o Conclusions and future work. | Introduction Introduction | Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 2

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Introduction: o Feature tracking is a complex problem for which computational solutions had evolved considerably in the past decade. o Applications of motion tracking are usual: surveillance, object deformation analysis, traffic monitoring, etc. o Some common difficulties are: o several features to be tracked simultaneously; o appearance/disappearance of features along the image sequence; o long image sequences to be processed; o etc. | Introduction Introduction | Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 3

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL o Existing approaches: o They try to find good compromises between the accuracy of the motion tracking and the involved computational cost. o Examples: o Pfinder (Wren, Azarbayejani, Darell, Pentland,1997) A real-time system for tracking people in order to interpret their behavior. Expects only one user in the image scene and that the scene is quasi-static; o Bayesian networks simplified by gradually discarding the influence of the past information on the current decisions. o Tracking with Kalman Filter is a widespread technique for object tracking; although other filters have recently become more usual, they have also revealed some problems too. | Introduction Introduction | Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 4

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Methodology adopted: o Kalman Filter is used to estimate the features’ positions along the image sequence; o For the matching (data association), between measures (real features) and filter’s estimates, we use Optimization of the global correspondence based on Mahalanobis Distance; o To deal with the problem of appearance, occlusion and disappearance of the tracked features, we employ a Features’ Management model. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 5

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Kalman Filter: o Kalman Filter is an optimal recursive Bayesian stochastic method, but assumes Gaussian posterior density functions at every time step; o Erroneous estimations, for instances in problems involving non-linear motion, can be corrected overcome by using adequate approaches in the matching step. o In this work: o the system state is composed by the positions, velocities and accelerations of the tracked features (points); o new measurements are incorporated in the system model whenever a new image frame is evaluated. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 6

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Matching: o For each feature estimated, there may exist, at most, one new measurement to correct its estimated position. o With Kalman’s usual approach, the predicted search area for each tracked feature is given by an ellipse (whose area will decrease as convergence is obtained and vice-versa). o Some problems: o there may not exist any real feature in the search area or there might be several instead; o even if there is only one correspondence for each feature, there is no guarantee that the best set of correspondences is achieved. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 7

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Matching: o We use optimization techniques to obtain the best set of correspondences between predictions and measurements; o To establish the best global set of correspondences we use the Simplex method; o The cost of each correspondence is given by the Mahalanobis Distance. o Simplex Method: o An iterative algebraic procedure used to determine at least one optimal solution for each assignment problem. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 8

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Matching: o Mahalanobis Distance: o The distance between two features is normalized by its statistical variations; o Its values are inversely proportional to the quality of the prediction/measurement correspondence; o To optimize the global correspondences, we minimize the cost function based on the Mahalanobis Distance. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 9

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Matching: o Occlusion/Appearance: o Assignment restriction (1 to 1) not satisfied – problem solved with addition of fictitious variables: o Features matched with fictitious variables are considered unmatched; o Unmatched tracked feature – it is assumed that the feature has been occluded, but the tracking process is maintained by including its predicted position in the measurement vector although with higher uncertainty; o Unmatched measurement – we consider it as a new feature and initialize its tracking process. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 10

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Management Model: o When a feature disappeared of the scene: Is it just occluded? It was removed definitively? Should we keep its tracking? o This decision is of greater importance if many features are being tracked, if the image sequence is long, if the tracking is in real-time, etc; o We use a management model in which a confidence value is associated to each feature: o In each frame, if a feature is visible then its confidence value is increased, else it is decreased; o If a minimum value of the confidence value is reached, then is considered that the feature has definitively disappeared and its tracking will cease (if it reappears, its tracking will be initialized); o In this work, the confidence values are integers between 0 and 5, and initialized as 3. | Introduction | Methodology Methodology | Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 11

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Experimental Results: o Using synthetic data: o Blobs A, B with horizontal translation and C, D with rotation: A C B D Prediction Uncertainty Area Measurement Correspondence Results | Introduction | Methodology | Results Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 12

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Experimental Results: o Using synthetic data: o Continuation ... Blobs C, D invert their rotation direction: A C ... B D Prediction Uncertainty Area Measurement Correspondence Results | Introduction | Methodology | Results Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 13

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Experimental Results: o Using synthetic data: o Management of the tracked features - blobs (dis)appear randomly: A B C D E Confidence Values: Prediction Uncertainty Area Measurement Correspondence Result | Introduction | Methodology | Results Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 14

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Experimental Results: o Using real data: o Tracking 5 blobs in human gait analysis: Prediction Uncertainty Area Measurement Correspondence Result | Introduction | Methodology | Results Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 15

TRACKING FEATURES WITH KALMAN FILTERING, MAHALANOBIS DISTANCE AND A MANAGEMENT MODEL Experimental Results: o Using real data: o Tracking mice in a lab environment during 547 frames: (with very significant changes in the direction of the motion) | Introduction | Methodology | Results Results | Conclusions| Future Work | Raquel R. Pinho, Miguel V. Correia, João Manuel R. S. Tavares 16