Tracking using Goal CONDENSATION: Model-based visual tracking in - PowerPoint PPT Presentation

Tracking using Goal CONDENSATION: • Model-based visual tracking in dense Conditional Density clutter at near video frame rates Propagation M. Isard and A. Blake, CONDENSATION – Conditional density propagation for visual tracking, Int. J. Computer Vision 29 (1), 1998, pp. 4-28. Example of Approach CONDENSATION Algorithm • Probabilistic framework for tracking objects such as curves in clutter using an iterative sampling algorithm • Model motion and shape of target • Top-down approach • Simulation instead of analytic solution 1

Probabilistic Framework Notation X State vector, e.g., curve’s position and orientation • Object dynamics form a temporal Markov ( ) ( ) Z chain Measurement vector, e.g., image edge locations Χ − = p x p x x | | t t t t − 1 1 p ( X ) Prior probability of state vector; summarizes prior • Observations, z t , are independent (mutually domain knowledge, e.g., by independent measurements and w.r.t process) ( ) ( ) p ( Z ) Probability of measuring Z ; fixed for any given image − = ∏ t p Z x X p x X 1 p z x , | | ( | ) − − − = t t t t t i i i 1 1 1 1 p ( Z | X ) Probability of measuring Z given that the state is • Use Bayes’ rule X ; compares image to expectation based on state p ( X | Z ) Probability of X given that measurement Z has occurred; called state posterior Tracking as Estimation Factored Sampling • Compute state posterior, p ( X | Z ), and select next • Generate a set of samples that approximates the posterior p ( X | Z ) state to be the one that maximizes this (Maximum a Posteriori (MAP) estimate) s = s s N ( 1 ) ( ) • Sample set generated from { ,..., } • Measurements are complex and noisy, so p ( X ); each sample has a weight posterior cannot be evaluated in closed form (“probability”) • Particle filter (iterative sampling) idea: Stochastically approximate the state posterior p s i ( ) ( ) π = z with a set of N weighted particles, ( s , π ), where s i N � p s j is a sample state and π is its weight ( ) ( ) z j = • Use Bayes’ rule to compute p ( X | Z ) 1 p z x = p z x ( ) ( | ) 2

Estimating Target State Factored Sampling N =15 X ����� ����� ������������� ����������������� • CONDENSATION for one image ������������� ������������� Bayes’ Rule CONDENSATION Algorithm This is what you may know a priori, or what This is what you can you can predict 1. Select : Randomly select N particles from {s t-1(n) } evaluate based on weights π t-1(n) ; same particle may be picked multiple times ( factored sampling ) Z X X 2. Predict : Move particles according to p p ( | ) ( ) X Z = deterministic dynamics ( drift ), then perturb p ( | ) Z individually ( diffuse ) p ( ) 3. Measure : Get a likelihood for each new sample by comparing it with the image’s local appearance, i.e., based on p (z t |x t ); then update This is what you want. Knowing p ( X | Z ) will tell us what is the weight accordingly to obtain {(s t(n) , π t(n) )} This is a constant for a most likely state X . given image 3

s n − π n ( ) ( ) 1 , Notes on Updating k k − 1 Posterior at time k -1 • Enforcing plausibility: Particles that drift represent impossible configurations are discarded diffuse Predicted state • Diffusion modeled with a Gaussian at time k ( n s ) • Likelihood function: Convert “goodness of observation k prediction” score to pseudo-probability density measure – More markings closer to predicted markings → Posterior higher likelihood at time k ( , n π n s ) ( ) k k State Posterior Animation State Posterior ����� ����� ������������� 4

Evaluating p ( Z | X ) Object Motion Model M • For video tracking we need a way to � = + φ φ p z x qp z clutter p z x m p ( | ) ( | ) ( | , ) ( ) propagate probability densities, so we need m m = 1 a “motion model” such as where φ m = {true measurement is z m } X t +1 = A X t + B W t where W is a noise term for m = 1,…, M , and q = 1 - Σ m p ( φ m ) and A and B are state transition matrices that can be learned from training sequences is the probability that the target is • The state, X , of an object, e.g., a B-spline not visible curve, can be represented as a point in a − 2 − < δ 6D state space of possible 2D affine x z if x z φ = m m m m transformations of the object m ρ otherwise Dancing Example Hand Example 5

Pointing Hand Example Glasses Example • 6D state space of affine transformations of a spline curve • Edge detector applied along normals to the spline • Autoregressive motion model 3D Model-based Example Minerva • 3D state space: image position + angle • Museum tour guide robot that used • Polyhedral model of object CONDENSATION to track its position in the museum Desired Location Exhibit 6

Advantages of Particle Filtering • Nonlinear dynamics, measurement model easily incorporated • Copes with lots of false positives • Multi-modal posterior okay (unlike Kalman filter) • Multiple samples provides multiple hypotheses • Fast and simple to implement 7