CS 4495 Computer Vision Tracking 3: Follow the pixels Aaron Bobick - PowerPoint PPT Presentation

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick CS 4495 Computer Vision Tracking 3: Follow the pixels… Aaron Bobick School of Interactive Computing

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Administrivia • PS6 out Thurs Nov 14, due Nov 24 th • EXAM: Tues before Thanksgiving, Nov 26 th (not 21 st !!) • Covers concepts and basics • Example questions will be posted on calendar by November 19 th • PS7 out Tue Nov 26 th , due technically Friday, December 5. • Extension available until Sunday, Dec 7, a day that will live in infamy, 11:55pm. • Problem set resubmission policy: • Full questions only • You get 50% credit to replace whatever you got last time on that question. • Must be submitted by: SUNDAY DECEMBER 7, 11:55pm • No prior notice needed. • Grades posted as available.

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Tracking as induction • Base case: • Assume we have initial prior that predicts state in absence of any evidence: P ( X 0 ) • At the first frame , correct this given the value of Y 0 = y 0 • Given corrected estimate for frame t : • Predict for frame t +1 • Correct for frame t +1 predict correct

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Kalman: Propagation of Gaussian densities

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Propagation of general densities

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Particle Filters: Basic Idea ( x ) p x t set of n (weighted) particles X t Density is represented by both where the particles are and their weight .

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Graphical steps particle filtering

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Bayes Filters: Framework • Given: • Stream of observations z and action data u: =  { , , , } data u z u z − 1 2 1 t t t • Sensor model P(z t |x t ). • Action model P(x t+1 |u t ,x t ) . • Prior probability of the system state P(x). • Wanted: • Estimate of the state X of a dynamical system. • The posterior of the state is also called Belief : =  ( ) ( | , , , ) Bel x P x u z u z − 1 2 1 t t t t

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Bayes Rule reminder Prior before measurement ( | ) ( ) p z x p x = ( | ) p x z ( ) p z = η ( | ) ( ) p z x p x

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Particle Filter Algorithm (Sequential Importance Resampling) = < > j j { , , , } S x w u z 1. Algorithm particle_filter − − − 1 1 1 t t t t t = ∅ η = , 0 2. S t = 3. For Resample (generate i new samples)  1 i n 4. Sample index j(i) from the discrete distribution given by w t-1 Sample from using and Control 5. ( ) i ( | , ) j i p x x u u x x − − 1 t t t t 1 t t w = 6. Compute importance weight (or reweight) i i ( | ) p z x t t t η = η + i 7. Update normalization factor w t = ∪ < > 8. Insert i i { , } S S x w t t t t = 9. For  1 i n w = η 10. i i Normalize weights / w t t

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Smithsonian Museum of American History…

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick 25

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick 26

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick 27

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick 28

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick How about simple vision….

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Using Ceiling Maps for Localization Dellaert, et al. 1997

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Vision-based Localization P(z|x) z h(x)

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Under a Light P(z|x) : Measurement z:

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Next to a Light P(z|x) : Measurement z:

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Elsewhere P(z|x) : Measurement z:

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Global Localization Using Vision

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Odometry Only

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Using Vision

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick A detail: Resampling method can matter • Given : Set S of weighted samples. • Wanted : Random sample, where the probability of drawing x i is given by w i . • Typically done n times with replacement to generate new sample set S’ . • Or even not done except when needed…too many low weight particles.

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Resampling w 1 w 1 w n w n w 2 w 2 W n-1 W n-1 w 3 w 3 • Stochastic universal sampling • Systematic resampling • Roulette wheel • Linear time complexity • Binary search, n log n • Easy to implement, low variance

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Resampling Algorithm 1. Algorithm systematic_resampling ( S,n ): = ∅ = 1 2. ' , S c w 1 =  Generate cdf 3. For 2 i n = + i 4. c c w − 1 i i − = 1 5. Initialize threshold 1 ~ [0, ], 1 u U n i =  1 j n Draw samples … 6. For u > c 7. While ( ) Skip until next threshold reached j i = i + 1 i 8. { } − 1 = ∪ < > i ' ' , S S x n 9. Insert = + − 1 u u n 10. Increment threshold + 1 j j (Also called stochastic universal sampling) 11. Return S’

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick PF: Practical Considerations • If dealing with highly peaked observations • Add noise to observation and prediction models • Better proposal distributions: e.g., perform Kalman filter step to determine proposal • Overestimating noise often reduces number of required samples • Recover from failure by selectively adding samples from observations • Recover from failure by uniformly adding some samples • Can Resample only when necessary (efficiency of representation measured by variance of weights)

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick To do real tracking… State ( | ) ( ) p z x p x = ( | ) p x z ( ) p z ( | , ) p x x u − 1 t t t = η ( | ) ( ) p z x p x State dynamics Sensor model • x is the “state”. But of what? The object? Some representation of the object? • z is the “measurement”. But what measurement? And how does it relate to the state? • Where do you get your dynamics from?

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick The source…

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Particle filter tracking - state • The “object” to be tracked here is a hand initialized contour . Could have been the image pixels. Which is better? • Its state is its affine deformation. How many parameters? • Each particle represents those six parameters. Picture of the states represented by The mean state 39 the top weighted particles [Isard 1998]

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick More complex state • Tracking of a hand movement using an edge detector • State is translation and rotation of hand plus angle of each finger; 12 DOF

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick Particle filter tracking - measurement • Suppose x is a hand initialized contour. • What is z? • Gaussian in Distance to nearest high- contrast feature summed over the contour. [Isard 1998]

Tracking 3: Real tracking CS 4495 Computer Vision – A. Bobick More tracking contours • Head tracking with contour models (Zhihong et al. 2002) • How did it do occlusion? • With velocity? • Without velocity? • How did you get “dynamics”?