Tracking Articulated Objects Alexander (Sasha) Lambert CS7495 Fall - PowerPoint PPT Presentation

Tracking Articulated Objects Alexander (Sasha) Lambert CS7495 – Fall 2014

Tracking From Depth • Infra-red point-clouds (structured light) • Affordable sensors (Primesense) • Large body of work on people-tracking ฀ Complex tracking methods ฀ ML-based approach (ex. Decision Kinect One sensor (Microsoft) forests) http://www.blogcdn.com/ http://www.creativeapplications.net/ 2 2

Tracking – Robotics • Move towards cheaper robot manipulators ฀ Problem: poor encoders, flexible joints • Would like a generalized approach • Applications: ฀ Human/Robot, Robot/Robot interaction (ex. Task collaboration) ฀ Self-calibration (Manipulators) ฀ Interacting with every day objects (drawers, doors, can-openers...) • Realtime, robust to occlusion • Useability and generality 3 3

Tracking • DART (RSS '2014) - Schmidt, Newcombe, Fox ฀ Generalized framework ฀ Realtime, GPU-optimized ฀ Requires only kinematic model & part geometries 4 4

HEADER 5 5

HEADER 6 6

Kalman Filter 7 7

Extended Kalman Filter • Non-linear functions (dynamics, measurement) ฀ Idea: local linearization ฀ Preserves Gaussian shape 8 8

Extended Kalman Filter Predictor Correcto r Predictor Correcto r 9 9

DART Tracker Experiments → const. dyn. Model prior(t) = posterior(t-1) 10 10

DART Tracker • Develop a measurement model • θ – pose parameters (state) • D – Depth map “Tracking Energy” 11 11

Point Cloud Registration • Correspondence between model and data • ICP – Iterative Closest Point ฀ Data association: many variants (CP, P2PL, KNN) http://taylorwang.files.wordpress.com/ 12 12

Point Cloud Registration Non-Linear Optimization (Fitzgibbon '01) • Gradient-based iteration • Levenberg-Marquardt algorithm • Signed Distance Function ( SDF ) ฀ Can be pre-computed 13 13

Objective Function • Schmidt et al. → use similar SDF mapping for articulated objects in 3D • θ – pose parameters (state) • x – 3d position (camera frame) • u – pixel index 14 14

Objective Function • Composite SDF for articulated body • k – frame index • T – camera/frame transform 15 15

Optimization 1.Taylor-series expansion (Gauss-Newton approximation) 2.Jacobian computation 3. Iteration Update 16 16

Symmetric Formulation Dealing with Occlusions • Use 'Free Space' to constrain objective function • Augment probability with model- point prediction • SDF of observation constricting model prediction 17 17

HEADER Robotics/courses/CS7442-Computer_Vision/CS7495-presentation/slides/video/optim.mp4#Play Video 18 18

Results – Hand tracking • Parallelized computation (GPU) • Hand-pose dataset (Qian et al. '12) –Models with 26 d.o.f –Frequent, rapid occlusions • Qian et al. , Oikonomidis et. al ฀ ICP + PSO Average distance b/w prediction & ground truth (mm) 19 19

Results – Body tracking • EVAL dataset (Ganaparthi et al. '12) –Models with 48 d.o.f –% of joints within 10cm • Ganaparthi et al. –ICP + free-space • Ye & Yang ('14) –GMM –Shape estimation 20 20

Results - Servoing • Grasping with robot manipulator • Provide visual feedback –Updated state to controller • Improved accuracy –3/10 vs 10/10 21 21

Results box/GTRobotics/courses/CS7442-Computer_Vision/CS7495-presentation/slides/video/results.mp4 22 22

HEADER THANK YOU! 23 23