Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local - PowerPoint PPT Presentation

Automatic Keypoint Detection on 3D Faces Using a Dictionary of Local Shapes Clement Creusot, Nick Pears, Jim Austin Advanced Computer Architecture group Department of Computer science 3DIMPVT, Hangzhou, China, May 2011

Aim What Why How Results Conclusion Keypoints detection (NOT LANDMARKS) Similar to any of 14 learnt features (Dictionary of local shapes) Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 2 / 8

Part of a bigger project What Why Long Term Objective Gap in Research How Results Conclusion Positions + Labels Landmarking Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 3 / 8

Part of a bigger project What Why Long Term Objective Gap in Research How Results Conclusion Keypoint Detection Labeling Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 3 / 8

Gap in Research What Why Most literature: Long Term Objective Gap in Research 3 points max or single-point-of-failure design How Weak features often discarded Results Conclusion Almost no work on combining more than 2 descriptors Little literature that examine multiple descriptors over multiple scales Most people focused on landmarking, without giving the intermediate results on candidate detection (keypoints) Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 4 / 8

Workflow What Why OFFLINE How Results Conclusion Train Meshes × D Descriptor Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Workflow What Why Statistical OFFLINE How Distributions Results Conclusion Landmarks Train Meshes × D Descriptor Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Workflow What Why Statistical OFFLINE How Distributions Results Conclusion Landmarks Train Meshes × D × D × 14 Descriptor Maps Score Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Workflow What Why Statistical Descriptor OFFLINE How Distributions Weights Results Conclusion . . . . . . . . . . . . LDA Landmarks Train Meshes × D × D × 14 Descriptor Maps Score Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Workflow What Why Statistical Descriptor ONLINE How Distributions Weights Results Conclusion Dictionary of local shapes . . . . . . . . . . . . Test Meshes × D × D × 14 Descriptor Maps Score Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Workflow What Why Statistical Descriptor ONLINE How Distributions Weights Results Conclusion Dictionary of local shapes . . . . . . . . . . . . Test Meshes × D × D × 14 × 14 Descriptor Maps Score Maps Mixed Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Workflow What Why Statistical Descriptor ONLINE How Distributions Weights Results Conclusion Dictionary of local shapes . . . . . . . . . . . . Test × 1 Meshes × D × D × 14 × 14 Final Map Keypoints Descriptor Maps Score Maps Mixed Maps Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 5 / 8

Results Sparse selection (max 1%) What Why Reapeatable (same subject registration) How ∼ 75% (at 10 mm) Results Close to human hand-placed landmarks Results average All: ∼ 85% (at 10 mm) Examples average Nose: ∼ 99% (at 10 mm) Conclusion average Eyes: ∼ 90% (at 10 mm) High proportion of the local shapes retreived ∼ 11.88/14 (at 10 mm) 100 100 00 00 01 01 02 02 03 03 95 95 Percentage of Match Percentage of Match 04 04 05 05 06 06 07 07 90 90 08 08 09 09 10 10 11 11 12 12 85 85 13 13 80 80 5 10 15 20 25 30 5 10 15 20 25 30 Matching Acceptance Radius ( mm ) Matching Acceptance Radius ( mm ) Configuration 1 Configuration 2 Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 6 / 8

Examples What Why How Results Results Examples Conclusion 0.224 0.224 0.612 0.612 1.00 1.00 0.265 0.265 0.632 0.632 1.00 1.00 0.270 0.270 0.635 0.635 1.00 1.00 Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 7 / 8

Conclusion Good points: What Detects ”weak” features Why No single-point-of-failure design How Limitations: Results Can be time consuming Conclusion article: 7s, now: 0.5s (8 desc.) Linear combination of scores Future Work: Non linear methods (boosting, kernel methods) Structural matching to deduce correspondences Comparison with a new clustering technique for keypoint detection Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 8 / 8

Conclusion Good points: What Detects ”weak” features Why No single-point-of-failure design How Limitations: Results Can be time consuming Conclusion article: 7s, now: 0.5s (8 desc.) Linear combination of scores Future Work: Non linear methods (boosting, kernel methods) Structural matching to deduce correspondences Comparison with a new clustering technique for keypoint detection Thank You For Listening! http://www.cs.york.ac.uk/~creusot Clement Creusot 3DIMPVT, Hangzhou, China, May 2011, 8 / 8