3dmatch
play

3DMatch Learning Local Geometric Descriptors from RGB-D - PowerPoint PPT Presentation

Sep 15, 2023 •150 likes •349 views

3DMatch Learning Local Geometric Descriptors from RGB-D Reconstructions Andy Zeng, Shuran Song, Matthias Niener, Matthew Fisher, Jianxiong Xiao, Thomas Funkhouser Matching Features in 3D Data with Local 3D Descriptors match descriptor

  1. 3DMatch Learning Local Geometric Descriptors from RGB-D Reconstructions Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, Thomas Funkhouser

  2. Matching Features in 3D Data with Local 3D Descriptors match descriptor descriptor 0.58 0.21 0.92 0.67 0.04 0.53 0.58 0.21 0.92 0.67 0.04 0.53

  3. Matching Features in 3D Data with Local 3D Descriptors commodity depth cameras Applications of Matching 3D Features: - Scan registration and loop closures for 3D reconstruction - Model registration for pose estimation - 3D mesh correspondence

  4. Matching Local Features in 3D Scans is Hard ? noisy Partial Surfaces and Occlusion Sensor Limitations dense sparse Point Density Changes Other Anomalies

  5. Matching Local Features in 3D Scans is Hard Previous local 3D descriptors only address part of the problem. Partial Surfaces and Occlusion FPFH [Rusu et al.] Sensor Limitations Spin-Images [Johnson et al.] Point Density Changes SHOT [Salti et al.] Other Anomalies Goal: train a data-driven local 3D descriptor that learns from example correspondences on real 3D scans.

  6. 3DMatch: Data-Driven Local 3D Descriptor match! 0.58 0.21 0.92 0.67 0.04 0.53 … 0.58 0.21 0.92 0.67 0.04 0.53 … 3DMatch 3DMatch 3D ConvNet 3D ConvNet TDF TDF 3D patch

  7. 3DMatch: Data-Driven Local 3D Descriptor matches non-matches

  8. Training Data? Problem: how do we get the training data? Manually label correspondences? Extremely Challenging Partial Surfaces and Occlusion - time consuming Sensor Limitations Point Density Changes - prone to errors Other Anomalies Is there a way to obtain training data automatically?

  9. Self-Supervised Learning from RGB-D Reconstructions frame #7459 frame #4903 long-range correspondence

  10. Self-Supervised Learning from RGB-D Reconstructions 50+ RGB-D reconstructions > 8 million correspondences 3 applications

  11. Application #1: 3D Reconstruction

  12. Application #1: 3D Reconstruction 3DMatch + RANSAC geometric registration

  13. Application #1: 3D Reconstruction Rusu et al. 3DMatch + RANSAC

  14. Application #1: 3D Reconstruction generalization requirement: low

  15. Application #2: 6D Object Pose Estimation full partial smaller scale

  16. Application #3: Mesh Correspondences

  17. Application #3: Mesh Correspondences

  18. Takeaways 3DMatch: Data-Driven Local 3D Descriptor RGB-D Reconstructions as Training Data Code & Benchmarks: http://3dmatch.cs.princeton.edu

  19. 3DMatch Learning Local Geometric Descriptors from RGB-D Reconstructions Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, Thomas Funkhouser Code & Benchmarks: http://3dmatch.cs.princeton.edu

Recommend

More recommend