3d reconstruction reconstruction method
play

3D RECONSTRUCTION Reconstruction method Reconstruction from images - PowerPoint PPT Presentation

3D RECONSTRUCTION Reconstruction method Reconstruction from images Reconstruction from video Using Kinect Raw Depth Image Infrared laser projector Monochrome CMOS sensor Demo Kinect Raw data Real-time Reconstruction Pipeline Measurement


  1. 3D RECONSTRUCTION

  2. Reconstruction method Reconstruction from images Reconstruction from video

  3. Using Kinect Raw Depth Image Infrared laser projector Monochrome CMOS sensor

  4. Demo Kinect Raw data

  5. Real-time Reconstruction

  6. Pipeline Measurement Noise Surface Compute Update Raw Depth Reduction Pose Prediction Surface Vertex Reconstruction Image Bilateral Estimation ICP and Normal TSDF Ray-cast Filtering Map r k R k T gk V k, N k S k V k, N k Input: 20 frames * 640 * 480 * 12 = 614.8 MB/s

  7. Bilateral Filtering Demo

  8. Pipeline Measurement Noise Surface Compute Update Raw Depth Reduction Pose Prediction Surface Vertex Reconstruction Image Bilateral Estimation ICP and Normal TSDF Ray-cast Filtering Map r k R k T gk V k, N k S k V k, N k

  9. ICP 3D shape alignment SVD

  10. ICP 3D shape alignment Demo

  11. Pipeline Measurement Noise Surface Compute Update Raw Depth Reduction Pose Prediction Surface Vertex Reconstruction Image Bilateral Estimation ICP and Normal TSDF Ray-cast Filtering Map r k R k T gk V k, N k S k V k, N k

  12. TSDF Signed Distance Function The value in the cube corresponds to the signed distance to the closest zero crossing( surface).

  13. TSDF Signed Distance Function Truncated Signed Distance Function

  14. TSDF Signed Distance Function Truncated Signed Distance Function Integrate the cubes from different position.

  15. TSDF -1 Depth Map from Kinect

  16. TSDF -1 -0.2 Depth Map from Kinect

  17. TSDF 0.05 -1 -0.2 Depth Map from Kinect

  18. TSDF 0.05 -1 0.2 -0.2 Depth Map from Kinect

  19. TSDF 1 0.05 -1 0.2 -0.2 Depth Map from Kinect

  20. TSDF 1 1 0.05 -1 0.2 -0.2 Depth Map from Kinect

  21. TSDF 1 1 0.05 -1 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 We have depth maps from different camera positions, -1 -1 -0.5 0.05 1 1 how can we integrate them together ? -1 -1 -0.5 0.1 1 1 Integration? or update? Weighted? or add up? -0.8 -1 -0.05 0.3 1 1 What makes integration -1 -0.5 -0.03 0.5 1 1 possible ?

  22. TSDF 1 1 0.05 -1 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 We have depth maps from different camera positions, -1 -1 -0.5 0.05 1 1 how can we integrate them together ? -1 -1 -0.5 0.1 1 1 Integration? or update? Weighted? or add up? -0.8 -1 -0.05 0.3 1 1 What makes integration -1 -0.5 -0.03 0.5 1 1 possible ?

  23. TSDF 1 1 0.05 -1 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 We have depth maps from different camera positions, -1 -1 -0.5 0.05 1 1 how can we integrate them together ? -1 -1 -0.5 0.1 1 1 Integration? or update? Weighted? or add up? -0.8 -1 -0.05 0.3 1 1 What makes integration -1 -0.5 -0.03 0.5 1 1 possible ?

  24. TSDF 1 1 0.05 -1 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 To get the surface behind the surface. The camera is -1 -1 -0.5 0.05 1 1 moving! -1 -1 -0.5 0.1 1 1 Only part of distance data is needed, so we can -0.8 -1 truncate the distance. -0.05 0.3 1 1 -1 -0.5 -0.03 0.5 1 1

  25. TSDF 1 time update ! 1 1 0.05 -0.3 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 To get the surface behind the surface. The camera is -1 -1 -0.5 0.05 1 1 moving! -1 -1 -0.5 0.1 1 1 Only part of distance data is needed to represent the -0.8 -1 object, so we can truncate -0.05 0.3 1 1 the distance. -1 -0.5 -0.03 0.5 1 1

  26. TSDF 2 times update ! 1 1 0.05 0 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 To get the surface behind the surface. The camera is -1 -1 -0.5 0.05 1 1 moving! -1 -1 -0.5 0.1 1 1 Only part of distance data is needed to represent the -0.8 -1 object, so we can truncate -0.05 0.3 1 1 the distance. -1 -0.5 -0.03 0.5 1 1

  27. TSDF 3 times update ! 1 1 0.05 0.3 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 To get the surface behind the surface. The camera is -1 -1 -0.5 0.05 1 1 moving! -1 -1 -0.5 0.1 1 1 Only part of distance data is needed to represent the -0.8 -1 object, so we can truncate -0.05 0.3 1 1 the distance. -1 -0.5 -0.03 0.5 1 1

  28. Pipeline Measurement Noise Surface Compute Update Raw Depth Reduction Pose Prediction Surface Vertex Reconstruction Image Bilateral Estimation ICP and Normal TSDF Ray-cast Filtering Map r k R k T gk V k, N k S k V k, N k

  29. RAY CASTING Cast only, no chasing. Transfer the TSDF cube in to some thing the computer can understand, Vertex fusion. Take a photo using X-ray.

  30. RAY CASTING 1 1 0.05 -1 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 -1 -1 -0.5 0.05 1 1 -1 -1 -0.5 0.1 1 1 Detect the sign change. -0.8 -1 -0.05 0.3 1 1 Two scales search -1 -0.5 -0.03 0.5 1 1 Linear regression

  31. RAY CASTING 1 1 0.05 -1 0.5 -0.2 -1 -0.8 -0.1 0.3 1 1 -1 -1 -0.5 0.05 1 1 1 -1 -1 -0.5 0.1 1 Detect the sign change. -0.8 -1 -0.05 0.3 1 1 Two scales search -1 -0.5 -0.03 0.5 1 1 Linear regression Normal Vectors

  32. Real-time Reconstruction Demo

  33. Reference [1] KinectFusion: Real-Time Dense Surface Mapping and Tracking. Microsoft Research [2] B. Curless and M. Levoy. A volumetric method for building complex models from range images. [3] M. Harris, S. Sengupta, and J. D. Owens. Parallel prefix sum (scan) with CUDA. In H. Nguyen, editor, GPU Gems 3, chapter 39, pages 851 – 876. Addison Wesley, August 2007. 3.5 [4] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In Proceedings of the ICCV, 1998. [5] C. Rasch and T. Satzger. Remarks on the O(N) implementation of the fast marching method. [6] Y. Chen and G. Medioni. Object modeling by registration of multiple range images. Image and Vision Computing (IVC), 10(3):145 – 155,1992 [7] Kok-Lim Low Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration

Recommend


More recommend