3d shapenets a deep representation for volumetric shape
play

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling - PowerPoint PPT Presentation

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling by Wu, Song, Khosla, Yu, Zhang, Tang, Xiao presented by Abhishek Sinha 1 3D Shape Prior 2 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric


  1. Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  2. Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  3. Dataset 21

  4. Big 3D Data Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  5. Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  6. Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category 23 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  7. Big 3D Data 151,128 models 660 categories Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  8. Applications 25

  9. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  10. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  11. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  12. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  13. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  14. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  15. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  16. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  17. 2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al

  18. 2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al

  19. 2.5D Completion & Recognition Training on CAD models and no discriminative tuning! [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. 27 Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  20. View Planning for Recognition 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  21. View Planning for Recognition Volumetric representation 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  22. View Planning for Recognition sofa? ? dresser? Volumetric representation What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  23. View Planning for Recognition sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  24. View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  25. View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  26. View Planning for Recognition Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  27. View Planning for Recognition 3D ShapeNets Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  28. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  29. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  30. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  31. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  32. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  33. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  34. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  35. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  36. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 Mathematically, this is equivalent to evaluate the conditional mutual information: 29 Slide Credit: Wu et al

  37. Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  38. Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. Our algorithm stands out as the uncertainty of the first view increases 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  39. Back Propagation Fine-tuning 3D ShapeNets 4000 object label 1200 2 512 filters of stride 1 5 160 filters of stride 2 13 48 filters of stride 2 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  40. Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  41. Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  42. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  43. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  44. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  45. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  46. Back Propagation Fine-tuning 3D ShapeNets 3D CNN 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  47. As a 3D Feature Extractor 32 Slide Credit: Wu et al

  48. As a 3D Feature Extractor Mesh Classification & Retrieval 32 Slide Credit: Wu et al

  49. As a 3D Feature Extractor Mesh Classification & Retrieval 2.5D object recognition 32 [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu et al

Recommend


More recommend