Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Dataset 21
Big 3D Data Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category 23 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Big 3D Data 151,128 models 660 categories Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Applications 25
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Slide Credit: Wu et al
2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al
2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al
2.5D Completion & Recognition Training on CAD models and no discriminative tuning! [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. 27 Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition Volumetric representation 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition sofa? ? dresser? Volumetric representation What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
View Planning for Recognition 3D ShapeNets Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al
Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 Mathematically, this is equivalent to evaluate the conditional mutual information: 29 Slide Credit: Wu et al
Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. Our algorithm stands out as the uncertainty of the first view increases 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 object label 1200 2 512 filters of stride 1 5 160 filters of stride 2 13 48 filters of stride 2 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
Back Propagation Fine-tuning 3D ShapeNets 3D CNN 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015
As a 3D Feature Extractor 32 Slide Credit: Wu et al
As a 3D Feature Extractor Mesh Classification & Retrieval 32 Slide Credit: Wu et al
As a 3D Feature Extractor Mesh Classification & Retrieval 2.5D object recognition 32 [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu et al
Recommend
More recommend