Volumetric and Multi-View CNNs for Object Classification on 3D Data - PowerPoint PPT Presentation

Volumetric and Multi-View CNNs for Object Classification on 3D Data Charles R. Qi*, Hao Su*, Matthias Nießner, Angela Dai, MengyuanYan, Leonidas J.Guibas

Rich Applications of 3D Augmented Robot Reality Perception

3D Representations for Generic Object Classification Volumetric Multi-Views 3DShapeNets by Z. Wu et MVCNN by H. Su et al. al. CVPR 15 ICCV 15 VoxNet by D. Maturana et DeepPano by B. Shi et al. al. IEEE/RSJ 15 IEEE/SPL 15

Volumetric CNNs Revisited Volumetric CNNs 3DShapeNets by Z. Wu et al. CVPR 15

Multi-View CNNs Revisited Multi-View CNNs MVCNN by H. Su et al. ICCV 15

Shape Classification Results Revisited 95 90.1% 90 85 77.3% 80 75 70 3DShapeNets MVCNN Wu et al. Su et al.

Shape Classification Results Revisited 95 90.1% 90 85 77.3% 80 Big gap between 75 volumetric and multi-view 70 based methods Why? 3DShapeNets MVCNN Wu et al. Su et al.

Cause 1: Architecture and Engineering LeNet, 1998 AlexNet, 2012

Cause 1: Architecture and Engineering LeNet, 1998 3DShapeNets, 2015 AlexNet, 2012

Cause 2: Resolution Multi-View CNNs MVCNN Su et al. 224x224 Images

Cause 2: Resolution Multi-View CNNs Volumetric CNNs MVCNN Su et al. 3DShapeNets Wu et al. 30x30x30 Volumes 224x224 Images

Diagnosis of Causes: Variable Control • Same resolution, study architectures • Same architecture, look into resolutions

Sphere Rendering Occupancy Grid Image Polygon Mesh 30x30x30 224x224

Sphere Rendering Same “3D Resolution” Occupancy Grid Image Polygon Mesh 30x30x30 224x224

Investigation into Architecture Multi-View Different 3D CNN Image CNN Architecture Same 3D Resolution (30x30x30) Sphere Rendering Occupancy Grid Images Volumes

CNNs with Same 3D Resolution Inputs 88 Shape Classification Accuracy 86 84 82 80 78 76 74 72 MVCNN with Sphere 3DShapeNets Rendering Images Wu et al.

Novel 3D CNN Architectures  3D NIN with Subvolume Supervision Push Harder for Learning Better!

Novel 3D CNN Architectures  Anisotropic Probing Network

Results of Our Novel 3D CNNs 88 Shape Classification Accuracy 86 84 82 80 78 76 74 72 MVCNN with 3DShapeNets Ours 3D CNN Sphere Rendering Wu et al. Images

Results of Our Novel 3D CNNs Closed the Gap under same 3D Resolution 88 Shape Classification Accuracy 86 84 82 80 78 76 74 72 MVCNN with 3DShapeNets Ours 3D CNN Sphere Rendering Wu et al. Images

Investigation into Resolution Multi-View Multi-View Same 3D CNN Image CNN Image CNN Architecture Different 3D Resolution Standard Rendering Sphere Rendering 30x30x30 Images Images Volume

Performance Trend wrt 3D Resolution 94 92 Accuracy (%) 90 88 86 MVCNN-Sphere 84 82 0 50 100 150 200 250 3D Resolution

Performance Trend wrt 3D Resolution 94 92 Accuracy (%) 90 88 86 MVCNN-Sphere 84 Our 3D CNN 82 0 50 100 150 200 250 3D Resolution

Generalization to Real Scans Shape retrieval on scan data Real Scan Dataset 243 objects 12 categories

Volumetric and Multi-View CNNs for Object Classification on 3D Data Code and Data Available Online! http://graphics.stanford.edu/projects/3dcnn/ Welcome to Our Poster #38!