3d shapenets a deep representation for volumetric shape
play

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling - PowerPoint PPT Presentation

Jul 12, 2023 •186 likes •1.32k views

3D ShapeNets: A Deep Representation for Volumetric Shape Modeling by Wu, Song, Khosla, Yu, Zhang, Tang, Xiao presented by Abhishek Sinha 1 3D Shape Prior 2 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric

  1. Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  2. Sampling •••• object label Gibbs Sampling generation process: 20 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  3. Dataset 21

  4. Big 3D Data Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  5. Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  6. Big 3D Data Query Keyword: common object categories from the SUN database that contain no less than 20 object instances per category 23 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  7. Big 3D Data 151,128 models 660 categories Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  8. Applications 25

  9. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  10. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  11. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  12. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  13. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  14. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  15. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  16. 2.5D Completion & Recognition 26 Slide Credit: Wu et al

  17. 2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al

  18. 2.5D Completion & Recognition 26 Gibbs sampling with clamping Slide Credit: Wu et al

  19. 2.5D Completion & Recognition Training on CAD models and no discriminative tuning! [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. 27 Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  20. View Planning for Recognition 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  21. View Planning for Recognition Volumetric representation 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  22. View Planning for Recognition sofa? ? dresser? Volumetric representation What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  23. View Planning for Recognition sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  24. View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  25. View Planning for Recognition Next-Best-View sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  26. View Planning for Recognition Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  27. View Planning for Recognition 3D ShapeNets Aha! Next-Best-View It is a sofa! sofa? ? dresser? Not sure. Volumetric Look from representation another view? What is it? bathtub? Where to look next? New depth map 28 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  28. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  29. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  30. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  31. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  32. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  33. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  34. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  35. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 29 Slide Credit: Wu et al

  36. Deep View Planning 0.8 0.5 0.8 0.3 0.3 0.8 0.3 0.4 0.8 0.4 0.7 0.3 0.3 0.8 0.8 0.2 Mathematically, this is equivalent to evaluate the conditional mutual information: 29 Slide Credit: Wu et al

  37. Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  38. Deep View Planning Recognition Accuracy from Two Views. Based on the algorithms’ choice, we obtain the actual depth map for the next view and recognize the objects using two views by our 3D ShapeNets. Our algorithm stands out as the uncertainty of the first view increases 30 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  39. Back Propagation Fine-tuning 3D ShapeNets 4000 object label 1200 2 512 filters of stride 1 5 160 filters of stride 2 13 48 filters of stride 2 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  40. Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  41. Back Propagation Fine-tuning 3D ShapeNets 4000 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  42. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  43. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  44. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  45. Back Propagation Fine-tuning 3D ShapeNets 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  46. Back Propagation Fine-tuning 3D ShapeNets 3D CNN 4000 object label object label 1200 1200 2 2 512 filters of 512 filters of stride 1 stride 1 5 5 160 filters of 160 filters of stride 2 stride 2 13 13 48 filters of 48 filters of stride 2 stride 2 30 30 31 Slide Credit: Wu, Song et al. 3D ShapeNets: A Deep Representation for Volumetric Shape Modeling, CVPR 2015

  47. As a 3D Feature Extractor 32 Slide Credit: Wu et al

  48. As a 3D Feature Extractor Mesh Classification & Retrieval 32 Slide Credit: Wu et al

  49. As a 3D Feature Extractor Mesh Classification & Retrieval 2.5D object recognition 32 [29] R. Socher, B. Huval, B. Bhat, C. D. Manning, and A. Y. Ng. Convolutional-recursive deep learning for 3d object classification. In NIPS 2012. Slide Credit: Wu et al

Recommend

Shape Representation Jin Xie Department of Computer Science and Engineering Nanjing University

Shape Representation Jin Xie Department of Computer Science and Engineering Nanjing University

Deep Learning Based 3D Shape Representation Jin Xie Department of Computer Science and Engineering Nanjing University of Science and Technology, China 1 Outline Overview of 3D deep learning Deep learned 3D shape feature for retrieval

875 views • 32 slides
Systematical Parameterization, Storage and Representation of Volumetric DICOM Data for

Systematical Parameterization, Storage and Representation of Volumetric DICOM Data for

Systematical Parameterization, Storage and Representation of Volumetric DICOM Data for Visualization: 3D Presentation States (3DPR) Dr. M. Alper SELVER Dokuz Eylul University Electrical and Electronics Engineering Research Group FH-Juelich

471 views • 19 slides
Practical Deep Learning micha.codes / fastforwardlabs.com 1 / 70 deep learning can seem

Practical Deep Learning micha.codes / fastforwardlabs.com 1 / 70 deep learning can seem

Practical Deep Learning micha.codes / fastforwardlabs.com 1 / 70 deep learning can seem mysterious 2 / 70 let's nd a way to just build a function 3 / 70 Feed Forward Layer # X.shape == (512,) # output.shape == (4,) # weights.shape ==

1.05k views • 70 slides
3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates: multi-view images depth map volumetric polygonal mesh point cloud primitive-based CAD models 3D representation Candidates: multi-view images

1.32k views • 86 slides
3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates:

3D Deep Learning on Geometric Forms Hao Su Many 3D representations are available Candidates: multi-view images depth map volumetric polygonal mesh point cloud primitive-based CAD models 3D representation Candidates: multi-view images

1.34k views • 75 slides
LAPLACE-BELTRAMI EIGENFUNCTIONS FOR DEFORMATION INVARIANT SHAPE REPRESENTATION Raif Rustamov

LAPLACE-BELTRAMI EIGENFUNCTIONS FOR DEFORMATION INVARIANT SHAPE REPRESENTATION Raif Rustamov

LAPLACE-BELTRAMI EIGENFUNCTIONS FOR DEFORMATION INVARIANT SHAPE REPRESENTATION Raif Rustamov Department of Mathematics Purdue University, West Lafayette, IN Motivation Deformable shapes Computer graphics Shape modeling

326 views • 22 slides
Chapter 8 Shape representation and description 8.1 Matching 2 8.1 Matching we wish to

Chapter 8 Shape representation and description 8.1 Matching 2 8.1 Matching we wish to

Chapter 8 Shape representation and description 8.1 Matching 2 8.1 Matching we wish to match some model to the data At simplest, known binary patterns representing characters of a font may be sought in properly aligned scans of

406 views • 27 slides
Lecture 5: Data Representation 1 / 43 Data Representation Discussion Deep learning job postings

Lecture 5: Data Representation 1 / 43 Data Representation Discussion Deep learning job postings

Data Representation Lecture 5: Data Representation 1 / 43 Data Representation Discussion Deep learning job postings have collapsed in past 6 months Something Ive learned: when non-engineers ask for an AI or ML implementation, they almost

618 views • 43 slides
Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with

Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with

Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with Localized Data with V l V l Volumetric Data Volumetric Data i D i D Robert Weersink, Harsimran Braisch, Greg Bootsma David Jaffray Greg

308 views • 20 slides
SPARSE VOLUMETRIC REPRESENTATION OF TIME-LAPSE POINT CLOUD Innfarn Yoo, 05.08.2017 Introduction

SPARSE VOLUMETRIC REPRESENTATION OF TIME-LAPSE POINT CLOUD Innfarn Yoo, 05.08.2017 Introduction

SPARSE VOLUMETRIC REPRESENTATION OF TIME-LAPSE POINT CLOUD Innfarn Yoo, 05.08.2017 Introduction Previous Work AGENDA Method Result Future Work 2 INTRODUCTION Time-lapse Point Cloud Dataset Captured external point cloud data by captured

667 views • 35 slides
Shape representation as distribution of geometric features: from currents to oriented varifolds

Shape representation as distribution of geometric features: from currents to oriented varifolds

Workshop on Statistical modeling for shapes and imaging Shape representation as distribution of geometric features: from currents to oriented varifolds Irne Kaltenmark Inria - Institut de Mathmatiques de Bordeaux Universit de Bordeaux

644 views • 47 slides
Shape Representation and Description Alexandre Falc ao Institute of Computing - University of

Shape Representation and Description Alexandre Falc ao Institute of Computing - University of

Shape Representation and Description Alexandre Falc ao Institute of Computing - University of Campinas afalcao@ic.unicamp.br Alexandre Falc ao MC920/MO443 - Indrodu c ao ao Proc. de Imagens Introduction A segmented object may

1.23k views • 77 slides
animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation

animation 1 animation shape specification as a function of time 2 animation representation many ways to represent changes with time intent artistic motion physically-plausible motion efficiency typically not a major problem control most

757 views • 74 slides
Meaning Representation and Semantic Analysis Ling 571 Deep Processing Techniques for NLP

Meaning Representation and Semantic Analysis Ling 571 Deep Processing Techniques for NLP

Meaning Representation and Semantic Analysis Ling 571 Deep Processing Techniques for NLP February 9, 2011 Roadmap Meaning representation: Event representations Temporal representation Semantic Analysis

626 views • 52 slides
Caffeine: Towards Uniformed Representation Introduction and Acceleration for Deep Convolutional

Caffeine: Towards Uniformed Representation Introduction and Acceleration for Deep Convolutional

FPGA C.Zhang et al. Caffeine: Towards Uniformed Representation Introduction and Acceleration for Deep Convolutional Motivation Neural Networks Uniformed CNN Representation Caffeine Design Chen Zhang, Zhenman Fang, Peipei Zhou et al.

445 views • 31 slides
Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M.

Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M.

Laplace-Beltrami Eigenfunctions for Deformation Invariant Shape Representation Author: Raif M. Rustamov Presenter: Dan Abretske Johns Hopkins 2007 Outline Motivation and Background Laplace-Beltrami Operator Global Point Signature

314 views • 18 slides
Toward Automated Pattern Discovery: Deep Representation Learning with Spatial-Temporal-Networked

Toward Automated Pattern Discovery: Deep Representation Learning with Spatial-Temporal-Networked

Toward Automated Pattern Discovery: Deep Representation Learning with Spatial-Temporal-Networked Data Collective, Dynamic, and Structured Analysis Yanjie Fu Outline 5 Background and Motivation Collective Representation Learning

788 views • 65 slides
Representation Learning UCA Deep Learning School - Deep in France Nice 2017 Soufiane Belharbi

Representation Learning UCA Deep Learning School - Deep in France Nice 2017 Soufiane Belharbi

Representation Learning UCA Deep Learning School - Deep in France Nice 2017 Soufiane Belharbi Romain Hrault Clment Chatelain Sbastien Adam soufiane.belharbi@insa-rouen.fr LITIS lab., Apprentissage team - INSA de Rouen, France 13 June,

1.3k views • 86 slides
DEEP LEARNING FRAMEWORK FOR CYBER-ENABLED MANUFACTURING ADI DITY TYA A BALU SA SAMBIT IT

DEEP LEARNING FRAMEWORK FOR CYBER-ENABLED MANUFACTURING ADI DITY TYA A BALU SA SAMBIT IT

GPU-ACCELERATED DEEP LEARNING FRAMEWORK FOR CYBER-ENABLED MANUFACTURING ADI DITY TYA A BALU SA SAMBIT IT GHADAI AI SO SOUMIK K SA SARKA KAR ADARS RSH H KR KRISH SHNAMUR AMURTHY THY Outline Volumetric Deep Learning based

325 views • 32 slides
Deep Representation and Reinforcement Learning Soumalya Sarkar, PhD for Anomaly Detection and

Deep Representation and Reinforcement Learning Soumalya Sarkar, PhD for Anomaly Detection and

Deep Representation and Reinforcement Learning Soumalya Sarkar, PhD for Anomaly Detection and Control Senior Research scientist, UTRC in Multi-modal Aerospace Applications May 9 @ GTC 2017 This document contains no technical data subject to the

1.09k views • 43 slides
Structured Computation and Representation in Deep Reinforcement Learning Jessica B. Hamrick

Structured Computation and Representation in Deep Reinforcement Learning Jessica B. Hamrick

Structured Computation and Representation in Deep Reinforcement Learning Jessica B. Hamrick AAMAS/ICML/IJCAI Workshop on Prediction and Generative Modeling in RL Stockholm, Sweden July 15, 2018 Understanding the World in Terms of Objects and

1.71k views • 159 slides
A Deep Representation for Invariance and Music Classification Chiyuan Zhang, Georgios

A Deep Representation for Invariance and Music Classification Chiyuan Zhang, Georgios

A Deep Representation for Invariance and Music Classification Chiyuan Zhang, Georgios Evangelopoulos, Stephen Voinea, Lorenzo Rosasco, Tomaso Poggio. Center for Brains, Minds and Machines (CBMM) Computer Science and Artificial Intelligence

440 views • 19 slides
Deformable Organ Contour Transfer with Deep Inverse Shape Encoding (DISE) Networks for

Deformable Organ Contour Transfer with Deep Inverse Shape Encoding (DISE) Networks for

Deformable Organ Contour Transfer with Deep Inverse Shape Encoding (DISE) Networks for Auto-segmentation in Low Contrast Regions Tiancheng Liu 1 Xiaobai Sun 1 Fang-fang Yin 2 Lei Ren 2 1 Department of Computer Science, Duke University, USA 2

572 views • 20 slides
Shape Analysis Syntax of the pointer language p | n | a 1 op a a 2 | nil a ::= Goal: to obtain

Shape Analysis Syntax of the pointer language p | n | a 1 op a a 2 | nil a ::= Goal: to obtain

Shape Analysis Syntax of the pointer language p | n | a 1 op a a 2 | nil a ::= Goal: to obtain a finite representation of the shape of the heap of a ::= x | x. sel p language with pointers. b ::= true | false | not b | b 1 op b b 2 | a 1 op

430 views • 11 slides

More recommend