BSP-Net: Generating Compact Meshes via Binary Space Partitioning - PowerPoint PPT Presentation

BSP-Net: Generating Compact Meshes via Binary Space Partitioning Zhiqin Chen Andrea Tagliasacchi Hao (Richard) Zhang Simon Fraser University Google Research Simon Fraser University Presented by 陈之钦，西蒙弗雷泽大学 Zhiqin Chen, Ph.D. student, Simon Fraser University

3D shape representations Mesh ShapeNet PolyGen Chang et al, arXiv Nash et al, arXiv 2

3D shape representations Voxel 3D-R2N2 3DGAN Choy et al, ECCV 2016 Wu et al, NIPS 2016 3

3D shape representations Point cloud z (128d) FC 512 FC 1024 FC 6144 = 2048x3 Learning Representations and Generative Models for 3D Point Clouds Achlioptas et al, ICML 2018 4

3D shape representations Warping template meshes 3DN AtlasNet Wang et al, CVPR 2019 Groueix et al, CVPR 2018 Pixel2mesh 5 Wang et al, ECCV 2018

3D shape representations implicit field IM-NET OccNet DeepSDF Chen & Zhang, CVPR 2019 Mescheder et al, CVPR 2019 Park et al, CVPR 2019 6

Compactness / low-poly / sharp details Chen & Zhang, CVPR 2019

Compactness / low-poly / sharp details

Represent shapes as Binary Space Partitioning trees max-pooling min-pooling 9 *CSG operations – Constructive Solid Geometry operations

10

11

Network architecture D [: , 0] [: , 1] [: , 2] [: , 3] [: , 4] [: , 5] [: , 6] [: , 7] [: , 8] D nxp T pxc C nxc C [: , 0] [: , 1] [: , 2] S *nx1 -1 max-pooling shape surface 0 min-pooling +1 12

Network architecture max-pooling min-pooling 13

Network architecture max-pooling min-pooling 14

Replacing max/min with weighted sum 15

Replacing max/min with weighted sum D [: , 0] [: , 1] [: , 2] [: , 3] [: , 4] [: , 5] [: , 6] [: , 7] [: , 8] D nxp relu(x) + 0 relu(D nxp ) + 0 0 + T pxc 0 C nxc =relu( D nxp ) T + relu(1-x) relu( 1 - C nxc ) 1 ≈0 relu( 1 - C nxc ) 1 cx1 S +nx1 16

Replacing max/min with weighted sum D [: , 0] [: , 1] [: , 2] [: , 3] [: , 4] [: , 5] [: , 6] [: , 7] [: , 8] D nxp relu(x) + 0 relu(D nxp ) + 0 0 + T pxc 0 C nxc =relu( D nxp ) T + [ 1 - C nxc ] [0,1] 1 ≈0 ≈relu( 1 - C nxc ) [ 1 - C nxc ] [0,1] 1 cx1 S +nx1 ≈relu( 1 - C nxc ) 1 cx1 17

Replacing max/min with weighted sum D [: , 0] [: , 1] [: , 2] [: , 3] [: , 4] [: , 5] [: , 6] [: , 7] [: , 8] D nxp relu(x) + 0 relu(D nxp ) + 0 0 + T pxc 0 C nxc =relu( D nxp ) T + [ 1 - C nxc ] [0,1] 1 ≈0 ≈relu( 1 - C nxc ) [ 1 - C nxc ] [0,1] W cx1 S +nx1 ≈relu( 1 - C nxc ) 1 cx1 18

Replacing max/min with weighted sum D [: , 0] [: , 1] [: , 2] [: , 3] [: , 4] [: , 5] [: , 6] [: , 7] [: , 8] D nxp relu(x) + 0 relu(D nxp ) + 0 0 + T pxc 0 C nxc =relu( D nxp ) T + [ 1 - C nxc ] [0,1] 1 ≈0 ≈relu( 1 - C nxc ) [ 1 - C nxc ] [0,1] W cx1 S +nx1 ≈relu( 1 - C nxc ) 1 cx1 19 [ [ 1 - relu( xP T ) T ] [0,1] W ] [0,1]

Replacing max/min with weighted sum D [: , 0] [: , 1] [: , 2] [: , 3] [: , 4] [: , 5] [: , 6] [: , 7] [: , 8] D nxp relu(x) + 0 relu(D nxp ) + 0 0 + T pxc 0 C nxc =relu( D nxp ) T + [ 1 - C nxc ] [0,1] 1 ≈0 ≈relu( 1 - C nxc ) [ 1 - C nxc ] [0,1] W cx1 S +nx1 ≈relu( 1 - C nxc ) 1 cx1 20 [ [ 1 - relu( xP T ) T ] [0,1] W ] [0,1]

Visualizing training - Initialization Output shape Continuous weights Convexes Continuous weights Planes 21

Visualizing training - Continuous phase 22

Visualizing training – Discrete phase Min-pooling Binary weights 23

Visualizing training – Discrete phase Min-pooling Binary weights 24

Visualizing training – Discrete phase 25

Toy dataset – 12 examples 26

Toy dataset - Reconstruction 27

3D Reconstruction – Part Decomposition Volumetric Primitives Tulsiani et al, CVPR 2017 SuperQuadrics Paschalidou et al, CVPR 2019 BAE-NET Chen et al, ICCV 2019 BSP-NET (ours) Reconstruction ground truth 28

29

Single View Reconstruction (RGB→3D) ResNet ResNet ResNet ResNet ResNet BSP-Net BSP-Net BSP-Net BSP-Net BSP-Net 30

Input image AtlasNet (1 sphere) Groueix et al, CVPR 2018 AtlasNet (25 patches) Groueix et al, CVPR 2018 31

Quantitative results 32

Thank you. Q&A 33