sagnet structure aware generative network for 3d shape
play

SAGNet: Structure-aware Generative Network for 3D- Shape Modeling - PowerPoint PPT Presentation

Oct 02, 2022 •220 likes •595 views

SAGNet: Structure-aware Generative Network for 3D- Shape Modeling Zhijie Wu Shenzhen University Xiang Wang Shenzhen University Di Lin Shenzhen

  1. SAGNet: Structure-aware Generative Network for 3D- Shape Modeling Zhijie Wu Shenzhen University Xiang Wang Shenzhen University Di Lin Shenzhen University Dani Lischinski The Hebrew University of Jerusalem Daniel Cohen-Or Shenzhen University Hui Huang Shenzhen University

  2. Outline Motivation Method Result Conclusion SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  3. Outline Motivation Method Result Conclusion SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  4. Motivation Method Results Conclusion = Background [Wu et al. NIPS 2016. “Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling”] SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  5. Motivation Method Results Conclusion = Related Work work 01 work 02 [Nash et.al 2017] [Li et.al 2017] work 03 work 04 [Wang et.al 2018] [Schor et.al 2018] SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  6. Motivation Method Results Conclusion = Motivations Tenon-mortise joint Each tenon-mortise joint consists of two parts. One part, in orange, has a cavity into which the second part, in blue, exactly fits. Thus, if the relation between the relative position of the parts and their geometry is not learned well by the network, it is unlikely that the network would succeed in generating the orange parts with a correctly sized and placed cavity. SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  7. Motivation Method Results Conclusion = Motivations Tenon-mortise joint Each tenon-mortise joint consists of two parts. One part, in orange, has a cavity into which the second part, in blue, exactly fits. Thus, if the relation between the relative position of the parts and their geometry is not learned well by the network, it is unlikely that the network would succeed in generating the orange parts with a correctly sized and placed cavity. SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  8. Outline Introduction Method Results Conclusion SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  9. Motivation Method Results Conclusion = Framework Pipeline SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  10. Motivation Method Results Conclusion = Framework Pipeline To better jointly analyse the geometry and structure information, we follow the message passing strategy to update these two information iteratively, share similar spirit of [Xu et al. 2017]. SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC) 10

  11. Motivation Method Results Conclusion = Framework Pipeline To better jointly analyse the geometry and structure information, we follow the message passing strategy to update these two information iteratively, share similar spirit of [Xu et al. 2017]. SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC) 11

  12. Motivation Method Results Conclusion = Framework Pipeline To better jointly analyse the geometry and structure information, we follow the message passing strategy to update these two information iteratively, share similar spirit of [Xu et al. 2017]. SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC) 12

  13. Motivation Method Results Conclusion = Architecture SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  14. Motivation Method Results Conclusion = Architecture ×512 ×512 GRU GRU 1×512 1×512 k k Encoder Decoder 1×512 GRU Latent GRU Encoder Code Decoder GRU GRU Encoder Decoder 1×512 1×512 ×512 ×512 K K SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  15. Motivation Method Results Conclusion = Training strategy SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  16. Motivation Method Results Conclusion = Training strategy Posterior Collapse Problems! SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  17. Motivation Method Results Conclusion = Training strategy VAEs are hard to train when combined with powerful autoregressive decoders or RNNs. This is due to the “posterior collapse” problem: the model ends up relying solely on the properties of the decoder while ignoring the latent variables, which become uninformative. [Bowman et al. Generating Sentences from a Continuous Space] SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  18. Motivation Method Results Conclusion = Training strategy(First phase) SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  19. Motivation Method Results Conclusion = Training strategy(Second phase) × 5 1 2 × 5 1 2 GRU 1 × 5 1 2 GRU 1 × 5 1 2 k k Encoder Decoder 1 × 5 1 2 GRU Latent GRU Encoder Code Decoder GRU GRU Encoder Decoder 1×512 1×512 × 5 1 2 × 5 1 2 K K SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  20. Motivation Method Results Conclusion = Testing Procedure kx512x512 1x512 kx512 GRU 3D Deconv Decoder GRU Z Decoder 1x512 GRU FC Decoder 1x512 kx512 kx12 SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  21. Motivation Method Results Conclusion = Ablation study frameworks The architectures of three ablation study baseline models. The top left diagram denotes the No-attention baseline model. The top right diagram corresponds to No-GRU baseline model. The diagram that lies at bottom indicates the baseline model of Concatenation . SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  22. Outline Introduction Method Results Conclusion SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  23. Motivation Method Results Conclusion = Embedding space SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  24. Motivation Method Results Conclusion = kNN results SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  25. Motivation Method Results Conclusion = Interpolation results SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  26. Motivation Method Results Conclusion = The synthesis result comparison SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  27. Motivation Method Results Conclusion = Tenon-mortise joint SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  28. Motivation Method Results Conclusion = Results on tenon-mortise joints We randomly generate 1000 test samples with the trained network and measure how well the convex parts fit into the cavity of the non-convex ones. To quantitatively measure the fitting accuracy, we calculate, Ro (Re), the portion of occupancy(empty) voxels of the non-convex part that are overlapped with occupancy voxels of the convex part. Then the smaller score R = 1 − (Re −Ro) indicates better fitting status between the two parts. SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  29. Motivation Method Results Conclusion = Results on tenon-mortise joints SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  30. Motivation Method Results Conclusion = Application Results Shape completion SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  31. Motivation Method Results Conclusion = Application Results Structure-geometry Translation SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  32. Motivation Method Results Conclusion = Application Results Geometry-structure Translation SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

  33. Outline Introduction Method Results Conclusion SAGNet: Structure-aware Generative Network for 3D-Shape Modeling Visual Computing Research Center (VCC)

Recommend

A Probabilistic Model for Component Based Shape Synthesis Evangelos Kalogerakis, Siddhartha

A Probabilistic Model for Component Based Shape Synthesis Evangelos Kalogerakis, Siddhartha

A Probabilistic Model for Component Based Shape Synthesis Evangelos Kalogerakis, Siddhartha Chaudhuri, Daphne Koller, Vladlen Koltun Stanford University Goal: generative model of shape Goal: generative model of shape Challenge: understand shape

1.31k views • 55 slides
Generative vs. discriminative Generative Discriminative Belief network A is more More

Generative vs. discriminative Generative Discriminative Belief network A is more More

Generative vs. discriminative Generative Discriminative Belief network A is more More robust modular Dont need precise model Class-conditional densities are specification, so long as it is likely to be local, characteristic

1.09k views • 15 slides
Smooth Shape-Aware Functions with Controlled Extrema Alec Jacobson 1 1 ETH Zurich Tino Weinkauf 2

Smooth Shape-Aware Functions with Controlled Extrema Alec Jacobson 1 1 ETH Zurich Tino Weinkauf 2

Smooth Shape-Aware Functions with Controlled Extrema Alec Jacobson 1 1 ETH Zurich Tino Weinkauf 2 2 MPI Saarbrcken Olga Sorkine 1 August 9, 2012 Real-time deformation relies on smooth, shape-aware functions input shape + handles August 9,

1.06k views • 87 slides
Generative Models for Complex Network Structure Aaron Clauset @aaronclauset Computer Science

Generative Models for Complex Network Structure Aaron Clauset @aaronclauset Computer Science

Generative Models for Complex Network Structure Aaron Clauset @aaronclauset Computer Science Dept. & BioFrontiers Institute University of Colorado, Boulder External Faculty, Santa Fe Institute 4 June 2013 NetSci 2013, Complex

820 views • 44 slides
Learning the Structure of Generative Models without Labeled Data Stephen Bach Bryan He

Learning the Structure of Generative Models without Labeled Data Stephen Bach Bryan He

Learning the Structure of Generative Models without Labeled Data Stephen Bach Bryan He Alex Ratner Chris R Stanford University This Talk We study structure learning for generative models in which a latent variable

599 views • 33 slides
Introduction to Generative Adversarial Network (GAN) Hongsheng Li Department of Electronic

Introduction to Generative Adversarial Network (GAN) Hongsheng Li Department of Electronic

Introduction to Generative Adversarial Network (GAN) Hongsheng Li Department of Electronic Engineering Chinese University of Hong Kong Adversarial adj. 1 Generative Models Density Estimation ( | ) p y x

1.35k views • 98 slides
Generative Adversarial Network and it its Applications to Human La Language Processing

Generative Adversarial Network and it its Applications to Human La Language Processing

Generative Adversarial Network and it its Applications to Human La Language Processing Hung-yi Lee Full version of the tutorial Outline Part I: General Introduction of Generative Adversarial Network (GAN) Part II: Applications

917 views • 78 slides
The 2D shape structure dataset: A user annotated open access database A. Carlier, G. Morin K.

The 2D shape structure dataset: A user annotated open access database A. Carlier, G. Morin K.

The 2D shape structure dataset: A user annotated open access database A. Carlier, G. Morin K. Leonard S. Hahmann M. Collins IRIT INP Toulouse CSUCI INRIA Grenoble GISHWHES 24-06-2016 SMI'16 1 Motivation [Liu et al.] Convex shape

399 views • 22 slides
for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

Deep Generative Modeling for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text Praveen Narayanan Ford Motor Company PROJECT DESCRIPTION Use of DNNs increasingly prevalent as a solution for many data

893 views • 53 slides
LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David

LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David

LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David Alvarez-Melis, Andreas Krause, Stefanie Jegelka Pos oster #173 173 Poster #173 173 & Charlotte Bunne Generative Modeling generative = network

250 views • 13 slides
Network-aware Service Placement in Community Network Clouds Mennan Selimi mselimi@ac.upc.edu

Network-aware Service Placement in Community Network Clouds Mennan Selimi mselimi@ac.upc.edu

Network-aware Service Placement in Community Network Clouds Mennan Selimi mselimi@ac.upc.edu Advisors: Prof. Felix Freitag (UPC), Prof. Luis Vega (IST/INESC-ID) EMJD-DC Summer Event 2 nd of June, 2016 Activity Network - started in 2004 -

576 views • 23 slides
Demand Aware Network ( DAN ) Design Some Results and Open Questions Chen Avin Joint work with

Demand Aware Network ( DAN ) Design Some Results and Open Questions Chen Avin Joint work with

Demand Aware Network ( DAN ) Design Some Results and Open Questions Chen Avin Joint work with Stefan Schmid, Kaushik Mondal, Alexandr Hercules, Andreas Loukas Motivation Demand Aware Network Design? self-adjust the networks

249 views • 20 slides
Unsupervised Learning of Visual Structure Using Predictive Generative Networks William Lotter,

Unsupervised Learning of Visual Structure Using Predictive Generative Networks William Lotter,

Unsupervised Learning of Visual Structure Using Predictive Generative Networks William Lotter, Gabriel Kreiman & David Cox Harvard University, Cambridge, USA Article overview by Ilya Kuzovkin Computational Neuroscience Seminar University

964 views • 45 slides
Deep-Q: Traffic-driven QoS Inference using Deep Generative Network Shihan Xiao , Dongdong He,

Deep-Q: Traffic-driven QoS Inference using Deep Generative Network Shihan Xiao , Dongdong He,

Deep-Q: Traffic-driven QoS Inference using Deep Generative Network Shihan Xiao , Dongdong He, Zhibo Gong Network Technology Lab, Huawei Technologies Co., Ltd., Beijing, China 1 Background What is a QoS Model? Traffic D elay, jitter, packet

464 views • 27 slides
Structure-aware fuzzing for Clang and LLVM with libprotobuf-mutator Kostya Serebryany, Vitaly

Structure-aware fuzzing for Clang and LLVM with libprotobuf-mutator Kostya Serebryany, Vitaly

Structure-aware fuzzing for Clang and LLVM with libprotobuf-mutator Kostya Serebryany, Vitaly Buka, Matt Morehouse; Google October 2017 Agenda Fuzzing Fuzzing Clang/LLVM Fuzzing Clang/LLVM better (structure-aware)

914 views • 35 slides
Towards Personalized Review Summarization via User-Aware Sequence Network ADVISOR: JIA-LING, KOH

Towards Personalized Review Summarization via User-Aware Sequence Network ADVISOR: JIA-LING, KOH

Towards Personalized Review Summarization via User-Aware Sequence Network ADVISOR: JIA-LING, KOH SOURCE: AAAI-19 SPEAKER: WEI, LAI DATE: 2020/5/25 OUTLINE 01 03 INTRODUCTION EXPERIMENTS 02 04 METHOD CONCLUSION 2 4 User-aware

822 views • 30 slides
Network Coding-Aware Queue Network Coding Aware Queue Management for Unicast Flows over Coded

Network Coding-Aware Queue Network Coding Aware Queue Management for Unicast Flows over Coded

Network Coding-Aware Queue Network Coding Aware Queue Management for Unicast Flows over Coded Wireless Networks Coded Wireless Networks Hlya Sefero lu, Athina Markopoulou University of California Irvine University of California, Irvine

287 views • 24 slides
Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Generative Adversarial Text to Image Synthesis

666 views • 39 slides
Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative

Applications of GANs Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks Generative Adversarial Text to Image Synthesis

614 views • 27 slides
Towards Structure-Aware Earth System Data Management Jakob Lttgau, Julian Kunkel, Bryan N.

Towards Structure-Aware Earth System Data Management Jakob Lttgau, Julian Kunkel, Bryan N.

Towards Structure-Aware Earth System Data Management Towards Structure-Aware Earth System Data Management Jakob Lttgau, Julian Kunkel, Bryan N. Lawrence, Sandro Fiore, Huang Hua Jakob Luettgau (DKRZ) Middleware for Earth System Data PDSW SC17

341 views • 6 slides
Probabilistic Models of Cognition: Generative models Table of Contents Chapter

Probabilistic Models of Cognition: Generative models Table of Contents Chapter

Probabilistic Models of Cognition: Generative models Table of Contents Chapter Content Generative Model Example: Plinko Machine Working Model can be used captures some for simulation structure of the world in useful way

825 views • 78 slides
GeniusRoute: A New Analog Routing Paradigm Using Generative Neural Network Guidance Keren Zhu ,

GeniusRoute: A New Analog Routing Paradigm Using Generative Neural Network Guidance Keren Zhu ,

GeniusRoute: A New Analog Routing Paradigm Using Generative Neural Network Guidance Keren Zhu , Mingjie Liu, Yibo Lin, Biying Xu, Shaolan Li, Xiyuan Tang, Nan Sun and David Z. Pan ECE Department The University of Texas at Austin This work is

525 views • 26 slides
A Gradual, Semi-Discrete Approach to Generative Network Training via Explicit Wasserstein

A Gradual, Semi-Discrete Approach to Generative Network Training via Explicit Wasserstein

A Gradual, Semi-Discrete Approach to Generative Network Training via Explicit Wasserstein Minimization Yucheng Chen 1 Matus Telgarsky 1 Chao Zhang 1 Bolton Bailey 1 Daniel Hsu 2 Jian Peng 1 1 Department of Computer Science, UIUC, Urbana, IL 2

99 views • 8 slides
PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes Rundi Wu Yixin Zhuang Kai Xu Hao

PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes Rundi Wu Yixin Zhuang Kai Xu Hao

PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes Rundi Wu Yixin Zhuang Kai Xu Hao Zhang Baoquan Chen 1,4 1 2 3,4 1,4 1 Center on Frontiers of Computing Studies, Peking University National University of Defense Technology

773 views • 20 slides

More recommend