gradnet unsupervised deep screened poisson reconstruction
play

GradNet: Unsupervised Deep Screened Poisson Reconstruction for - PowerPoint PPT Presentation

Aug 28, 2022 •455 likes •963 views

GradNet: Unsupervised Deep Screened Poisson Reconstruction for GradientDomain Rendering Jie Guo 1 Mengtian Li 1 Quewei Li 1 Yuting Qiang 1 Bingyang Hu 1 Yanwen Guo 1 LingQi Yan 2 1 State Key Lab for Novel Software 2 University of California,

  1. GradNet: Unsupervised Deep Screened Poisson Reconstruction for Gradient‐Domain Rendering Jie Guo 1 Mengtian Li 1 Quewei Li 1 Yuting Qiang 1 Bingyang Hu 1 Yanwen Guo 1 Ling‐Qi Yan 2 1 State Key Lab for Novel Software 2 University of California, Santa Barbara Technology, Nanjing University sa2019.siggraph.org

  2. Path tracing Light Mirror Diffuse Diffuse SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  3. Path tracing error / 2 = samples * 4 SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  4. Gradient‐domain Rendering Base Path Offset Path Light Mirror Diffuse Diffuse SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  5. Gradient‐domain Rendering SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  6. Gradient‐domain Rendering SA2019.SIGGRAPH.ORG 6 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  7. Screened Poisson Reconstruction • For a base image and gradients rendered by any gradient‐ domain algorithms, we can reconstruct the final image by solving the following optimization problem SA2019.SIGGRAPH.ORG 7 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  8. Screened Poisson Reconstruction • For a base image and gradients rendered by any gradient‐ domain algorithms, we can reconstruct the final image by solving the following optimization problem Data term Gradient term SA2019.SIGGRAPH.ORG 8 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  9. Screened Poisson Reconstruction • Lp norm p = 2 means L 2 reconstruction p = 1 means L 1 reconstruction SA2019.SIGGRAPH.ORG 9 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  10. Regularization • A regularized version of the screened Poisson solver can be written as Regularizer SA2019.SIGGRAPH.ORG 9 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  11. Regularized Screen Poisson Reconstruction Rendering‐specific features SA2019.SIGGRAPH.ORG 1 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  12. Our Idea Rendering‐specific features SA2019.SIGGRAPH.ORG 1 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  13. Related Work METHOD DEEP LEARNING BASED AUXILIARY BUFFERS PERFORMANCE L1 × × ≈0.45s GPU CV [Rousselle et al. 2016] × × ≈2s CPU LTS [Ha et al. 2019] × × ≈1.7s CPU NFOR [Bitterli et al. 2016] × √ ≈200s CPU REG [Manzi et al. 2016] × √ ≈60s GPU KPCN [Bako et al. 2017] √ Supervised √ ≈1.7s GPU [Kettunen et al. 2019] √ Supervised √ ≈0.3s GPU SA2019.SIGGRAPH.ORG 4 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  14. Related Work METHOD DEEP LEARNING BASED AUXILIARY BUFFERS PERFORMANCE L1 × × ≈0.45s GPU CV [Rousselle et al. 2016] × × ≈2s CPU LTS [Ha et al. 2019] × × ≈1.7s CPU NFOR [Bitterli et al. 2016] × √ ≈200s CPU REG [Manzi et al. 2016] × √ ≈60s GPU KPCN [Bako et al. 2017] √ Supervised √ ≈1.7s GPU [Kettunen et al. 2019] √ Supervised √ ≈0.3s GPU SA2019.SIGGRAPH.ORG 4 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  15. Related Work (REG [Manzi et al. 2016]) Feature Bases via Truncated SVD. Solved by an iteratively reweighted least squares (IRLS) approach SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  16. Related Work (KPCN) SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  17. Related Work ([Kettunen et al. 2019]) 1. Using gradients as an additional feature 2. Adopting a new perceptual loss SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  18. Our Method • Deep learning based • Unsupervised • Fast to reconstruct high‐quality image SA2019.SIGGRAPH.ORG 1 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  19. Our Solution • Replace the traditional optimization in screened Poisson reconstruction with GradNet SA2019.SIGGRAPH.ORG 10 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  20. Network Architecture • Multi‐branch auto‐encoder with dual skip connection Low‐frequency contents High‐frequency details SA2019.SIGGRAPH.ORG 20 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  21. Network Architecture • Multi‐branch auto‐encoder with dual skip connection Data branch G‐branch for generating derivative Gradient branch SA2019.SIGGRAPH.ORG 21 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  22. Impact of the Branches One‐branch encoder weaken the effects of sparse image gradients SA2019.SIGGRAPH.ORG 22 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  23. Dynamic Range Compression • We employ the μ‐law transformation to compress HDR data • The μ‐law transformation makes the training process easier than naïve log transformation SA2019.SIGGRAPH.ORG 23 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  24. Loss Function • The loss function contains 3 items • data item, gradient item and first‐order item • Data item: SA2019.SIGGRAPH.ORG 24 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  25. Loss Function • The loss function contains 3 items • data item, gradient item and first‐order item • Gradient item: SA2019.SIGGRAPH.ORG 25 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  26. Impact of the Gradient Loss I b I dx I dy w. gradient loss Reference w.o. gradient loss SA2019.SIGGRAPH.ORG 26 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  27. Loss Function • The loss function contains 3 items • data item, gradient item and first‐order item • The first‐order item: SA2019.SIGGRAPH.ORG 27 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  28. First‐order Regularization • The first‐order regularization defines as follow � 1 � � � � 𝐻 � � �𝐺 � � 𝑥 �,� 𝐽 � � 𝐽 � � 𝐺 � � 𝑂 𝑂 � ��� �∈� � Derivative of I i Difference of Neighboring pixels with respect to F i auxiliary features around index i SA2019.SIGGRAPH.ORG 28 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  29. First‐order Regularization • The first‐order regularization encourages nearby pixels to lie on a hyper‐plane parameterized by G G SA2019.SIGGRAPH.ORG 29 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  30. Impact of the First‐order Loss w.o. the first-order loss w. the first-order loss reference SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  31. Bias of Network μ will introduce bias to the reconstructed image Reference (mean: 0.204) μ = 16 (mean: 0.187) μ = 128 (mean: 0.176) μ = 1024 (mean: 0.163) SA2019.SIGGRAPH.ORG CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  32. Post‐Processing • We find a simple post‐processing step can reduce the bias Gaussian filter with r = 45 and σ = 15 Without P.P. With P.P. SA2019.SIGGRAPH.ORG 32 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  33. Training Details • Adam optimizer • β1 = 0.5 and β2 = 0.999 • Initial learning rate = 0.0001 and decays with the power of 0.95 for every other epoch • λ follows the schedule • Train main branches and G‐branch alternatively for 50 epochs with 32 mini‐batches SA2019.SIGGRAPH.ORG 33 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  34. Dataset • Randomly perturb 9 base scenes and render to 900 high‐ resolution images with 64 spp SA2019.SIGGRAPH.ORG 34 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  35. Dataset • 12654 patches with 256x256 resolution are extracted from these 900 high‐resolution images SA2019.SIGGRAPH.ORG 35 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

  36. Results SA2019.SIGGRAPH.ORG 36 CONFERENCE 17‐20 November 2019 ‐ EXHIBITION 18‐20 November 2019 ‐ BCEC , Brisbane, AUSTRALIA

Recommend

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Wanli Ouyang, Zsolt Kira, Lawrence Neal, Raymond Yeh, Junting Lou and Teck-Yian Lim Unsupervised Learning in General Unsupervised learning is

1.48k views • 115 slides
Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi,

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi,

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi, Tinne Tuytelaars Unsupervised Domain Adaptation When you expect the test data (Target) to be different from your training data (Source) DA in the

683 views • 39 slides
From unsupervised induction of linguistic structures from text towards applications in deep

From unsupervised induction of linguistic structures from text towards applications in deep

Alexander Panchenko From unsupervised induction of linguistic structures from text towards applications in deep learning In close collaboration with May 28, 2018 From unsupervised induction of linguistic structures to applications in deep

1.68k views • 142 slides
Poisson Distribution: Review Poisson Over Time Let B 1 Poisson( ) be the number of bikes

Poisson Distribution: Review Poisson Over Time Let B 1 Poisson( ) be the number of bikes

Poisson Distribution: Review Poisson Over Time Let B 1 Poisson( ) be the number of bikes that Values: are stolen on campus in one hour. (Go bears?) The Poisson Arrival Process Parameter(s) : What is the distribution of B 2 . 5 , the

924 views • 4 slides
Unsupervised Learning and Inverse Problems with Deep Neural Networks J oan Bruna, Stphane

Unsupervised Learning and Inverse Problems with Deep Neural Networks J oan Bruna, Stphane

Unsupervised Learning and Inverse Problems with Deep Neural Networks J oan Bruna, Stphane Mallat, Ivan Dokmanic, Martin de Hoop cole Normale Suprieure www.di.ens.fr/data Deep Convolutional Networks ( x ) x ( u ) L j L 1 f M (

520 views • 47 slides
Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders Generative Adversarial Networks Pr. Fabien MOUTARDE Center for Robotics MINES ParisTech PSL Universit Paris Fabien.Moutarde@mines-paristech.fr

777 views • 23 slides
Unsupervised Deep Learning Tutorial - Part 2 Alex Graves MarcAurelio Ranzato

Unsupervised Deep Learning Tutorial - Part 2 Alex Graves MarcAurelio Ranzato

Unsupervised Deep Learning Tutorial - Part 2 Alex Graves MarcAurelio Ranzato gravesa@google.com ranzato@fb.com NeurIPS, 3 December 2018 Overview Practical Recipes of Unsupervised Learning Learning representations Learning to

1.16k views • 103 slides
3D RECONSTRUCTION Reconstruction method Reconstruction from images Reconstruction from video

3D RECONSTRUCTION Reconstruction method Reconstruction from images Reconstruction from video

3D RECONSTRUCTION Reconstruction method Reconstruction from images Reconstruction from video Using Kinect Raw Depth Image Infrared laser projector Monochrome CMOS sensor Demo Kinect Raw data Real-time Reconstruction Pipeline Measurement

508 views • 34 slides
Newborn Screening What it means to me? What were you screened for when you were born? It

Newborn Screening What it means to me? What were you screened for when you were born? It

Newborn Screening What it means to me? What were you screened for when you were born? It depends on which state you were born in. If you were born in Kansas, prior to July 1, 2008, newborns were screened for PKU, galactosemia,

348 views • 6 slides
Punt: T , T , T are mutually independent . How about T , the number of

Punt: T , T , T are mutually independent . How about T , the number of

Poisson Distribution: Review Poisson Over Time - neg Let B 1 Poisson( ) be the number of bikes that integers non Values: are stolen on campus in one hour. (Go bears?) The Poisson Arrival Process Parameter(s) : X " rate "

278 views • 4 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
Updates on Vertex Reconstruction with Deep Learning in Pandora J. Ahmed. Second Year PhD Student.

Updates on Vertex Reconstruction with Deep Learning in Pandora J. Ahmed. Second Year PhD Student.

DUNE UK Meeting 11th of December 2019 Updates on Vertex Reconstruction with Deep Learning in Pandora J. Ahmed. Second Year PhD Student. Working on Pandora reconstruction Focusing on DUNE FD and ProtoDUNE. Supervisor: Dr John Marshall 1

549 views • 27 slides
Unsupervised Video Object Segmentation for Deep Reinforcement Learning Authors: Vik Goel,

Unsupervised Video Object Segmentation for Deep Reinforcement Learning Authors: Vik Goel,

Unsupervised Video Object Segmentation for Deep Reinforcement Learning Authors: Vik Goel, Jameson Weng, Pascal Poupart Presenter: Siliang Huang Outline Problem tackled Solution proposed RL background Architecture and

857 views • 30 slides
Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar George Arias Figueroa Advisor: Gerberth Ad n Ram rez Rivera Masters Thesis Defense February 19, 2018 Motivation Huge amount of unlabeled

1.26k views • 125 slides
Anomaly Detection and Categorization Using Unsupervised Deep Learning S6340 Thursday 7 th April

Anomaly Detection and Categorization Using Unsupervised Deep Learning S6340 Thursday 7 th April

Anomaly Detection and Categorization Using Unsupervised Deep Learning S6340 Thursday 7 th April 2016 GPU Technology Conference A. Stephen McGough , Noura Al Moubayed, Jonathan Cumming, Eduardo Cabrera, Peter Matthews, Toby P . Breckon, Ed

599 views • 29 slides
Unsupervised Deep Learning Tutorial Part 1 Alex Graves MarcAurelio Ranzato NeurIPS, 3

Unsupervised Deep Learning Tutorial Part 1 Alex Graves MarcAurelio Ranzato NeurIPS, 3

Unsupervised Deep Learning Tutorial Part 1 Alex Graves MarcAurelio Ranzato NeurIPS, 3 December 2018 Part 1 Alex Graves Introduction to unsupervised learning Autoregressive models Representation learning

1.47k views • 90 slides
Randomness in Computing L ECTURE 14 Last time Poisson distribution Poisson approximation

Randomness in Computing L ECTURE 14 Last time Poisson distribution Poisson approximation

Randomness in Computing L ECTURE 14 Last time Poisson distribution Poisson approximation Today Review Poisson distribution Poisson approximation Application: max load Application: Coupon Collector 3/17/2020 Sofya

357 views • 10 slides
Applying Deep Neural Network Techniques for LArTPC Data Reconstruction Laura Domine (Stanford

Applying Deep Neural Network Techniques for LArTPC Data Reconstruction Laura Domine (Stanford

Applying Deep Neural Network Techniques for LArTPC Data Reconstruction Laura Domine (Stanford University / SLAC) Fermilab Machine Learning Group meeting - 11/7/18 Plan 1. LArTPC & Deep Learning 2. Examples of applications: UResNet &

585 views • 39 slides
Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning

Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning

Adaptive Incident Radiance Field Sampling and Reconstruction Using Deep Reinforcement Learning Yuchi Huo, Rui Wnag, Ruzhang Zheng, Hualin Xu, Hujun Bao, Sung-eui Yoon KAIST and CAD&CG Global Illumination Path Guiding Filtering

257 views • 23 slides
Addressing The False Negative Problem of Deep Learning MRI Reconstruction Models by Paper #28

Addressing The False Negative Problem of Deep Learning MRI Reconstruction Models by Paper #28

Addressing The False Negative Problem of Deep Learning MRI Reconstruction Models by Paper #28 Adversarial Attacks and Robust Training Kaiyang Cheng*, Francesco Caliv*, Rutwik Shah, Misung Han, Sharmila Majumdar, Valentina Pedoia Disclosure

286 views • 25 slides
UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
A Hybrid Deep Learning Approach For Chaotic Time Series Prediction Based On Unsupervised Feature

A Hybrid Deep Learning Approach For Chaotic Time Series Prediction Based On Unsupervised Feature

A Hybrid Deep Learning Approach For Chaotic Time Series Prediction Based On Unsupervised Feature Learning Norbert Ayine Agana Advisor: Abdollah Homaifar Autonomous Control & Information Technology Institute (ACIT), Departmenent of

379 views • 35 slides
Samaritans Purse Philippines BSFP ACHIEVEMENT Number of Children Screened 6-59 months in

Samaritans Purse Philippines BSFP ACHIEVEMENT Number of Children Screened 6-59 months in

Samaritans Purse Philippines BSFP ACHIEVEMENT Number of Children Screened 6-59 months in Ormoc 12,988 Number of Children Screened 6-59 months in Isabel 3,826 Number of Children Screened 6-59 months in Merida 2,896 CMAM

463 views • 11 slides
CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement

CAPES:Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement Learning Yan li, Kenneth Chang, Oceane Bel, Ethan L. Miller, Darrel D. E. Long Performance Tuning Tuning systems parameters for high

343 views • 15 slides

More recommend