4 deep generative models
play

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning - PowerPoint PPT Presentation

Nov 19, 2023 •813 likes •1.48k views

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept. of Neuroradiology, Heidelberg University Hospital Div. of Medical Image Computing, DKFZ Heidelberg Faculty of Physics & Astronomy, Heidelberg

  1. 4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept. of Neuroradiology, Heidelberg University Hospital Div. of Medical Image Computing, DKFZ Heidelberg Faculty of Physics & Astronomy, Heidelberg University

  2. Challenges in MIC 2 Data Shortage Transfer learning Noisy labels and data 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  3. Basic Principle of Generative Models 3 Assumption Z Observations X generated from latent variables Z via mapping f f(x|z) X 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  4. Basic Principle of Generative Models 4 Assumption Z Observations X generated from latent variables Z via mapping f f(x|z) Goal • Be able to generate more samples that follow distribution of X X • Z interpretable in some way 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  5. Basic Principle of Deep Generative Models 5 Z f(x|z) X [pexels.com, pixabay.com, pngimg.com] 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  6. Basic Principle of Deep Generative Models 6 Realism Z Panda-ness TM f(x|z) X [pexels.com, pixabay.com, pngimg.com] 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  7. Generative Adversarial Networks

  8. 8 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  9. 9 [https://twitter.com/goodfellow_ian] 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  10. Basic GAN Layout 10 [https://deeplearning4j.org/generative-adversarial-network] [1] Generative Adversarial Networks , Goodfellow et al., 2014, NIPS 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  11. GAN Learning Objective 11 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  12. GAN Learning Objective 12 D(real) → 1 D(fake) → 0 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  13. GAN Learning Objective 13 D(real) → 1 D(fake) → 0 D(fake) → 1 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  14. GAN Learning Objective 14 D(real) → 1 D(fake) → 0 D(fake) → 1 • Trying to find saddle point → Very hard to optimize • Lot of work on different objectives and „tricks“ for training [2] Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks Radford et al., 2015, arXiv:1511.06434 [3] Are GANs Created Equal? A Large Scale Study , Lucic et al., 2017, arXiv:1711.10337 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  15. Original Examples 15 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  16. Important Concepts Conditional GAN 16 General case Generative models make no default assumptions for p(z) → Could be random noise and/or real data 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  17. Important Concepts Conditional GAN 17 [4] Adversarial Networks for the Detection of Aggressive Prostate Cancer , Kohl et al., 2017, NIPS Workshop 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  18. Important Concepts Conditional GAN 18 [4] Adversarial Networks for the Detection of Aggressive Prostate Cancer , Kohl et al., 2017, NIPS Workshop 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  19. Important Concepts CycleGAN 19 Assumption Have two unpaired sets A,B of images with some set- specific characteristic (e.g. photos & paintings) Goal Be able to transform image so it looks like images in different set 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  20. Important Concepts CycleGAN 20 Assumption Have two unpaired sets A,B of images with some set- specific characteristic (e.g. photos & paintings) Goal Be able to transform image so it looks like images in different set Naive Approach GANs that take images from A(B) and create images that similar to others from B(A) 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  21. Important Concepts CycleGAN 21 Assumption Have two unpaired sets A,B of images with some set- specific characteristic (e.g. photos & paintings) Goal Be able to transform image so it looks like images in different set Naive Approach GANs that take images from A(B) and create images that similar to others from B(A) → no guarantee that output looks similar to input 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  22. Important Concepts CycleGAN 22 [5] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , Zhu et al., 2017, arXiv:1703.10593 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  23. Important Concepts CycleGAN 23 L1-Norm Cycle consistency loss [5] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , Zhu et al., 2017, arXiv:1703.10593 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  24. Important Concepts CycleGAN 24 [5] Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , Zhu et al., 2017, arXiv:1703.10593 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  25. Examples Progressive Growing 25 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  26. Examples Progressive Growing 26 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  27. Examples Progressive Growing 27 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  28. Examples Progressive Growing 28 Samples Nearest Neighbours 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  29. Image Similarity 29 • Pixel similarity • mean squared error (= L2 norm) • other norms 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  30. Image Similarity 30 • Pixel similarity • mean squared error (= L2 norm) • other norms • Semantic similarity • Inception score (score for entire model) • Combined distance of multiple feature layers in discriminator • Human evaluation (e.g. Mechanical Turk) 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  31. Examples MRI to CT Image Synthesis 31 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  32. Examples MRI to CT Image Synthesis 32 FCN architecture 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  33. Examples MRI to CT Image Synthesis 33 FCN architecture Combined adversarial & MSE loss 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  34. Examples MRI to CT Image Synthesis 34 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  35. Examples Domain Transfer for Lesion Segmentation 35 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  36. Examples Domain Transfer for Lesion Segmentation 36 Assumption (X, Y) in source domain, (X*) in target domain 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  37. Examples Domain Transfer for Lesion Segmentation 37 Assumption (X, Y) in source domain, (X*) in target domain ... + GE + Lesion Segmentation in source ... + SWI in target Goal Segmentation in target domain 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  38. Examples Domain Transfer for Lesion Segmentation 38 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  39. Examples Domain Transfer for Lesion Segmentation 39 DeepMedic architecture 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  40. Examples Domain Transfer for Lesion Segmentation 40 DeepMedic architecture Auxiliary adversarial loss ensures domain invariant feature maps 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  41. Examples Domain Transfer for Lesion Segmentation 41 Higher is better 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  42. Summary GANs 42 High-quality, high-resolution outputs possible ✔ ✔ Adversarial training extremely versatile Difficult to train ✖ No inference (latent representation from data) ✖ 11.3.2018 l Deep Generative Models l Jens Petersen, Div. of Medical Image Computing

  43. Variational Autoencoders

Recommend

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu <

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu <

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu < caseychu@stanford.edu > Jose Blanchet Peter Glynn Deep generative models Deep generative models Variational inference Deep generative models

275 views • 16 slides
Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van

Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van

Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van Veen and Ajil Jalal, Sriram Vishwanath and Eric Price, UT Austin Outline Generative Models Using generative models for Inverse

1.17k views • 101 slides
Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan

Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan

Introduction Variational Inference Deep Generative Models Summary Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan Fall 2015 Rahul G. Krishnan Learning Deep Generative Models Introduction

1.22k views • 83 slides
Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders Generative Adversarial Networks Pr. Fabien MOUTARDE Center for Robotics MINES ParisTech PSL Universit Paris Fabien.Moutarde@mines-paristech.fr

777 views • 23 slides
Bias and Generalization in Deep Generative Models Shengjia Zhao*, Hongyu Ren*, Arianna Yuan,

Bias and Generalization in Deep Generative Models Shengjia Zhao*, Hongyu Ren*, Arianna Yuan,

Bias and Generalization in Deep Generative Models Shengjia Zhao*, Hongyu Ren*, Arianna Yuan, Jiaming Song, Noah Goodman and Stefano Ermon *equal contribution Success in Generative Modeling of Images Brock A, et al. "Large scale gan

350 views • 19 slides
Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and

Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and

A New Framework For Hybrid Models An Application: Deep Hybrid Models Supervised and Semi-Supervised Experiments Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and Stefano Ermon Department of Computer

523 views • 29 slides
A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides
Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey

Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey

Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 due next Wednesday 2. Accept CMT peer review invitations These are required (part of your final project grade)

652 views • 34 slides
Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew

Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew

Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew Delong, David Duvenaud, Brendan J. Fey Presented by Yomna Omar on February 16th, 2018 What is DNA sequencing? The process of determining the order

533 views • 22 slides
What deep generative models can do for you: Opportunities, challenges, and open questions Giulia

What deep generative models can do for you: Opportunities, challenges, and open questions Giulia

What deep generative models can do for you: Opportunities, challenges, and open questions Giulia Fanti Carnegie Mellon University 1 Kiran Zinan Lin Hao Liang Alankar Jain Thekumparampil Chen Wang Sewoong Oh Vyas Sekar 2 Classifying

540 views • 51 slides
Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar George Arias Figueroa Advisor: Gerberth Ad n Ram rez Rivera Masters Thesis Defense February 19, 2018 Motivation Huge amount of unlabeled

1.26k views • 125 slides
Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir

Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir

Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir Mohamed, Max Welling What is Deep Learning very good at? Classifying highly structured data -ImageNet -Part of Speech Tagging -MNIST

195 views • 18 slides
Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490 HST.506 David Gifford Lecture 9 March 5, 2020 Generative Models http://mit6874.github.io 1 Why generative models? We can sample new examples from a

747 views • 52 slides
Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted

Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted

Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining Erik Daxberger*, Austin Tripp* & Jos e Miguel Hern andez-Lobato RealML @ ICML2020 2020-07-18 Problem Optimization of expensive,

292 views • 4 slides
Image Restoration with Deep Generative Models Raymond A. Yeh * , Teck-Yian Lim * , Chen Chen,

Image Restoration with Deep Generative Models Raymond A. Yeh * , Teck-Yian Lim * , Chen Chen,

Image Restoration with Deep Generative Models Raymond A. Yeh * , Teck-Yian Lim * , Chen Chen, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign

354 views • 34 slides
Variational Autoencoders + Deep Generative Models Matt Gormley Lecture 27 Dec. 4, 2019 1

Variational Autoencoders + Deep Generative Models Matt Gormley Lecture 27 Dec. 4, 2019 1

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Variational Autoencoders + Deep Generative Models Matt Gormley Lecture 27 Dec. 4, 2019 1 Reminders

989 views • 86 slides
On Casting Importance Weighted Autoencoder to an EM Algorithm to Learn Deep Generative Models

On Casting Importance Weighted Autoencoder to an EM Algorithm to Learn Deep Generative Models

On Casting Importance Weighted Autoencoder to an EM Algorithm to Learn Deep Generative Models D.Kim 1 and J.Hwang 2 and Y.Kim 1 Speaker : Dongha Kim 1 Department of Statistics, Seoul National University, South Korea 2 SK Telecom, South Korea

671 views • 31 slides
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models S.M. Eslami,N. Heess, T.

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models S.M. Eslami,N. Heess, T.

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models S.M. Eslami,N. Heess, T. Weber, Y. Tassa, D. Szepesvari, K.Kavukcuoglu, G. E. Hinton Nicolas Brandt nbrandt@cs.toronto.edu Origins Structured generative methods : Deep

1.13k views • 19 slides
Generative Adversarial Nets(GANs) Troy Cary and Chenzhi Zhao A generative adversarial net is

Generative Adversarial Nets(GANs) Troy Cary and Chenzhi Zhao A generative adversarial net is

Generative Adversarial Nets(GANs) Troy Cary and Chenzhi Zhao A generative adversarial net is a type of neural net, used in deep learning/machine learning problems The goal of a GAN is to train two simultaneous models: a generative model

441 views • 18 slides
Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo Dai 1 , Dale Schuurmans 1 1 Google Brain, 2 DeepMind Graph generative models Given a set of graphs {G 1 , G 2 , , G N }, fit a probabilistic model

516 views • 39 slides
Generative Models I Ian Goodfellow, Sta ff Research Scientist, Google Brain MILA Deep Learning

Generative Models I Ian Goodfellow, Sta ff Research Scientist, Google Brain MILA Deep Learning

Generative Models I Ian Goodfellow, Sta ff Research Scientist, Google Brain MILA Deep Learning Summer School Montral, Qubec 2017-06-27 Density Estimation (Goodfellow 2017) Sample Generation Training examples Model samples (Goodfellow

921 views • 57 slides
Introduction to Deep Models Part I: Classifiers and Generative Networks Nick Winovich Department

Introduction to Deep Models Part I: Classifiers and Generative Networks Nick Winovich Department

TensorFlow Workshop 2018 Introduction to Deep Models Part I: Classifiers and Generative Networks Nick Winovich Department of Mathematics Purdue University July 2018 SIAM@Purdue 2018 - Nick Winovich Introduction to Deep Models : Part I

554 views • 28 slides
Event Generation and Statistical Sampling with Deep Generative Models Rob Verheyen Introduction

Event Generation and Statistical Sampling with Deep Generative Models Rob Verheyen Introduction

Event Generation and Statistical Sampling with Deep Generative Models Rob Verheyen Introduction Event generation is really hard! 2 Introduction Can we use deep neural networks to do event generation? Possible applications: Faster

685 views • 23 slides
Deep Learning Generative Models in Wireless Networks Wireless AI Innovation @ Verizon (WAIV)

Deep Learning Generative Models in Wireless Networks Wireless AI Innovation @ Verizon (WAIV)

Deep Learning Generative Models in Wireless Networks Wireless AI Innovation @ Verizon (WAIV) March 2019 Confidential and proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or Confidential and

575 views • 34 slides

More recommend