generative modeling by estimating gradients of the data
play

Generative Modeling by Estimating Gradients of the Data Distribution - PowerPoint PPT Presentation

Jul 14, 2023 •614 likes •1.12k views

Generative Modeling by Estimating Gradients of the Data Distribution Yang Song, Stefano Ermon David Zimmerer Medical Image Analysis ( #MIA-san-mia ) DKFZ Author Division Generative modeling 02.11.16 | Page2 2 | Author Division

  1. Generative Modeling by Estimating Gradients of the Data Distribution Yang Song, Stefano Ermon David Zimmerer Medical Image Analysis ( #MIA-san-mia ) DKFZ

  2. Author Division Generative modeling 02.11.16 | Page2 2 |

  3. Author Division Generative modeling 02.11.16 | Page3 3 |

  4. Author Division Generative modeling 02.11.16 | Page4 4 |

  5. Author Division Generative modeling 02.11.16 | Page5 5 |

  6. Author Division Generative modeling 02.11.16 | Page6 6 |

  7. Author Division Generative modeling 02.11.16 | Page7 7 |

  8. Author Division Generative modeling 02.11.16 | Page8 8 |

  9. Author Division “New” Idea: 02.11.16 | Page9 Generative Modeling by Estimating Gradients of the Data Distribution 9 |

  10. Author Division “New” Idea: 02.11.16 | Page10 Generative Modeling by Estimating Gradients of the Data Distribution 10 |

  11. Author Division “New” Idea: 02.11.16 | Page11 Generative Modeling by Estimating Gradients of the Data Distribution 11 |

  12. Author Division “New” Idea: 02.11.16 | Page12 Generative Modeling by Estimating Gradients of the Data Distribution 12 |

  13. Author Division Score matching 02.11.16 | Page13 [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 13 |

  14. Author Division Score matching 02.11.16 | Page14 → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 14 |

  15. Author Division Score matching 02.11.16 | Page15 → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 15 |

  16. Author Division Score matching 02.11.16 | Page16 → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 16 |

  17. Author Division Score matching 02.11.16 | Page17 → → [1] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. 17 |

  18. Author Division Trivial Implementation (on MNIST) 02.11.16 | Page18 18 |

  19. Author Division Trivial Implementation (on MNIST) 02.11.16 | Page19 19 |

  20. Author Division Trivial Implementation (on MNIST) 02.11.16 | Page20 20 |

  21. Author Division Spoiler: Improved Results 02.11.16 | Page21 21 |

  22. Author Division Spoiler: Improved Results 02.11.16 | Page22 22 |

  23. Author Division Spoiler: Improved Results 02.11.16 | Page23 23 |

  24. Author Division Spoiler: Improved Results 02.11.16 | Page24 24 |

  25. Author Division What are the problems ? 02.11.16 | Page25 25 |

  26. Author Division What are the problems ? 02.11.16 | Page26 26 |

  27. Author Division What are the problems ? 02.11.16 | Page27 27 |

  28. Author Division What are the problems ? 02.11.16 | Page28 → 28 |

  29. Author Division What are the problems ? 02.11.16 | Page29 → 29 |

  30. Author Division What are the problems ? 02.11.16 | Page30 → 30 |

  31. Author Division What are the problems ? 02.11.16 | Page31 → → 31 |

  32. Author Division How to sample: 02.11.16 | Page32 32 |

  33. Author Division How to sample: 02.11.16 | Page33 33 |

  34. Author Division How to sample: 02.11.16 | Page34 34 |

  35. Author Division How to sample: 02.11.16 | Page35 35 |

  36. Author Division How to sample: 02.11.16 | Page36 36 |

  37. Author Division How to sample: 02.11.16 | Page37 37 |

  38. Author Division Approach 02.11.16 | Page38 38 |

  39. Author Division Approach 02.11.16 | Page39 39 |

  40. Author Division Results 02.11.16 | Page40 40 |

  41. Author Division Results: Qualitative 02.11.16 | Page41 41 |

  42. Author Division Results: Qualitative 02.11.16 | Page42 42 |

  43. Author Division Results: Quantitative 02.11.16 | Page43 43 |

  44. Author Division Results: Reproducible 02.11.16 | Page44 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 44 |

  45. Author Division Results: Reproducible 02.11.16 | Page45 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 45 |

  46. Author Division Results: Reproducible 02.11.16 | Page46 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 46 |

  47. Author Division Results: Reproducible 02.11.16 | Page47 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 47 |

  48. Author Division Results: Reproducible 02.11.16 | Page48 → [2] A. Matosevic. Reproducibility Challenge – Generative Modeling by Estimating Gradients of the Data Distribution, NeurIPS 2019 Reproducibility Challenge Blind Report, https://openreview.net/forum?id=SkxCSTqG6H 48 |

  49. Author Division The End 02.11.16 | Page49 49 |

Recommend

CoT: Cooperative Training for Generative Modeling of Discrete Data

CoT: Cooperative Training for Generative Modeling of Discrete Data

CoT: Cooperative Training for Generative Modeling of Discrete Data https://github.com/desire2020/CoT Sidi Lu, Lantao Yu, Siyuan Feng, Yaoming Zhu, Weinan Zhang, and Yong Yu Shanghai Jiao Tong University Autoregressive Models Autoregressive

399 views • 16 slides
Modeling Velocity Gradients in an OBC, First-Break Positioning Algorithm Noel Zinn Western

Modeling Velocity Gradients in an OBC, First-Break Positioning Algorithm Noel Zinn Western

Modeling Velocity Gradients in an OBC, First-Break Positioning Algorithm Noel Zinn Western Geophysical EAGE Geneva 1997 In this talk I provide an overview of the modeling of vertical and lateral velocity gradients that can be sources of

403 views • 25 slides
Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation)

Conditional Generative Adversarial Networks (and a brief look at image-to-image translation) Final Presentation Peter Bromley Generative Models What is generative modeling? Data: Samples from high dimensional probability distribution p real

621 views • 25 slides
Mining for Cost Estimating Relations from Limited Complex Data Modeling Approaches for NASA

Mining for Cost Estimating Relations from Limited Complex Data Modeling Approaches for NASA

Mining for Cost Estimating Relations from Limited Complex Data Modeling Approaches for NASA Robotic Earth/Space Science Projects 2015 ICEAA Workshop 6/11/15 Shawn Hayes and Mark Jacobs Victory Solutions MIPSS Team 1 OUTLINE 1. Introduction

537 views • 36 slides
Dealing with Data Gradients: Backing Out & Calibration Nathaniel Osgood Agent-Based

Dealing with Data Gradients: Backing Out & Calibration Nathaniel Osgood Agent-Based

Dealing with Data Gradients: Backing Out & Calibration Nathaniel Osgood Agent-Based Modeling Bootcamp for Health Researchers August 24, 2011 A Key Deliverable! Reference mode Specification & Learning Parameter sensitivity

665 views • 51 slides
for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

Deep Generative Modeling for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text Praveen Narayanan Ford Motor Company PROJECT DESCRIPTION Use of DNNs increasingly prevalent as a solution for many data

893 views • 53 slides
Vanishing and exploding gradients RECURREN T N EURAL N ETW ORK S F OR LAN GUAGE MODELIN G IN P

Vanishing and exploding gradients RECURREN T N EURAL N ETW ORK S F OR LAN GUAGE MODELIN G IN P

Vanishing and exploding gradients RECURREN T N EURAL N ETW ORK S F OR LAN GUAGE MODELIN G IN P YTH ON David Cecchini Data Scientist Training RNN models RECURRENT NEURAL NETWORKS FOR LANGUAGE MODELING IN PYTHON Example: a = f ( W , a , x )

623 views • 28 slides
Statistical modeling responses of bioassessment indices to eutrophication stressor gradients

Statistical modeling responses of bioassessment indices to eutrophication stressor gradients

Statistical modeling responses of bioassessment indices to eutrophication stressor gradients Presentation to Stakeholder WG Friday, March 17, 2017 Objectives Support decisions on numeric guidance (e.g., a numeric objective) for

441 views • 25 slides
Flat Metric Minimization with Applications in Generative Modeling Thomas M ollenhoff Daniel

Flat Metric Minimization with Applications in Generative Modeling Thomas M ollenhoff Daniel

Flat Metric Minimization with Applications in Generative Modeling Thomas M ollenhoff Daniel Cremers Motivation Latent concepts often induce an orientation of the data. 1/12 Simard et al. 1992, 1998; Rifai et al. 2011 Motivation Latent

855 views • 49 slides
Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo Dai 1 , Dale Schuurmans 1 1 Google Brain, 2 DeepMind Graph generative models Given a set of graphs {G 1 , G 2 , , G N }, fit a probabilistic model

516 views • 39 slides
Blended Conditional Gradients: The unconditioning of conditional gradients Joint work with Gabor

Blended Conditional Gradients: The unconditioning of conditional gradients Joint work with Gabor

Blended Conditional Gradients: The unconditioning of conditional gradients Joint work with Gabor Braun, Sebastian Pokutta, Steve Wright Dan Tu 1/9 CONDITIONAL GRADIENTS: PROJECTION-FREE Given a polytope ! , solve the optimization problem: min

109 views • 9 slides
An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative

An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative

An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative modeling Boosting Alternating decision trees Boosting and over-fitting Applications 2 Toy Example Computer receives telephone call

735 views • 61 slides
Modeling Heterogeneous Statistical Patterns in High- dimensional Data by Adversarial

Modeling Heterogeneous Statistical Patterns in High- dimensional Data by Adversarial

Modeling Heterogeneous Statistical Patterns in High- dimensional Data by Adversarial Distributions: An Unsupervised Generative Framework (FIRD) Han Zhang 1 Wenhao Zheng 3 Charley Chen 1 Kevin Gao 1 Yao Hu 3 Ling Huang 2 Wei Xu 1 1 Tsinghua

409 views • 30 slides
HACK TB Estimating subnational TB burden in Pakistan Kate LeGrand, MPH, CPH Jennifer Ross, MD,

HACK TB Estimating subnational TB burden in Pakistan Kate LeGrand, MPH, CPH Jennifer Ross, MD,

HACK TB Estimating subnational TB burden in Pakistan Kate LeGrand, MPH, CPH Jennifer Ross, MD, MPH January 7, 2020 Overview 1. Tuberculosis epidemiology 2. Sources of data 3. Data challenges in the TB landscape 4. IHME geospatial modeling

345 views • 16 slides
Estimating Daily Streamlow at Ungaged Sites in NH: A Modeling Update Neil M. Fennessey

Estimating Daily Streamlow at Ungaged Sites in NH: A Modeling Update Neil M. Fennessey

Estimating Daily Streamlow at Ungaged Sites in NH: A Modeling Update Neil M. Fennessey HYSR/Hydrologic Services HYSRConsult@gmail.com NH Department of Environmental Services Concord, NH March 29, 2018 Estimating Daily Flows at Ungaged Sites

533 views • 32 slides
LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David

LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David

LEARNING GENERATIVE MODELS ACROSS INCOMPARABLE SPACES Cha harlot otte Bunne unne , David Alvarez-Melis, Andreas Krause, Stefanie Jegelka Pos oster #173 173 Poster #173 173 & Charlotte Bunne Generative Modeling generative = network

250 views • 13 slides
Estimating Financial Risk through Monte Carlo Simulation Modeling Value at Risk (VaR) with

Estimating Financial Risk through Monte Carlo Simulation Modeling Value at Risk (VaR) with

Estimating Financial Risk through Monte Carlo Simulation Modeling Value at Risk (VaR) with Linear Regression Under Normal Distribution Assumption Outline - What Are We Getting Into? - Basic Terms - Monte Carlo Risk Modeling - Results /

1.46k views • 41 slides
Generative modeling - Electromagnetic shower of a calorimeter Paul KLEIN 24 th of April 2017

Generative modeling - Electromagnetic shower of a calorimeter Paul KLEIN 24 th of April 2017

Generative modeling - Electromagnetic shower of a calorimeter Paul KLEIN 24 th of April 2017 Issue : Simulations (Geant4 Package) are time consuming and CPU intensive REAL LIFE SIMULATION VS ~ O(10 9 ) events/year ~ O(0,1) event/min ~

509 views • 23 slides
Tensor Networks for Generative Modeling From Boltzmann machines to Born machines, and back ang (

Tensor Networks for Generative Modeling From Boltzmann machines to Born machines, and back ang (

Tensor Networks for Generative Modeling From Boltzmann machines to Born machines, and back ang ( ) Lei W https://wangleiphy.github.io Institute of Physics, Beijing Han, Wang, Fan, LW, Zhang, PRX18 Chen, Cheng, Xie, LW, Xiang, PRB

825 views • 80 slides
Analysis-by-Synthesis a.k.a Generative Modeling Tejas D Kulkarni (tejask@mit.edu) Monday,

Analysis-by-Synthesis a.k.a Generative Modeling Tejas D Kulkarni (tejask@mit.edu) Monday,

Analysis-by-Synthesis a.k.a Generative Modeling Tejas D Kulkarni (tejask@mit.edu) Monday, October 19, 15 Traditional paradigm for AI research Traditional machine learning and pattern recognition has been remarkable successful at questions

1k views • 61 slides
Outline Last time Image gradients Seam carving gradients as energy Edges

Outline Last time Image gradients Seam carving gradients as energy Edges

CS 376 Spring 2018 - Lecture 4 1/30/2018 Outline Last time Image gradients Seam carving gradients as energy Edges and binary images Today Tues Jan 30, 2018 Gradients edges and contours Kristen Grauman

647 views • 20 slides
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling Jiajun Wu* Chengkai Zhang* Tianfan Xue Bill Freeman Josh Tenenbaum NIPS 2016 (* indicates equal contribution) Outline Synthesizing 3D shapes

967 views • 65 slides
Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference

Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference

Generative Adversarial Networks Ian Goodfellow Research Scientist GPU Technology Conference San Jose, California 2016-04-05 Generative Modeling Have training examples: x p train ( x ) Want a model that can draw samples: x p

317 views • 16 slides
Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative Networks Amir A. Soltani Haibin Huang Jiajun Wu Tejas Kulkarni Josh Tenenbaum Samples Out-of-Sample Generalization 07/21/2017 Motivation -

783 views • 55 slides

More recommend