Bias and Generalization in Deep Generative Models Shengjia Zhao*, - PowerPoint PPT Presentation

Feb 22, 2023 •143 likes •350 views

Bias and Generalization in Deep Generative Models Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman and Stefano Ermon *equal contribution Success in Generative Modeling of Images Brock A, et al. "Large scale gan

Bias and Generalization in Deep Generative Models Shengjia Zhao*, Hongyu Ren*, Arianna Yuan, Jiaming Song, Noah Goodman and Stefano Ermon *equal contribution
Success in Generative Modeling of Images Brock A, et al. "Large scale gan training for high fidelity natural image synthesis."
Goal: Understanding Generalization How do generative models generalize?
Generalization Example: Object Count
Empirical Study of Generalization: Method • Design datasets • Train generative models (VAE, GAN, PixelCNN) • Observe generalization behavior • Find common patterns
Generalization Example: Object Count
Generalization in Feature Space: Numerosity Generates a log-normal Frequency Frequency shaped distribution 2 1 2 3 4 # Objects # Objects Training Distribution Generated Distribution (Observed)
Multiple Numerosities Frequency ? 2 7 # Objects Training Distribution
Multiple Numerosities: Only 2 Frequency Frequency 2 7 # Objects 1 2 3 4 # Objects Training Distribution Generated Distribution
Multiple Numerosities: Only 7 Frequency Frequency 2 7 # Objects 6 7 8 9 # Objects Training Distribution Generated Distribution
Multiple Numerosities: Additive Hypothesis Frequency Frequency 2 7 # Objects 1 2 3 4 6 7 8 9 # Objects Training Distribution Generated Distribution (Observed)
Additive Hypothesis with 2 and 4 Objects Frequency Frequency 2 4 1 2 3 4 5 6 # Objects # Objects Training Distribution Generated Distribution ( Hypothesized )
Actual Result: Prototype Enhancement 3 objects most likely, even though no training Frequency Frequency image contains 3 objects! 2 4 1 2 3 4 5 6 # Objects # Objects Training Distribution Generated Distribution ( Observed )
Prototype Enhancement Similar pattern for other features: Frequency color , size , location Frequency 2 4 1 2 3 4 5 6 # Objects # Objects Training Distribution Generated Distribution ( Observed )
Multiple Features
Memorization vs. Generalization
Memorization vs. Generalization
Different Setups, Similar Results - Different features (shape, color, size, numerosity, etc.) - Different models: (VAE, GAN, PixelCNN, etc.) - Different architectures (fully connected, convolutional, etc.) - Different hyper-parameters (network size, learning rate, etc.)
Conclusion • New methodology: design datasets to probe generative models • Observed common patterns across different setups Welcome to our poster session for further discussions! Tuesday 5-7pm @ Room 210 & 230 AB #6 Code available at github.com/ermongroup/BiasAndGeneralization

Recommend

Invertible Generative Models for Inverse Problems Mitigating Representation Error and Dataset Bias

Invertible Generative Models for Inverse Problems Mitigating Representation Error and Dataset Bias M. Asim, M. Daniels, O. Leong, P . Hand, A. Ahmed Inverse Problems with Generative Models as Image Priors Inverse Problems with Generative Models

369 views • 22 slides

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu <

Probability Functional Descent: A Unifying Perspective on GANs, VI, and RL Casey Chu < caseychu@stanford.edu > Jose Blanchet Peter Glynn Deep generative models Deep generative models Variational inference Deep generative models

275 views • 16 slides

Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van

Deep Generative models for Inverse Problems Alex Dimakis joint work with Ashish Bora, Dave Van Veen and Ajil Jalal, Sriram Vishwanath and Eric Price, UT Austin Outline Generative Models Using generative models for Inverse

1.17k views • 101 slides

Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan

Introduction Variational Inference Deep Generative Models Summary Learning Deep Generative Models Inference & Representation Lecture 12 Rahul G. Krishnan Fall 2015 Rahul G. Krishnan Learning Deep Generative Models Introduction

1.22k views • 83 slides

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders

Deep-Learning: Unsupervised Generative models Deep Belief Networks Deep Stacked AutoEncoders Generative Adversarial Networks Pr. Fabien MOUTARDE Center for Robotics MINES ParisTech PSL Universit Paris Fabien.Moutarde@mines-paristech.fr

777 views • 23 slides

Structure: Exposition General problem Purpose / stated goal(s) Experimental setup

Structure: Exposition General problem Purpose / stated goal(s) Experimental setup summary Result summary 1 Experimental Evaluation: Bias and Generalization in Deep Generative Models Nicholay Topin MLD, Carnegie Mellon

352 views • 22 slides

Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and

A New Framework For Hybrid Models An Application: Deep Hybrid Models Supervised and Semi-Supervised Experiments Deep Hybrid Models: Bridging Discriminative and Generative Approaches Volodymyr Kuleshov and Stefano Ermon Department of Computer

523 views • 29 slides

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides

Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey

Variational Inference and Generative Models CS 294-112: Deep Reinforcement Learning Sergey Levine Class Notes 1. Homework 3 due next Wednesday 2. Accept CMT peer review invitations These are required (part of your final project grade)

652 views • 34 slides

Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew

Generating & Designing DNA with Deep Generative Models Nathan Killoran, Leo J. Lee, Andrew Delong, David Duvenaud, Brendan J. Fey Presented by Yomna Omar on February 16th, 2018 What is DNA sequencing? The process of determining the order

533 views • 22 slides

What deep generative models can do for you: Opportunities, challenges, and open questions Giulia

What deep generative models can do for you: Opportunities, challenges, and open questions Giulia Fanti Carnegie Mellon University 1 Kiran Zinan Lin Hao Liang Alankar Jain Thekumparampil Chen Wang Sewoong Oh Vyas Sekar 2 Classifying

540 views • 51 slides

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept.

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept. of Neuroradiology, Heidelberg University Hospital Div. of Medical Image Computing, DKFZ Heidelberg Faculty of Physics & Astronomy, Heidelberg

1.48k views • 65 slides

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar

Deep Generative Models for Clustering: A Semi-supervised and Unsupervised Approach Jhosimar George Arias Figueroa Advisor: Gerberth Ad n Ram rez Rivera Masters Thesis Defense February 19, 2018 Motivation Huge amount of unlabeled

1.26k views • 125 slides

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative

Synthesizing 3D Shapes via Modeling Multi-View Depth Maps and Silhouettes with Deep Generative Networks Amir A. Soltani Haibin Huang Jiajun Wu Tejas Kulkarni Josh Tenenbaum Samples Out-of-Sample Generalization 07/21/2017 Motivation -

783 views • 55 slides

Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir

Semi-supervised Learning with Deep Generative Models Diedrik P. Kingma, Danilo J. Rezende, Shakir Mohamed, Max Welling What is Deep Learning very good at? Classifying highly structured data -ImageNet -Part of Speech Tagging -MNIST

195 views • 18 slides

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490

Computational Systems Biology Deep Learning in the Life Sciences 6.802 6.874 20.390 20.490 HST.506 David Gifford Lecture 9 March 5, 2020 Generative Models http://mit6874.github.io 1 Why generative models? We can sample new examples from a

747 views • 52 slides

Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted

Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining Erik Daxberger*, Austin Tripp* & Jos e Miguel Hern andez-Lobato RealML @ ICML2020 2020-07-18 Problem Optimization of expensive,

292 views • 4 slides

Image Restoration with Deep Generative Models Raymond A. Yeh * , Teck-Yian Lim * , Chen Chen,

Image Restoration with Deep Generative Models Raymond A. Yeh * , Teck-Yian Lim * , Chen Chen, Alexander G. Schwing, Mark Hasegawa-Johnson, Minh N. Do Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign

354 views • 34 slides

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November

Making Generative Classifiers Robust to Selection Bias Andrew Smith Charles Elkan November 30th, 2007 Outline What is selection bias? Types of selection bias. Overcoming learnable bias with weighting. Overcoming bias with

736 views • 59 slides

Variational Autoencoders + Deep Generative Models Matt Gormley Lecture 27 Dec. 4, 2019 1

10-418 / 10-618 Machine Learning for Structured Data Machine Learning Department School of Computer Science Carnegie Mellon University Variational Autoencoders + Deep Generative Models Matt Gormley Lecture 27 Dec. 4, 2019 1 Reminders

989 views • 86 slides

On Casting Importance Weighted Autoencoder to an EM Algorithm to Learn Deep Generative Models

On Casting Importance Weighted Autoencoder to an EM Algorithm to Learn Deep Generative Models D.Kim 1 and J.Hwang 2 and Y.Kim 1 Speaker : Dongha Kim 1 Department of Statistics, Seoul National University, South Korea 2 SK Telecom, South Korea

671 views • 31 slides

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models S.M. Eslami,N. Heess, T.

Attend, Infer, Repeat: Fast Scene Understanding with Generative Models S.M. Eslami,N. Heess, T. Weber, Y. Tassa, D. Szepesvari, K.Kavukcuoglu, G. E. Hinton Nicolas Brandt nbrandt@cs.toronto.edu Origins Structured generative methods : Deep

1.13k views • 19 slides

Generative Adversarial Nets(GANs) Troy Cary and Chenzhi Zhao A generative adversarial net is

Generative Adversarial Nets(GANs) Troy Cary and Chenzhi Zhao A generative adversarial net is a type of neural net, used in deep learning/machine learning problems The goal of a GAN is to train two simultaneous models: a generative model

441 views • 18 slides

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo

Scalable Deep Generative Modeling for Sparse Graphs Hanjun Dai 1 , Azade Nazi 1 , Yujia Li 2 , Bo Dai 1 , Dale Schuurmans 1 1 Google Brain, 2 DeepMind Graph generative models Given a set of graphs {G 1 , G 2 , , G N }, fit a probabilistic model

516 views • 39 slides