Lecture 20: GANS CS109B Data Science 2 Pavlos Protopapas and Mark Glickman 1
Outline Review of AE and VAE GANS Motivation Formalism Training Game Theory, minmax Challenges: • Big Samples • Modal collapse CS109B, P ROTOPAPAS , G LICKMAN 2
Outline Review of AE and VAE GANS Motivation Formalism Training Game Theory, minmax Challenges: • Big Samples • Modal collapse CS109B, P ROTOPAPAS , G LICKMAN 3
Generating Data (is exciting) CS109B, P ROTOPAPAS , G LICKMAN 4 https://arxiv.org/pdf/1708.05509.pdf
Generating Data (is exciting) CS109B, P ROTOPAPAS , G LICKMAN 5
Autoencoder ( 𝒋 𝟑 We train the two networks by minimizing the reconstruction loss function: 𝓜 = ∑ 𝒚 𝒋 − 𝒚 x 𝑦 , Latent Space Z DECODER ENCODER This is an autoencoder. It gets that name because it automatically finds the best way to encode the input so that the decoded version is as close as possible to the input. CS109B, P ROTOPAPAS , G LICKMAN 6
CS109B, P ROTOPAPAS , G LICKMAN 7
CS109B, P ROTOPAPAS , G LICKMAN 8
Variational Autoencoders CS109B, P ROTOPAPAS , G LICKMAN 9
Variational Autoencoders CS109B, P ROTOPAPAS , G LICKMAN 10
Example code (could help a lot for HW7) https://github.com/Harvard-IACS/2019-computefest/tree/master/Wednesday/auto_encoder CS109B, P ROTOPAPAS , G LICKMAN 11
Generating Data We saw how to generate new data with a VAE in Lecture 19. CS109B, P ROTOPAPAS , G LICKMAN 12
Outline Review of AE and VAE GANS Motivation Formalism Training Game Theory, minmax Challenges: • Big Samples • Modal collapse CS109B, P ROTOPAPAS , G LICKMAN 13
Generating Data In this lecture we’re going to look at a completely different approach to generating data that is like the training data. This technique lets us generate types of data that go far beyond what a VAE offers. Generative Adversarial Network , or GAN . It’s based on a smart idea where two different networks are put against one another, with the goal of getting one network to create new samples that are different from the training data, but are so close that the other network can’t tell which are synthetic and which belong to the original training set. CS109B, P ROTOPAPAS , G LICKMAN 14
Generative models Imagine we want to generate data from a distribution, x ∼ p ( x ) e.g. x ∼ N ( µ, σ ) CS109B, P ROTOPAPAS , G LICKMAN
Generative models But how do we generate such samples? z ∼ Unif(0 , 1) CS109B, P ROTOPAPAS , G LICKMAN
Generative models But how do we generate such samples? z ∼ Unif(0 , 1) x = ln z CS109B, P ROTOPAPAS , G LICKMAN
Generative models In other words we can think that if we choose z ~𝑽𝒐𝒋𝒈𝒑𝒔𝒏 then there is a mapping: x = f ( z ) such as: x ∼ p ( x ) where in general 𝑔 is some complicated function. We already know that Neural Networks are great in learning complex functions. CS109B, P ROTOPAPAS , G LICKMAN
Generative models We would like to construct our generative model, which we would like to train to generate lightcurves like these from scratch . A generative model in this case could be one large neural network that outputs lightcurves: samples from the model . Markov Boltzmann Machines Chain Deep Belief Networks Explicit Density Variational Variational Deep Autoencoders (VAE) Generative Markov Generative Adversarial Models Chain Networks (GAN) Implicit Density Generative Moment Direct Matching Networks (GMM) CS109B, P ROTOPAPAS , G LICKMAN
Adversarial 2 Athletes: Messi and Ronaldo. Adversaries! CS109B, P ROTOPAPAS , G LICKMAN 20
Generative Adversarial Networks (GANs) David Gary Spam Filterer Marketer Yes, it is spam Spam No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 21
Generative Adversarial Networks (GANs) Discarded a valid email David checks the results Spam Spam Spam Spam Yes, it is spam Spam Spam No, it is not spam Allow some spams CS109B, P ROTOPAPAS , G LICKMAN 22
Generative Adversarial Networks (GANs) David and Gary It was spam, for real It was not spam learned what went wrong Spam Spam Yes, it is spam Spam Spam Spam Spam No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 23
Generative Adversarial Networks (GANs) Gary David Find more sophisticated words The email contains than spam the word spam Yes, it is spam No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 24
Generative Adversarial Networks (GANs) Discarded a valid email David and Gary learned what went Yes, it is spam wrong No, it is not spam Allow fewer spams CS109B, P ROTOPAPAS , G LICKMAN 25
Generative Adversarial Networks (GANs) David It was spam, for real It was not spam learned what went wrong Yes, it is spam No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 26
Generative Adversarial Networks (GANs) Understanding confusion matrix through an example label: it was not spam False Positive False Positive true false positive TP FP prediction Yes or True M + = matches label? prediction: true false TN negative FN yes, it is spam No or False SPAM: POSITIVE TRUE/FALSE: If prediction and true match do not match POSITIVE/NEGATIVE: Prediction class CS109B, P ROTOPAPAS , G LICKMAN 27
Generative Adversarial Networks (GAN) label: it is spam Yes prediction D No action for discriminator. Generator matches label? prediction: must do better. yes, it is spam Show spam email It was spam, for real It was not spam Yes, it is spam True positive (TP): the discriminator see a spam and predicts correctly. No need for further actions for discriminator. Generator must do a better job. No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 28
Generative Adversarial Networks (GANs) label: it is spam prediction D matches label? prediction: it is not spam No show spam email It was spam, for real It was not spam Discriminator learn more about spam Yes, it is spam False Negative (FN): the discriminator see an email and predict it as spam even though it is not. The discriminator learn more No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 29
Generative Adversarial Networks (GANs) label: it is not spam Yes prediction D matches label? prediction: it is spam No show spam email Discriminator learn more about spam It was spam, for real It was not spam Yes, it is spam False positive (FP): generator try to fool discriminator and the discriminator fails. The generator succeeded and the generator is forced to improve. No, it is not spam CS109B, P ROTOPAPAS , G LICKMAN 30
Generative Adversarial Networks (GANs) label: it is not a spam Yes prediction D No action for discriminator and no matches label? prediction: action for generator. yes, it is not a spam Show spam email It was spam, for real It was not spam Yes, it is spam True negative (TN): generator try to fool discriminator, however the discriminator predict correctly. The generator No, it is not spam learn what was wrong and try something else. CS109B, P ROTOPAPAS , G LICKMAN 31
The Discriminator The discriminator is very simple. It takes a sample as input, and its output is a single value that confidence that sample is real reports the network’s confidence that the input is from the training set, rather than being a fake. Discriminator There are not many restrictions on what the discriminator is. sample CS109B, P ROTOPAPAS , G LICKMAN 32
The Generator The generator takes as input a sample bunch of random numbers. If we build our generator to be deterministic, then the same input will always produce the same output. Generator In that sense we can think of the input values as latent variables. But here the latent variables weren’t noise discovered by analyzing the input, as they were for the VAE. CS109B, P ROTOPAPAS , G LICKMAN 33
The Generator Is it really noise? Remember our VAE Mean µ q ϕ ( z | x ) Sample Encoder from N ( µ , σ ) Decoder z SD σ p θ ( x | z ) CS109B, P ROTOPAPAS , G LICKMAN 34
The Generator Is it really noise? Remember our Variational Autoencoder. Mean µ q ϕ ( z | x ) Sample from N ( µ , σ ) Decoder z SD σ p θ ( x | z ) CS109B, P ROTOPAPAS , G LICKMAN 35
The Generator Is it really noise? Remember our Variational Autoencoder. Sample from N ( µ , σ ) Decoder z p θ ( x | z ) CS109B, P ROTOPAPAS , G LICKMAN 36
The Generator Is it really noise? Remember our Variational Autoencoder. Decoder z p θ ( x | z ) CS109B, P ROTOPAPAS , G LICKMAN 37
The Generator Is it really noise? Remember our Variational Autoencoder. z Generator CS109B, P ROTOPAPAS , G LICKMAN 38
The Generator So the random noise is not “ random ” but represents (an image in the example below) in the “ latent ” space. z Generator CS109B, P ROTOPAPAS , G LICKMAN 39
Learning The process – known as Learning Round - accomplishes three jobs: 1. The discriminator learns to identify features that characterize a real sample 2. The discriminator learns to identify features that reveal a fake sample 3. The generator learns how to avoid including the features that the discriminator has learned to spot CS109B, P ROTOPAPAS , G LICKMAN 40
CS109B, P ROTOPAPAS , G LICKMAN 41
Recommend
More recommend
