advanced machine learning
play

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 20: - PowerPoint PPT Presentation

Oct 13, 2023 •710 likes •1.11k views

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 20: Generative Adversarial Networks Jan-Willem van de Meent Slide credits: Ian Goodfellow Variational Autoencoders Input Hidden Mean Encoding Hidden Reconstructed Images

  1. Advanced Machine Learning CS 7140 - Spring 2019 Lecture 20: Generative Adversarial Networks Jan-Willem van de Meent Slide credits: Ian Goodfellow

  2. Variational Autoencoders Input Hidden Mean 
 Encoding Hidden Reconstructed 
 Images Units Std Dev (random) Units Images 784 256 2-50 256 784 (28 x 28) (28 x 28)

  3. Variational Autoencoders Assume prior:

  4. <latexit sha1_base64="xCnLHJxvOenf3Kf6H0xnPDc3fOI=">AGrHicfZRb9MwFMfTMcoItw0eYnoSydVQLTNgkNTUMTPCA0pt2kulSO46TWHCdznN0sfzM+CHwb7DSCJg64D7V8fv9zjn1OTphTUgjf/9VbebD6sP9o7bH75Omz5y/WN16eFVnJET5FGc34RQgLTAnDp4Ii9yjmEaUnweXn409vNrzAuSsRNxl+NpChNGYoKg0Eez9Z8ghWKOIJVf1BCIORZw5IF8TjbdPc8FRZnOJNsL1Pev7r9I70HruWtmjGt6ZCEoTxUM3k1M/BQo/fKAymJFqpN5QKYzHRf1nigphDJPOZXERQwoaeZVMs5Z2pc0zlSnN06SuZjO1gf+2K+WZ2+CejNw6nU021j9AaIMlSlmAlFYFJPAz8VUQi4Iolg7LgucQ3QJEzRWwZTXExlVQLlNawnwVTGROYoYZMwrQwL2IdGrhonqK5Dox5M2x9OJXGS4QLkrCmKkyV64Ix7odqsxkFNISK3n86UBJf7T9bhS83VEthOoJoJdf6R/bSDhGLMa2d0aBdu7NpOXPKf4L+QbzGTDMcM3KEtTyCIJrjFSE/0+ALOi5NhcRIwlYNAKWXBC1RrKrsLlo23SkrQSLBqgTZ2t4SZi2rozoLu3zdW9hVF1Z3qp296KZh2cGWFlvaELcg3s4Qd8bEeUFoxqz7xEt01SexHZQuMXWNjUuqZ0wELY/5vBs3X2ubPW5V5liZdlkmIE9SqOsMshxzKDJuProbIuaUpEQUsrYrW0XY/1Xa3g52qJpN+Wd2WSQKadWYzbezOxTxqMmZW3ZgCW9i8J1gHkLrB/YkHreBe3pZm/O3o4Dfx82xrsH9STb8157bxhk7g7Dj7zmfnyDl1UO9DL+qlPdYf90/6k/50ga70as0rp7H68W8FM24q</latexit> Objective: Evidence Lower Bound (ELBO) Reconstruction 
 KL between approximate 
 log-likelihood posterior and prior Log marginal KL between approximate likelihood posterior and posterior

  5. <latexit sha1_base64="RPJ/9dykIGmaIKoms+rtG6vULy0=">AGOnicfZRLa9wEICVNtum7itpj72Y7iWBJdhpSAKlEFpCckxDXhAviyzPekXkRyQ5L6F/lp/QP9Brb6W30lt7qOx16NoylcFIM98pBkpzBkV0vO+zj14ON979HjhifP02fMXLxeXh2LrOAEjkjGMn4aYgGMpnAkqWRwmnPAScjgJDz/VOpPLoELmqWH8iaHYLjlI4pwdKIRovj4FJdazcQNHz0e5ytVxgosCR04QkxTRYx/oR2jur0nl6vFShA4U/sP7u5U5L53AzkBiY0PSKN729Fi31v1quHaE7+e9FE9kdL83dBlJEigVQShoU4871cDhXmkhIGJptCQI7JOY7hzExTnIAYqupAtNvQHvpDNc5SCSlpmCmciATLiSUsYdGUkokJDLwZthYOVeklAkHjtGkVJtpxgjGpjhVZioKWQFaHex+1MobLwb+GubuoVwiGrC3/IG5msDMQdIa2RrfeBvbNlMXvCcwT/IK7EyGw4pXJEsSbCpTnAJRJ+Z8zG1EgWHciMqCBPV97XWFjxFjU2ld4JZ5bVWKmgkWHVG7uZwcqNGujGgm67fN1a2EUXNu2+juxlN42LDraw2MKGuAXxdobQGRNyQVmWvsZz9BVn4ztoGyGqWtcumTmxkfY8phPuvF8Qi32oFWZA12yBeZxgU+cgy4FjmfHy0l1ROWE0oVKoWq9tK5r+38ro28F2dLMpy38Yqh1tkSRkVWM2z87uUMKjJlfusgOLeRObFq4DzFtgfcAlad47v/262ZPjtVXfW/U/r/e39+qXbwG9QW/RMvLRJtpGe2gfHSGCvqCf6Df607vrfet97/2Yog/mapvXqDF6v/4CPnVJ8w=</latexit> Implicit Generative Models z x • Can generate samples x ~ p G ( x ) • Cannot evaluate density p G ( x ) • Requirement: G ( z ; θ ) is differentiable

  6. Maximum Likelihood θ ∗ = arg max E x ∼ p data log p model ( x | θ ) θ

  7. Generative Adversarial Networks D tries to make D(G(z)) near 0, D (x) tries to be G tries to make near 1 D(G(z)) near 1 Di ff erentiable D function D x sampled from x sampled from data model Di ff erentiable function G Input noise z

  8. <latexit sha1_base64="9sNKaSQkROSgekjd1VD40+OaNo=">AGl3icfZRbT9swFIBTxjqWXYDtadpLtb4UqaAEPA2xMbgbYC4TU1VOclpauE4me1AwfI/29t+xf7NnDSMJo7mSu2xz3cu9jk9fkowF47zp7XwbPF5+8XS/vV6zdvl1dW313yJGMBXAQJSdi1jzgQTOFCYEHgOmWAYp/AlX/zJdf3QLjOKHn4j6FYwisc4QEIfjVZ+ezGajqQXREweKdXxYkzLQvlV72123vUO9v5VTrOI476cgTMBU8kCESPUK1ZqyPQJjMfBIEnVK61LlMRxNxPDJ08Ojp16x+Wdr58Yzudex3c76k6NZdiW/Zs9cPv4ORytdZ8MpVscU3FLo7luzdTJaXfzlhUmQxUBFQBDnA9dJxVAiJnBAQCeUcUhRcIMiGiRohj4UBavrd9kXnvuDuU4oQJoUDGTKOYxEhPjMId59TSY6MDAqmHLw6HMvYTAcUSrVn6sbNsLYawrX2QmQ59koOTZ0YGSTn9nq+9u7qoawiAsCXfP6etPHYgYAC2Rve2+u7NnMmnGUgJPkJNjeTYMKNwFSRwjGupSQ6AG+n08oDxjkF9Een4su65SyoBnqLYp9LY3r5wqXtyPsGit+rY/RyWX1RD9wb0OTrwcB+NmGemIBuejN70UyjrIHNDYzIWZArJ4hNMaElGOSUOM+4zm6JOxGZTMWNc5dEj5MQGR7TSTOeTrDBntUqc6bydpknEIv0JFLS1JgSCQs/9PdYTEhOMaCy1KvTCtM/2+l9fVgh6ralPm378tDZCBT4rGrL6d2aF6UlW5/JYNWMSq2KxwDWBaA8sHzk979z6dDOFy80Nd2tj83S7u39QTr4l6P1yepZrVr7VvH1ol1YQWtdb31nXrR/tD+3P7W/t4hi60Spv3VmW1T/8CRIZmeQ=</latexit> Objective: Minimax Game D − E x ∼ p DATA ( x ) [ log D ( x )] − E z ∼ p ( z ) [ log ( 1 − D ( G ( z )))] max min G • Generator G ( z ) generates fake images. • Discriminator D ( x ) classifies real images as 1 
 and fake images as 0. • Objective for G : Maximize probability that G fools D . • Objective for D : Minimize probability that D is fooled.

  9. <latexit sha1_base64="pXpRo69NxwuPNk2867tQLRln0Yw=">AAAGl3icfZRbT9swFIBTxjqW3cr2NO2lWl+KVFDCEPA2xMbgbYC4TXVVOclpauFcZjtQsPzP9rZfsX8zJw2jiaO5Unvs852LfU6Pl1LCheP8aS09WX7afrby3H7x8tXrN53Vtxc8yZgP535CE3blYQ6UxHAuiKBwlTLAkUfh0rv+kusvb4BxksRn4i6FUYTDmEyIj4U+Gnd+o4jEY4l8FsivSnVRhGf5NmTyUG/tdRsd6P2NnGkdJ1E3HSMBM8F9GWCBVb9QrSkbUZiIIaJJ2C2dlSrESDgVo0dP9w+e+sXmn62dG8/lftd2u+uPjubplPyaPXf58Dsad3rOhlOsrim4pdCzynU8Xl3+hYLEzyKIhU8x50PXScVIYiaIT0EnlHFIsX+NQxhqMcYR8JEsXlu/yaL2zB3JSRILiP2KmcQRj7CYGoc5zKun/lQHBlYNWx6OZO4lAE7CuGrlRcq2UQATXfkiMxl4NAMlTw/3lXQG258G7uaOqiEMgpJwd52B/tSBkAHEJbK7NXC3d00mzVhK4RFycizPhkEMt34SRTgOdKnBV0P9PghinjHILyKRF8meq5Qy4DmqbQq9jRaVMyV1Ty4mWPRWHbtbwPKLaujOgO6bfN0b2M8mDIkp6KY3sxfNNM4a2MxgMxNiBsTqGUJjTEg5oUls3GeyQBd9MjGD0gWmrHHukupxEmDDYzptxtMpMdjTWmVOVd4uiwRmoR49SqIkBYZFwvI/3S0RU0oiIrgs9cq0IvH/rbS+HuxAVZsy//Y8eaAM0vdo0ZjVtzM7VE+qKpffsgELWRWbF64BTGtg+cA5qeedW59upnCxueE6G+7JVm9vv5x8K9YH66PVt1xrx9qzjqxj69zyW2ut762r1o/2+/bn9rf20RxdapU276zKap/8BfDiZjU=</latexit> <latexit sha1_base64="h/1wASbY8y+dyFEiXmgpamckOK4=">AGLHicfZTLbtQwFEDd0oESXlNYshkxmyJGVKqthukqlCVZan6kprRyHuzFh1EmM7fVn+Jb4A8RVsEGKH2ME3EKdBTOKAI0VXvuf6Pu2IMyqV73+Zm7+10Ll9Z/Gud+/+g4ePukuPj2SWCwKHJGOZOImwBEZTOFRUMTjhAnASMTiOzl5b/fE5CEmz9EBdcRgmeJLSMSVYFVuj7smb5fBcX5rn3queF4FJpqPdKjgUkmiY6ywMRVhvJ5d/9T3XvT4aPcPOr2/RW/XD1XCqhj6q1N1pa+BjGckTSBVhWMrTwOdqLFQlDAwXphL4Jic4QmcFmKE5BDXZbA9Grag2Cox1mqICU1M40TmWA1dTYtLOu7ZFo4BlF3W20OtT0lBknad0qSoznhTGMi3aUkek4YjkYvb+7bQ/WH85CFY3TAMREFdEsOkPiq8JTARAWiGba4NgfdNleC4g7+QbzEbjYAULkiWJDiNdXgOxJwW9QkhlbkAm4gOo0T3A2OMA9+ghU2p98JZ5aXROqwFWLa+iV3NYDbRArpyoOu2s64d7H0bFqop2DF0olftNM5b2NxhcxcSDiSaEUKrT+CSsix18hnP0OWcjF2nbIapemyPZMUdj7FzIp+243xKHXa/0Zl9Y8dlsBikuCiz2HGQWCVCXvpLqiaMpQJXWlN64VTf9vVeibznZMfSjtP4r0jnFIErFyMOu1cyeUiLjO2SxbsImoYzeNawF5A6wKbMnivQuar5srHK2uBP5K8G6tv7VdvXyL6Cl6hpZRgDbQFnqL9tAhIugT+o5+ol+dD53Pna+dbzfo/Fxl8wTVufHb9nRJI=</latexit> Exercise • Suppose that we could evaluate p DATA ( x ) and p G ( x ) • Can you express D ( x ) in terms of p DATA ( x ) and p G ( x ) ? SOLUTION:

Recommend

MACHINE LEARNING Kernel Canonical Correlation Analysis 1 ADVANCED MACHINE LEARNING ADVANCED

MACHINE LEARNING Kernel Canonical Correlation Analysis 1 ADVANCED MACHINE LEARNING ADVANCED

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING MACHINE LEARNING Kernel Canonical Correlation Analysis 1 ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Structure of todays and next weeks class 1) Briefly go through one

613 views • 28 slides
ADVANCED MACHINE LEARNING Non-linear regression techniques 1 1 ADVANCED MACHINE LEARNING

ADVANCED MACHINE LEARNING Non-linear regression techniques 1 1 ADVANCED MACHINE LEARNING

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Non-linear regression techniques 1 1 ADVANCED MACHINE LEARNING Regression: Principle N Map N-dim. input x to a continuous output y . Learn a function of the type:

1.29k views • 61 slides
ADVANCED MACHINE LEARNING Kernel PCA 11 ADVANCED MACHINE LEARNING Overview Todays Lecture

ADVANCED MACHINE LEARNING Kernel PCA 11 ADVANCED MACHINE LEARNING Overview Todays Lecture

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Kernel PCA 11 ADVANCED MACHINE LEARNING Overview Todays Lecture Brief Recap of Classical Principal Component Analysis (PCA) Derivation of kernel PCA Exercises to develop a

1.18k views • 28 slides
ADVANCED MACHINE LEARNING Mini-Project Overview Lecture : Prof. Aude Billard

ADVANCED MACHINE LEARNING Mini-Project Overview Lecture : Prof. Aude Billard

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Mini-Project Overview Lecture : Prof. Aude Billard (aude.billard@epfl.ch) Teaching Assistants : Nadia Figueroa, Ilaria Lauzana, Brice Platerrier 11 ADVANCED MACHINE LEARNING Deadlines for

463 views • 24 slides
MACHINE LEARNING Spectral Clustering 1 ADVANCED MACHINE LEARNING Outline of Todays Lecture

MACHINE LEARNING Spectral Clustering 1 ADVANCED MACHINE LEARNING Outline of Todays Lecture

ADVANCED MACHINE LEARNING MACHINE LEARNING Spectral Clustering 1 ADVANCED MACHINE LEARNING Outline of Todays Lecture Introduce the principle of spectral clustering Show extension for other transformations of the space

616 views • 38 slides
Discrete and Continuous Reinforcement Learning (not part of exam material) 1 1 ADVANCED

Discrete and Continuous Reinforcement Learning (not part of exam material) 1 1 ADVANCED

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Brief Overview of Discrete and Continuous Reinforcement Learning (not part of exam material) 1 1 ADVANCED MACHINE LEARNING Forms of Learning Supervised learning where the algorithm

1.27k views • 71 slides
Advanced Machine Learning Learning Kernels MEHRYAR MOHRI MOHRI@ COURANT INSTITUTE &amp;

Advanced Machine Learning Learning Kernels MEHRYAR MOHRI MOHRI@ COURANT INSTITUTE &amp;

Advanced Machine Learning Learning Kernels MEHRYAR MOHRI MOHRI@ COURANT INSTITUTE &amp; GOOGLE RESEARCH .. Outline Kernel methods. Learning kernels scenario. learning bounds. algorithms. Advanced Machine Learning -

776 views • 45 slides
ADVANCED MACHINE LEARNING Non-linear regression techniques (SVR and extensions, GPR, Gradient

ADVANCED MACHINE LEARNING Non-linear regression techniques (SVR and extensions, GPR, Gradient

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Non-linear regression techniques (SVR and extensions, GPR, Gradient Boosting) 1 1 ADVANCED MACHINE LEARNING Regression: Principle N Map N-dim. input to a continuous

906 views • 89 slides
Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction and Course Details Varun Kanade University of Oxford January 15, 2018 What is Machine Learning? 200-basic level categories Here: Six

466 views • 18 slides
ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets (Adapted from

ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets (Adapted from

ADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING Caveats and Techniques to Deal with Imbalanced Datasets (Adapted from H. He and E. A. Garcia, Learning from Imbalanced Data, IEEE Trans. Knowledge and Data Engineering, vol. 21, issue 9,

285 views • 24 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Dhruv Batra Virginia Tech What is this class about? Some of the most exciting developments in Machine Learning, AI, Statistics &amp;

925 views • 61 slides
CS 6784: Spring 2010 Advanced Topics in Machine Learning Review Guozhang Wang February 25, 2010

CS 6784: Spring 2010 Advanced Topics in Machine Learning Review Guozhang Wang February 25, 2010

CS 6784: Spring 2010 Advanced Topics in Machine Learning Review Guozhang Wang February 25, 2010 1 Machine Learning Tasks Machine learning tasks can be roughly categorized into three classes: su- pervised learning, unsupervised learning,

461 views • 7 slides
DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal

Introduction Recap Kernels Gaussian Processes References DD2434 - Advanced Machine Learning Gaussian Processes Carl Henrik Ek { chek } @csc.kth.se Royal Institute of Technology November 5th, 2015 Ek KTH DD2434 - Advanced Machine Learning

1.91k views • 170 slides
Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos

Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos

Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos Motivation Find meaningful low-dimensional structures hidden in high-dimensional observations. The human brain confronts the same problem in

1.06k views • 85 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Parameter Learning Structure Learning Readings: KF 18.1, 18.3; Barber 9.5, 10.4 Dhruv Batra Virginia

351 views • 32 slides
Advanced Machine Learning CS 7140 - Spring 2019 Lecture 24: Bayesian Optimization Jan-Willem van

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 24: Bayesian Optimization Jan-Willem van

Advanced Machine Learning CS 7140 - Spring 2019 Lecture 24: Bayesian Optimization Jan-Willem van de Meent Slide credits: Ryan Adams, Nando de Freitas Background: Multi-Armed Bandits Problem: Which machine has highest rate of payout?

805 views • 42 slides
An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning

An Exercise in An Exercise in Machine Learning Machine Learning http://www.cs.iastate.edu/~cs573x/bbsilab.html Machine Learning Software Preparing Data Building Classifiers Interpreting Results Machine Learning Software

496 views • 26 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets (Finish) Structure Learning Readings: KF 18.4; Barber 9.5, 10.4 Dhruv Batra Virginia Tech Administrativia HW1

596 views • 30 slides
Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by

Machine Learning By Alex Scarlatos What is Machine Learning? Machine Learning is the process by which computers can be trained through observation, rather than being explicitly programmed. Basically, a program is given input and adjusts its

556 views • 17 slides
Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning: Study of algorithms that improve their performance P at some task T

Machine Learning 10-601 Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 12, 2015 Today: Readings: What is machine learning? The Discipline of ML Decision tree learning Mitchell, Chapter 3

872 views • 29 slides
Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

398 views • 26 slides
Advanced Machine Learning Course IV - (Hierarchical) Clustering L. Omar Chehab (1) and Frdric

Advanced Machine Learning Course IV - (Hierarchical) Clustering L. Omar Chehab (1) and Frdric

Advanced Machine Learning Course IV - (Hierarchical) Clustering L. Omar Chehab (1) and Frdric Pascal (2) (1) Parietal Team, Inria (2) Laboratory of Signals and Systems (L2S), CentraleSuplec, University Paris-Saclay

812 views • 51 slides
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale

ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Bayes Nets: Inference Marginals, MPE, MAP Variable Elimination Readings: KF 9.1,9.2; Barber 5.1 Dhruv Batra Virginia Tech

667 views • 22 slides
Organon Analytics AI Platform We use our own advanced machine learning platform to help

Organon Analytics AI Platform We use our own advanced machine learning platform to help

Organon Analytics AI Platform We use our own advanced machine learning platform to help Turkcell analyse vast data pools and create new insights and propositions that would not have been possible 1. Reduce dependency on Data

802 views • 18 slides

More recommend