how good is the bayes posterior in deep neural networks
play

How Good is the Bayes Posterior in Deep Neural Networks Really? - PowerPoint PPT Presentation

Jan 28, 2024 •5 likes •385 views

Code: github.com/google-research/google-research/tree/ master/cold_posterior_bnn How Good is the Bayes Posterior in Deep Neural Networks Really? Florian Wenzel (Google Research Berlin) Joint first authors: Kevin Roth, Bas Veeling, and: Jakub

  1. Code: github.com/google-research/google-research/tree/ master/cold_posterior_bnn How Good is the Bayes Posterior in Deep Neural Networks Really? Florian Wenzel (Google Research Berlin) Joint first authors: Kevin Roth, Bas Veeling, and: Jakub Swiatkowski, Linh Tran, Stephan Mandt, Jasper Snoek, Tim Salimans, Rodolphe Jenatton, Sebastian Nowozin Florian Wenzel, 15 June 2020

  2. Bayesian Deep Learning Goal: enable Bayesian inference for deep networks to improve robustness of predictions! Active research field where most work focuses on improving approximate inference to get closer to the Bayes posterior 2 Florian Wenzel, 15 June 2020

  3. But is the Bayes posterior actually good? 3 Florian Wenzel, 15 June 2020

  4. <latexit sha1_base64="rTwF5qKwJnpd9PmqZdb1cRT7ZVM=">AB/HicbVC7TsMwFHV4lvIKdGSJqJCYqoSHYKxgYSwSfUhNVDmO01p17Mi+Qaqi8isDCDEyoew8Tc4bQZoOZLlo3PulY9PmHKmwXW/rZXVtfWNzcpWdXtnd2/fPjsaJkpQtEcql6IdaUM0HbwIDTXqoTkJOu+H4tvC7j1RpJsUDTFIaJHgoWMwIBiMN7JofSh7pSWKu3IcRBTwd2HW34c7gLBOvJHVUojWwv/xIkiyhAgjHWvc9N4UgxwoY4XRa9TNU0zGeEj7hgqcUB3ks/BT58QokRNLZY4AZ6b+3shxot8ZjLBMNKLXiH+5/UziK+DnIk0AyrI/KE4w5Ip2jCiZiBPjEwUM1kdMsIKEzB9VU0J3uKXl0nrOGdNy7vL+rNm7KOCjpCx+gUegKNdEdaqE2ImiCntErerOerBfr3fqYj65Y5U4N/YH1+QOnQZVt</latexit> <latexit sha1_base64="IY1J2jvCqr5IEA9S/aVNBI46QY=">ACSXicbVDLSgMxFM20Pmp9V26CRahBSkzPlAQoagLlxXsAzplzKRpG5p5kNwRy9jfc+POnf/gxoUirsy0XWjbAyGHc+5N7j1uKLgC03wzUumFxaXlzEp2dW19YzO3tV1TQSQpq9JABLhEsUE91kVOAjWCUjnitY3e1fJX79gUnFA/8OBiFreaTr8w6nBLTk5O7Dgu0Goq0Gnr5iG3oMyPA2x6BHiUivh7iIr7AYWHg8KdHh2trtr6I7XM896Wik8ubJXMEPEusCcmjCSpO7tVuBzTymA9UEKWalhlCKyYSOBVsmLUjxUJC+6TLmpr6xGOqFY+SGOJ9rbRxJ5D6+IBH6t+OmHgqGVBXJguqaS8R53nNCDpnrZj7YQTMp+OPOpHAEOAkVtzmklEQA0IlVzPimPSEJBh5/VIVjTK8+S2mHJOiqd3B7ny5eTODJoF+2hArLQKSqjG1RBVUTRM3pHn+jLeDE+jG/jZ1yaMiY9O+gfUulfRdCzKA=</latexit> <latexit sha1_base64="WGOL1vxHGku1eDCobZodwAJrpgc=">ACEHicbVDLSsNAFJ34rPUVdelmsIh1UxIf6EYo6sJlBfuAJpTJdNIOnTyYuRFKzCe48VfcuFDErUt3/o2TtgtvTDM4Zx7ueceLxZcgWV9G3PzC4tLy4WV4ura+samubXdUFEiKavTSESy5RHFBA9ZHTgI1olI4EnWNMbXOV685JxaPwDoYxcwPSC7nPKQFNdcyDuOwEBPqUiPQ6w/Y8SLRVcNAf6kDfQYkO8QXuGOWrIo1KjwL7AkoUnVOuaX041oErAQqCBKtW0rBjclEjgVLCs6iWIxoQPSY20NQxIw5ajgzK8r5ku9iOpXwh4xP6eSEmgco+6MzevprWc/E9rJ+CfuykP4wRYSMeL/ERgiHCeDu5ySiIoQaESq69YtonklDQGRZ1CPb0ybOgcVSxjyuntyel6uUkjgLaRXuojGx0hqroBtVQHVH0iJ7RK3oznowX4934GLfOGZOZHfSnjM8fHtWcoQ=</latexit> Bayesian Neural Networks (BNNs) Input Output Hidden Neural Network ) = p ( y i | x i , θ ) p ( D| θ ) = Different models obtained by different θ 4 Florian Wenzel, 15 June 2020

  5. <latexit sha1_base64="wiljDcQFTYfNdIHzdnPtl0J/I+0=">ACDHicbVDLSsNAFJ34rPVdelmsAh1UxIf6LKoC5cV7AOaUCaTSTt08mDmRigxH+DGX3HjQhG3foA7/8ZJm4W2HhjmcO653HuPGwuwDS/jYXFpeWV1dJaeX1jc2u7srPbVlEiKWvRSESy6xLFBA9ZCzgI1o0lI4ErWMcdXeX1zj2TikfhHYxj5gRkEHKfUwJa6leqc12I+GpcaC/1IYhA5I9YDsgMKREpNfZkXaZdXMCPE+sglRgWa/8mV7EU0CFgIVRKmeZcbgpEQCp4JlZTtRLCZ0RAasp2lIAqacdHJMhg+14mE/kvqFgCfq746UBCrfVjvzHdVsLRf/q/US8C+clIdxAiyk0F+IjBEOE8Ge1wyCmKsCaGS610xHRJKOj8yjoEa/bkedI+rlsn9bPb02rjsoijhPbRAaohC52jBrpBTdRCFD2iZ/SK3own48V4Nz6m1gWj6NlDf2B8/gBvL5vc</latexit> <latexit sha1_base64="IY1J2jvCqr5IEA9S/aVNBI46QY=">ACSXicbVDLSgMxFM20Pmp9V26CRahBSkzPlAQoagLlxXsAzplzKRpG5p5kNwRy9jfc+POnf/gxoUirsy0XWjbAyGHc+5N7j1uKLgC03wzUumFxaXlzEp2dW19YzO3tV1TQSQpq9JABLhEsUE91kVOAjWCUjnitY3e1fJX79gUnFA/8OBiFreaTr8w6nBLTk5O7Dgu0Goq0Gnr5iG3oMyPA2x6BHiUivh7iIr7AYWHg8KdHh2trtr6I7XM896Wik8ubJXMEPEusCcmjCSpO7tVuBzTymA9UEKWalhlCKyYSOBVsmLUjxUJC+6TLmpr6xGOqFY+SGOJ9rbRxJ5D6+IBH6t+OmHgqGVBXJguqaS8R53nNCDpnrZj7YQTMp+OPOpHAEOAkVtzmklEQA0IlVzPimPSEJBh5/VIVjTK8+S2mHJOiqd3B7ny5eTODJoF+2hArLQKSqjG1RBVUTRM3pHn+jLeDE+jG/jZ1yaMiY9O+gfUulfRdCzKA=</latexit> Bayesian Neural Networks (BNNs) Input Output Hidden Bayesian Neural Network p ( θ , D ) = p ( y i | x i , θ ) p ( θ ) Posterior: Distribution over likely models given the data p ( θ |D ) 5 Florian Wenzel, 15 June 2020

  6. <latexit sha1_base64="fXqnuC6GFqFpqTVuYlqBxuVUNsM=">ACXHicbVHLSgMxFM2MVfuwOiq4cRMsQoUyzLSKLou6cFnBPqBTSiZNbWjmQXJHKGN/0l03/opm2i607YGQw7n3ck9O/FhwBY6zMy93P7BYb5QLB2Vj0+s07OihJWZtGIpI9nygmeMjawEGwXiwZCXzBuv70Kat3P5hUPArfYBazQUDeQz7mlICWhpby/EiM1CzQV+rBhAGZD90a3iXd8uNGrZtG3uKBziu7ujAn9gLCEwoEenz/AYPrYpjO0vgbeKuSQWt0RpaX94oknAQqCKNV3nRgGKZHAqWDzopcoFhM6Je+sr2lIAqYG6TKcOb7WygiPI6lPCHip/p1ISaAyv7ozM6k2a5m4q9ZPYPwSHkYJ8BCulo0TgSGCGdJ4xGXjIKYaUKo5NorphMiCQX9H0Udgrv5G3Sqdtuw757va0H9dx5NElukJV5KJ71EQvqIXaiKIF+jHyRsH4NnNmySyvWk1jPXO/sG8+AWhaLY/</latexit> <latexit sha1_base64="uJWKBoYxcgei08sye4qA0oB9Su0=">ACD3icbVC7TgMxEPTxDOEVoKSxiEBJQXTHQ1BG0FCRApF0U+Z5NY8Z0tew8RnfIHNPwKDQUI0dLS8Tc4IQUQRrI8mtnV7k6kpbDo+5/e1PTM7Nx8biG/uLS8slpYW7+yKjUcalxJZW4iZkGKBGoUMKNsDiSMJ1Dsd+te3YKxQySX2NTRi1klEW3CGTmoWdkJtlEZFQ7jTtLRbK4WRki3bj92XhdgFZINyuVko+hV/BDpJgjEpkjHOm4WPsKV4GkOCXDJr64GvsZExg4JLGOTD1IJmvMc6UHc0YTHYRja6Z0C3ndKibWXcS5CO1J8dGYvtcENXGTPs2r/eUPzPq6fYPm5kItEpQsK/B7VTSd39w3BoSxjgKPuOMG6E25XyLjOMo4sw70I/p48Sa72KsF+5fDioFg9GceRI5tki5RIQI5IlZyRc1IjnNyTR/JMXrwH78l79d6+S6e8c8G+QXv/QtTmJw4</latexit> BNNs: Predictions In standard deep learning we optimize θ SGD (MAP) BNNs use samples from the posterior (ensemble of models) θ 1 , θ 2 , θ 3 , ... ∼ p ( θ |D ) ∝ exp( − U ( θ )) 6 Florian Wenzel, 15 June 2020

  7. <latexit sha1_base64="aOGT/zEJg6itkXTtYS7WSQOTsjM=">ACFnicbVDLSgMxFM3UV62vUZdugkVQ0DLjA12KblxWsA/olCGTpm1oZhKSO2IZ+xVu/BU3LhRxK+78G9PHQq0HQg7n3Mu90RKcAOe9+XkZmbn5hfyi4Wl5ZXVNXd9o2pkqimrUCmkrkfEMETVgEOgtWVZiSOBKtFvcuhX7tl2nCZ3EBfsWZMOglvc0rASqF7EBCltLwLTBqHBqvdPr7Hd/s4iKRomX5svyALgMyCM1e6Ba9kjcCnib+hBTRBOXQ/QxakqYxS4AKYkzD9xQ0M6KBU8EGhSA1TBHaIx3WsDQhMTPNbHTWAO9YpYXbUtuXAB6pPzsyEpvhirYyJtA1f72h+J/XSKF91sx4olJgCR0PaqcCg8TDjHCLa0ZB9C0hVHO7K6ZdogkFm2TBhuD/PXmaVA9L/lHp5Pq4eH4xiSOPtA2kU+OkXn6AqVUQVR9ICe0At6dR6dZ+fNeR+X5pxJzyb6BefjG31un5Y=</latexit> <latexit sha1_base64="fXqnuC6GFqFpqTVuYlqBxuVUNsM=">ACXHicbVHLSgMxFM2MVfuwOiq4cRMsQoUyzLSKLou6cFnBPqBTSiZNbWjmQXJHKGN/0l03/opm2i607YGQw7n3ck9O/FhwBY6zMy93P7BYb5QLB2Vj0+s07OihJWZtGIpI9nygmeMjawEGwXiwZCXzBuv70Kat3P5hUPArfYBazQUDeQz7mlICWhpby/EiM1CzQV+rBhAGZD90a3iXd8uNGrZtG3uKBziu7ujAn9gLCEwoEenz/AYPrYpjO0vgbeKuSQWt0RpaX94oknAQqCKNV3nRgGKZHAqWDzopcoFhM6Je+sr2lIAqYG6TKcOb7WygiPI6lPCHip/p1ISaAyv7ozM6k2a5m4q9ZPYPwSHkYJ8BCulo0TgSGCGdJ4xGXjIKYaUKo5NorphMiCQX9H0Udgrv5G3Sqdtuw757va0H9dx5NElukJV5KJ71EQvqIXaiKIF+jHyRsH4NnNmySyvWk1jPXO/sG8+AWhaLY/</latexit> BNNs: Predictions Predict by using an average of models X p ( y | x, θ s ) ≈ θ SGD (MAP) s θ 1 , θ 2 , θ 3 , ... ∼ p ( θ |D ) In this talk: A model is good if it predicts well (e.g. low cross entropy loss) 7 Florian Wenzel, 15 June 2020

  8. Bayesian Neural Networks (BNNs) Promises of BNNs * : • Robustness in generalization • Better uncertainty quantification (calibration) • Enables new deep learning applications (continual learning, sequential decision making, …) * [e.g., Neal 1995, Gal et al. 2016, Wilson 2019, Ovadia et al. 2019]. 8 Florian Wenzel, 15 June 2020

  9. Bayesian Neural Networks (BNNs) But in practice BNNs are rarely used! 9 Florian Wenzel, 15 June 2020

  10. Bayesian Neural Networks (BNNs) In practice: • Often, the Bayes posterior is worse than SGD point estimates • But Bayes predictions can be improved by the use of the Cold Posterior* For temperature T<1: We sharpen the posterior (over-count evidence) *Explicitly (or implicitly) used by most recent Bayesian DL papers [e.g., Li et al. 2016, Zhang et al. 2020, Ashukha et al. 2020]. 10 Florian Wenzel, 15 June 2020

  11. Bayesian Neural Networks (BNNs) θ Cold Posterior For temperature T<1: We sharpen the posterior (over-count evidence) 11 Florian Wenzel, 15 June 2020

  12. ResNet-20 / CIFAR-10 True Bayes posterior Optimal cold posterior CNN-LSTM / IMDB 12 Florian Wenzel, 15 June 2020

  13. The cold posterior sharply deviates from the Bayesian paradigm. What is the use of more accurate posterior approximations if the posterior is poor? 13 Florian Wenzel, 15 June 2020

  14. Our paper: Hypothesis for the origin of the improved performance of cold posteriors Inference Likelihood Prior Inaccurate SDE Simulation? Dirty likelihoods? Current priors used for BNN parameters are (batch-normalization, poor? Bias of SG-MCMC? dropout, data augmentation) The effect becomes Minibatch noise (which is stronger with increasing not Gaussian)? model depths and capacity? Bias-variance tradeoff induced by cold posterior? 14 Florian Wenzel, 15 June 2020

  15. Our paper: Hypothesis for the origin of the improved performance of cold posteriors Inference Likelihood Prior Inaccurate SDE Simulation? Dirty likelihoods? Current priors used for BNN parameters are (batch-normalization, poor? Bias of SG-MCMC? dropout, data augmentation) The effect becomes Minibatch noise (which is stronger with increasing not Gaussian)? model depths and capacity? Bias-variance tradeoff induced by cold posterior? 15 Florian Wenzel, 15 June 2020

  16. Our paper: Hypothesis for the origin of the improved performance of cold posteriors Inference Likelihood Prior Inaccurate SDE Simulation? Dirty likelihoods? Current priors used for BNN parameters are (batch-normalization, poor? Bias of SG-MCMC? dropout, data augmentation) The effect becomes Minibatch noise (which is stronger with increasing not Gaussian)? model depths and capacity? Bias-variance tradeoff induced by cold posterior? 16 Florian Wenzel, 15 June 2020

Recommend

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information Engineering The Chinese University of Hong Kong Deep Neural Networks are Everywhere Playing Go Making Medical Decision Understanding Scenes Deep Neural

1.1k views • 46 slides
On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl Dickstein Tom Brady Deep Neural Networks Recent successes in using Deep neural networks for image classification,

1.32k views • 17 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural

Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural

Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural Networks and loss surfaces Problems of Deep Architectures Optimization in Neural Networks Under fitting Proliferation of Saddle

1.67k views • 41 slides
Posterior odds interpretation of a sigmoid Artificial Intelligence: Neural Networks Michael S.

Posterior odds interpretation of a sigmoid Artificial Intelligence: Neural Networks Michael S.

Artificial Intelligence: Representation and Problem Solving 15-381 January 16, 2007 Neural Networks Topics decision boundaries linear discriminants perceptron gradient learning neural networks Artificial Intelligence:

539 views • 19 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Umut Gcl and Marcel A. J. van Gerven Article overview by Ilya Kuzovkin Computational Neuroscience Seminar University of Tartu

852 views • 63 slides
Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning and AI y t-1 y t y t+1 Computer vision h t-1 h t h t+1 Language translation Speech recognition h t-1 h t h t+1 Question answering x t-1 x t x t+1

454 views • 43 slides
Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides
ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G. Montavon, W. Samek, K.-R. Mller Part 2: Making Deep Neural Networks Transparent 5 March 2017 Making Deep Neural Nets Transparent DNN transparency

448 views • 44 slides
Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit: Vlad Morariu Training (Deep) Neural Networks Computational graphs Improvements to gradient descent Stochastic gradient descent Momentum

814 views • 21 slides
6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Monday (2/5) 30 minutes Topics: Optimization Basic neural networks Neural Network

619 views • 41 slides
DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at

DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at

DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at St. Petersburg October 10, 2017, Seoul, Korea Outline Birds eye overview of deep learning Convolutional neural networks Chris

743 views • 32 slides
Weight Parameterizations in Deep Neural Networks Sergey Zagoruyko e Paris-Est, Universit

Weight Parameterizations in Deep Neural Networks Sergey Zagoruyko e Paris-Est, Universit

Weight Parameterizations in Deep Neural Networks Weight Parameterizations in Deep Neural Networks Sergey Zagoruyko e Paris-Est, Universit Ecole des Ponts ParisTech December 26, 2017 Weight Parameterizations in Deep Neural Networks

1.05k views • 45 slides
CS480/680 Lecture 15: June 26, 2019 Deep Neural Networks [GBC] Chap. 6, 7, 8 University of

CS480/680 Lecture 15: June 26, 2019 Deep Neural Networks [GBC] Chap. 6, 7, 8 University of

CS480/680 Lecture 15: June 26, 2019 Deep Neural Networks [GBC] Chap. 6, 7, 8 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Outline Deep Neural Networks Gradient Vanishing Rectified linear units Overfitting

941 views • 25 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

LRP Ariyan Zarei Motivation Having More interpretable Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks to have more interpretable Papers and Demo Introduction Machine Learning models

373 views • 35 slides

More recommend