generalization of linearized neural networks staircase
play

Generalization of linearized neural networks: staircase decay and - PowerPoint PPT Presentation

Mar 18, 2023 •557 likes •1.3k views

Generalization of linearized neural networks: staircase decay and double descent Song Mei UC Berkeley July 23, 2020 Department of Mathematics, HKUST Deep Learning Revolution Machine translation Autonomous Vehicle Robotics Healthcare

  1. Generalization of linearized neural networks: staircase decay and double descent Song Mei UC Berkeley July 23, 2020 Department of Mathematics, HKUST

  2. Deep Learning Revolution Machine translation Autonomous Vehicle Robotics Healthcare Gaming Deep learning Communication Finance

  3. Deep Learning Revolution Machine translation Autonomous Vehicle Robotics Healthcare Gaming Deep learning Communication Finance “ACM named Yoshua Bengio, Geoffrey Hinton, and Yann LeCun recipients of the 2018 ACM A.M. Turing Award for conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing. ”

  4. But theoretically?

  5. But theoretically? WHEN and WHY does deep learning work?

  6. Call for Theoretical understandings “Alchemy”

  7. Call for Theoretical understandings “Alchemy” Science Reproducible Physical Mathematical Experiments Laws Theories

  8. ✩ ✩ What don’t we understand?

  9. ✩ ✩ What don’t we understand? Empirical Surprises [Zhang, et.al, 2015]: ◮ Over-parameterization: ★ parameters ✢ ★ training samples. ◮ Non-convexity. ◮ Efficiently fit all the training samples using SGD. ◮ Generalize well on test samples.

  10. What don’t we understand? Empirical Surprises [Zhang, et.al, 2015]: ◮ Over-parameterization: ★ parameters ✢ ★ training samples. ◮ Non-convexity. ◮ Efficiently fit all the training samples using SGD. ◮ Generalize well on test samples. Mathematical Challenges ✩ Non-convexity Why efficient optimization? ✩ Over-parameterization Why effective generalization?

  11. A gentle introduction to Linearization theory of neural networks

  12. Linearized neural networks (neural tangent model) ◮ Multi-layers neural network ❢ ✭ x ❀ θ ✮ , x ✷ R ❞ , θ ✷ R ◆ ❢ ✭ x ❀ θ ✮ ❂ W ▲ ✛ ✭ ✁ ✁ ✁ W ✷ ✛ ✭ W ✶ x ✮✮ ✿ ◮ Linearization around (random) parameter θ ✵ ❢ ✭ x ❀ θ ✮ ❂ ❢ ✭ x ❀ θ ✵ ✮ ✰ ❤ θ � θ ✵ ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✰ ♦ ✭ ❦ θ � θ ✵ ❦ ✷ ✮ ✿ ◮ Neural tangent model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ [Jacot, Gabriel, Hongler, 2018] [Chizat, Bach, 2018b]

  13. Linearized neural networks (neural tangent model) ◮ Multi-layers neural network ❢ ✭ x ❀ θ ✮ , x ✷ R ❞ , θ ✷ R ◆ ❢ ✭ x ❀ θ ✮ ❂ ✛ ✭ W ▲ ✛ ✭ ✁ ✁ ✁ W ✷ ✛ ✭ W ✶ x ✮✮✮ ✿ ◮ Linearization around (random) parameter θ ✵ ❢ ✭ x ❀ θ ✮ ❂ ❢ ✭ x ❀ θ ✵ ✮ ✰ ❤ θ � θ ✵ ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✰ ♦ ✭ ❦ θ � θ ✵ ❦ ✷ ✮ ✿ ◮ Neural tangent model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ [Jacot, Gabriel, Hongler, 2018] [Chizat, Bach, 2018b]

  14. Linearized neural networks (neural tangent model) ◮ Multi-layers neural network ❢ ✭ x ❀ θ ✮ , x ✷ R ❞ , θ ✷ R ◆ ❢ ✭ x ❀ θ ✮ ❂ ✛ ✭ W ▲ ✛ ✭ ✁ ✁ ✁ W ✷ ✛ ✭ W ✶ x ✮✮✮ ✿ ◮ Linearization around (random) parameter θ ✵ ❢ ✭ x ❀ θ ✮ ❂ ❢ ✭ x ❀ θ ✵ ✮ ✰ ❤ θ � θ ✵ ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✰ ♦ ✭ ❦ θ � θ ✵ ❦ ✷ ✮ ✿ ◮ Neural tangent model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ [Jacot, Gabriel, Hongler, 2018] [Chizat, Bach, 2018b]

  15. Linear regression over random features ◮ NT model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ ✣ ✭ x ✮ ✐ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ ◮ (Random) feature map: ✣ ✭ ✁ ✮ ❂ r θ ❢ ✭ ✁ ❀ θ ✵ ✮ ✿ R ❞ ✦ R ◆ . ◮ Training dataset: ✭ ❳ ❀ ❨ ✮ ❂ ✭ x ✐ ❀ ② ✐ ✮ ✐ ✷ ❬ ♥ ❪ . ◮ Gradient flow dynamics: ❞ ❞ t β t ❂ � r β ❫ β ✵ ❂ 0 ✿ E ❬✭ ② � ❢ NT ✭ x ❀ β t ❀ θ ✵ ✮✮ ✷ ❪ ❀ ◮ Linear convergence: ☞ t ✦ ❫ β ❂ ✣ ✭ ❳ ✮ ② ❨ .

  16. Linear regression over random features ◮ NT model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ ✣ ✭ x ✮ ✐ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ ◮ (Random) feature map: ✣ ✭ ✁ ✮ ❂ r θ ❢ ✭ ✁ ❀ θ ✵ ✮ ✿ R ❞ ✦ R ◆ . ◮ Training dataset: ✭ ❳ ❀ ❨ ✮ ❂ ✭ x ✐ ❀ ② ✐ ✮ ✐ ✷ ❬ ♥ ❪ . ◮ Gradient flow dynamics: ❞ ❞ t β t ❂ � r β ❫ β ✵ ❂ 0 ✿ E ❬✭ ② � ❢ NT ✭ x ❀ β t ❀ θ ✵ ✮✮ ✷ ❪ ❀ ◮ Linear convergence: ☞ t ✦ ❫ β ❂ ✣ ✭ ❳ ✮ ② ❨ .

  17. Linear regression over random features ◮ NT model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ ✣ ✭ x ✮ ✐ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ ◮ (Random) feature map: ✣ ✭ ✁ ✮ ❂ r θ ❢ ✭ ✁ ❀ θ ✵ ✮ ✿ R ❞ ✦ R ◆ . ◮ Training dataset: ✭ ❳ ❀ ❨ ✮ ❂ ✭ x ✐ ❀ ② ✐ ✮ ✐ ✷ ❬ ♥ ❪ . ◮ Gradient flow dynamics: ❞ ❞ t β t ❂ � r β ❫ β ✵ ❂ 0 ✿ E ❬✭ ② � ❢ NT ✭ x ❀ β t ❀ θ ✵ ✮✮ ✷ ❪ ❀ ◮ Linear convergence: ☞ t ✦ ❫ β ❂ ✣ ✭ ❳ ✮ ② ❨ .

  18. Linear regression over random features ◮ NT model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ ✣ ✭ x ✮ ✐ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ ◮ (Random) feature map: ✣ ✭ ✁ ✮ ❂ r θ ❢ ✭ ✁ ❀ θ ✵ ✮ ✿ R ❞ ✦ R ◆ . ◮ Training dataset: ✭ ❳ ❀ ❨ ✮ ❂ ✭ x ✐ ❀ ② ✐ ✮ ✐ ✷ ❬ ♥ ❪ . ◮ Gradient flow dynamics: ❞ ❞ t β t ❂ � r β ❫ β ✵ ❂ 0 ✿ E ❬✭ ② � ❢ NT ✭ x ❀ β t ❀ θ ✵ ✮✮ ✷ ❪ ❀ ◮ Linear convergence: ☞ t ✦ ❫ β ❂ ✣ ✭ ❳ ✮ ② ❨ .

  19. Linear regression over random features ◮ NT model: the linear part of ❢ ❢ NT ✭ x ❀ β ❀ θ ✵ ✮ ❂ ❤ β ❀ ✣ ✭ x ✮ ✐ ❂ ❤ β ❀ r θ ❢ ✭ x ❀ θ ✵ ✮ ✐ ✿ ◮ (Random) feature map: ✣ ✭ ✁ ✮ ❂ r θ ❢ ✭ ✁ ❀ θ ✵ ✮ ✿ R ❞ ✦ R ◆ . ◮ Training dataset: ✭ ❳ ❀ ❨ ✮ ❂ ✭ x ✐ ❀ ② ✐ ✮ ✐ ✷ ❬ ♥ ❪ . ◮ Gradient flow dynamics: ❞ ❞ t β t ❂ � r β ❫ β ✵ ❂ 0 ✿ E ❬✭ ② � ❢ NT ✭ x ❀ β t ❀ θ ✵ ✮✮ ✷ ❪ ❀ ◮ Linear convergence: ☞ t ✦ ❫ β ❂ ✣ ✭ ❳ ✮ ② ❨ .

  20. Neural network ✙ Neural tangent Theorem [Jacot, Gabriel, Hongler, 2018] (Informal) Consider neural networks ❢ ◆ ✭ x ❀ θ ✮ with number of neurons ◆ , and consider ❞ ❞ t θ t ❂ � r θ ❫ θ ✵ ❂ θ ✵ ❀ E ❬✭ ② � ❢ ◆ ✭ x ❀ θ t ✮✮ ✷ ❪ ❀ ❞ ❞ t β t ❂ � r β ❫ β ✵ ❂ 0 ✿ E ❬✭ ② � ❢ ◆ NT ✭ x ❀ β t ❀ θ ✵ ✮✮ ✷ ❪ ❀ Under proper (random) initialization, we have a.s. ◆ ✦✶ ❥ ❢ ◆ ✭ x ❀ θ t ✮ � ❢ ◆ NT ✭ x ❀ β t ✮ ❥ ❂ ✵ ✿ ❧✐♠

  21. Optimization success Gradient flow of training loss of NN converges to global min ... ... with over-parameterization and proper initialization [Jacot, Gabriel, Hongler, 2018], [Du, Zhai, Poczos, Singh, 2018], [Du, Lee, Li, Wang, Zhai, 2018], [Allen-Zhu, Li, Song 2018], [Zou, Cao, Zhou, Gu, 2018], [Oymak, Soltanolkotabi, 2018] [Chizat, Bach, 2018b], ....

  22. Optimization success Gradient flow of training loss of NN converges to global min ... ... with over-parameterization and proper initialization [Jacot, Gabriel, Hongler, 2018], [Du, Zhai, Poczos, Singh, 2018], [Du, Lee, Li, Wang, Zhai, 2018], [Allen-Zhu, Li, Song 2018], [Zou, Cao, Zhou, Gu, 2018], [Oymak, Soltanolkotabi, 2018] [Chizat, Bach, 2018b], .... Does linearization fully explain the success of neural networks?

  23. Optimization success Gradient flow of training loss of NN converges to global min ... ... with over-parameterization and proper initialization [Jacot, Gabriel, Hongler, 2018], [Du, Zhai, Poczos, Singh, 2018], [Du, Lee, Li, Wang, Zhai, 2018], [Allen-Zhu, Li, Song 2018], [Zou, Cao, Zhou, Gu, 2018], [Oymak, Soltanolkotabi, 2018] [Chizat, Bach, 2018b], .... Does linearization fully explain the success of neural networks? Our answer is No

  24. Generalization Empirically, the generalization of NT models are not as good as NN Table: Cifar10 experiments Architecture Classification error CNN 4%- (1) CNTK 23% (2) CNTK 11% (3) Compositioal Kernel 10% (1) [Arora, Du, Hu, Li, Salakhutdinov, Wang, 2019] , (2) [Li, Wang, Yu, Du, Hu, Salakhutdinov, Arora, 2019] , (3) [Shankar, Fang, Guo, Fridovich-Keil, Schmidt, Ragan-Kelley, Recht, 2020] .

  25. Performance gap: NN versus NT

  26. Two-layers neural network ◆ ❳ ❢ ◆ ✭ x ❀ Θ ✮ ❂ ❛ ✐ ✛ ✭ ❤ w ✐ ❀ x ✐ ✮ ❀ Θ ❂ ✭ ❛ ✶ ❀ w ✶ ❀ ✿ ✿ ✿ ❀ ❛ ◆ ❀ w ◆ ✮ ✿ ✐ ❂✶ ◮ Input vector x ✷ R ❞ . ◮ Bottom layer weights w ✐ ✷ R ❞ , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ . ◮ Top layer weights ❛ ✐ ✷ R , ✐ ❂ ✶ ❀ ✷ ❀ ✿ ✿ ✿ ❀ ◆ .

Recommend

Neural Networks + Backpropagation Last Class Softmax Classifier Generalization /

Neural Networks + Backpropagation Last Class Softmax Classifier Generalization /

CS4501: Introduction to Computer Vision Neural Networks + Backpropagation Last Class Softmax Classifier Generalization / Overfitting Pytorch Todays Class Global Features The perceptron model Neural Networks

838 views • 34 slides
Learning From Data Lecture 22 Neural Networks and Overfitting Approximation vs. Generalization

Learning From Data Lecture 22 Neural Networks and Overfitting Approximation vs. Generalization

Learning From Data Lecture 22 Neural Networks and Overfitting Approximation vs. Generalization Regularization and Early Stopping Minimizing E in More Efficienty M. Magdon-Ismail CSCI 4100/6100 recap: Neural Networks and Fitting the Data

464 views • 15 slides
CSC413/2516 Lecture 7: Generalization & Recurrent Neural Networks Jimmy Ba Jimmy Ba

CSC413/2516 Lecture 7: Generalization & Recurrent Neural Networks Jimmy Ba Jimmy Ba

CSC413/2516 Lecture 7: Generalization & Recurrent Neural Networks Jimmy Ba Jimmy Ba CSC413/2516 Lecture 7: Generalization & Recurrent Neural Networks 1 / 57 Overview Weve focused so far on how to optimize neural nets how to get

971 views • 60 slides
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao

Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao

Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao and Quanquan Gu Computer Science Department 1 / 14 Learning Over-parameterized DNNs Empirical observation on extremely wide deep neural networks

631 views • 14 slides
Bayesian feedforward Neural networks Seung-Hoon Na Chonbuk National University Neural networks

Bayesian feedforward Neural networks Seung-Hoon Na Chonbuk National University Neural networks

Bayesian feedforward Neural networks Seung-Hoon Na Chonbuk National University Neural networks compared to GPs Neural networks: Nonlinear generalization of GLMs Here, defined by a logistic regression model applied to a logistic

357 views • 24 slides
Neural Networks for Machine Learning Lecture 9a Overview of ways to improve generalization

Neural Networks for Machine Learning Lecture 9a Overview of ways to improve generalization

Neural Networks for Machine Learning Lecture 9a Overview of ways to improve generalization Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed Reminder: Overfitting The training data contains information

1.17k views • 39 slides
Disentangling Trainability and Generalization In Deep Neural Networks Lechao Xiao, Je ff rey

Disentangling Trainability and Generalization In Deep Neural Networks Lechao Xiao, Je ff rey

Disentangling Trainability and Generalization In Deep Neural Networks Lechao Xiao, Je ff rey Pennington and Samuel S. Schoenholz Google Brain Team, Google Research Colab Tutorial Two Fundamental Theoretical Questions in Deep Learning

473 views • 15 slides
Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26,

Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26,

Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26, 2019 Joint work with Andrea Montanari, Theodor Misiakiewicz, Behrooz Ghorbani Song Mei (Stanford University) Linearized two layers neural network

436 views • 39 slides
Lecture 4: Recurrent Neural Networks + Generalization Prof. Manolis Kellis Slides credit:

Lecture 4: Recurrent Neural Networks + Generalization Prof. Manolis Kellis Slides credit:

6.874, 6.802, 20.390, 20.490, HST.506 Computational Systems Biology Deep Learning in the Life Sciences Lecture 4: Recurrent Neural Networks + Generalization Prof. Manolis Kellis Slides credit: Geoffrey Hinton, Ian Goodfellow,

1.19k views • 86 slides
Neural Tangent Kernel Convergence and Generalization in Neural Networks Arthur Jacot Franck

Neural Tangent Kernel Convergence and Generalization in Neural Networks Arthur Jacot Franck

Neural Tangent Kernel Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clment Hongler arthur.jacot@epfl.ch franck.gabriel@epfl.ch clement.hongler@epfl.ch What happens during training? One step of Gradient

248 views • 5 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides
Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides
CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides
Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides
Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides
Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides
Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides
Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What well cover ... f ( x ) computer vision architectures - convolutional networks - data augmentation 1 - residual networks ... ...

1.29k views • 75 slides

More recommend