a mean field view of the landscape of two layers neural
play

A Mean Field View of the Landscape of Two-Layers Neural Networks - PowerPoint PPT Presentation

Apr 26, 2023 •93 likes •807 views

A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Stanford University November 14, 2018 Joint work with Andrea Montanari and Phan-Minh Nguyen Song Mei (Stanford University) Mean Field Dynamics for Neural Network

  1. A Mean Field View of the Landscape of Two-Layers Neural Networks Song Mei Stanford University November 14, 2018 Joint work with Andrea Montanari and Phan-Minh Nguyen Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 1 / 37

  2. Empirical surprise [Zhang, Bengio, Hardt, Recht, Vinyals, 2016] ◮ Overparameterized regime. ◮ Efficiently fit all the data. ◮ Generalize well. Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 2 / 37

  3. Empirical surprise ◮ Overparameterized regime. ◮ Efficiently fit all the data. ◮ Generalize well. Questions ◮ Why can complex neural network be optimized efficiently? ◮ Why does overparameterization not harm generalization? Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 3 / 37

  4. Two-layers neural networks Input layer Hidden layer Output layer Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 4 / 37

  5. Two-layers neural networks ◮ Parameter: θ ❂ ✭ θ ✶ ❀ ✿ ✿ ✿ ❀ θ ◆ ✮ ✷ R ◆ ✂ ❉ . ◮ Prediction: ◆ ② ✭ x ❀ θ ✮ ❂ ✶ ❳ ❫ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ ◮ An example: θ ✐ ❂ ✭ ❛ ✐ ❀ w ✐ ✮ , ✛ ❄ ✭ x ❀ θ ✐ ✮ ❂ ❛ ✐ ✛ ✭ ❤ x ❀ w ✐ ✐ ✮ . ◮ Data distribution: ✭ x ❀ ② ✮ ✘ P x ❀② . ◮ Risk function: ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 5 / 37

  6. Two-layers neural networks ◮ Parameter: θ ❂ ✭ θ ✶ ❀ ✿ ✿ ✿ ❀ θ ◆ ✮ ✷ R ◆ ✂ ❉ . ◮ Prediction: ◆ ② ✭ x ❀ θ ✮ ❂ ✶ ❳ ❫ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ ◮ An example: θ ✐ ❂ ✭ ❛ ✐ ❀ w ✐ ✮ , ✛ ❄ ✭ x ❀ θ ✐ ✮ ❂ ❛ ✐ ✛ ✭ ❤ x ❀ w ✐ ✐ ✮ . ◮ Data distribution: ✭ x ❀ ② ✮ ✘ P x ❀② . ◮ Risk function: ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 5 / 37

  7. Two-layers neural networks ◮ Parameter: θ ❂ ✭ θ ✶ ❀ ✿ ✿ ✿ ❀ θ ◆ ✮ ✷ R ◆ ✂ ❉ . ◮ Prediction: ◆ ② ✭ x ❀ θ ✮ ❂ ✶ ❳ ❫ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ ◮ An example: θ ✐ ❂ ✭ ❛ ✐ ❀ w ✐ ✮ , ✛ ❄ ✭ x ❀ θ ✐ ✮ ❂ ❛ ✐ ✛ ✭ ❤ x ❀ w ✐ ✐ ✮ . ◮ Data distribution: ✭ x ❀ ② ✮ ✘ P x ❀② . ◮ Risk function: ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 5 / 37

  8. Two-layers neural networks ◮ Parameter: θ ❂ ✭ θ ✶ ❀ ✿ ✿ ✿ ❀ θ ◆ ✮ ✷ R ◆ ✂ ❉ . ◮ Prediction: ◆ ② ✭ x ❀ θ ✮ ❂ ✶ ❳ ❫ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ ◮ An example: θ ✐ ❂ ✭ ❛ ✐ ❀ w ✐ ✮ , ✛ ❄ ✭ x ❀ θ ✐ ✮ ❂ ❛ ✐ ✛ ✭ ❤ x ❀ w ✐ ✐ ✮ . ◮ Data distribution: ✭ x ❀ ② ✮ ✘ P x ❀② . ◮ Risk function: ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 5 / 37

  9. Two-layers neural networks ◮ Parameter: θ ❂ ✭ θ ✶ ❀ ✿ ✿ ✿ ❀ θ ◆ ✮ ✷ R ◆ ✂ ❉ . ◮ Prediction: ◆ ② ✭ x ❀ θ ✮ ❂ ✶ ❳ ❫ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ ◮ An example: θ ✐ ❂ ✭ ❛ ✐ ❀ w ✐ ✮ , ✛ ❄ ✭ x ❀ θ ✐ ✮ ❂ ❛ ✐ ✛ ✭ ❤ x ❀ w ✐ ✐ ✮ . ◮ Data distribution: ✭ x ❀ ② ✮ ✘ P x ❀② . ◮ Risk function: ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ✐ ✮ ✿ ◆ ✐ ❂✶ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 5 / 37

  10. Two-layers neural networks Input layer Hidden layer Output layer w 1 a 1 w 2 a 2 w 3 a 3 a 4 w 4 Figure: θ ✐ ❂ ✭ ❛ ✐ ❀ w ✐ ✮ . Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 6 / 37

  11. Related literatures (before 2018) ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ❥ ✮ ✿ ◆ ❥ ❂✶ ◮ Landscape analysis: [Soudry, Carmon, 2016], [Freeman, Bruna, 2016], [Ge, Lee, Ma, 2017], [Soltanolkotabi, Javanmard, Lee, 2017], [Zhong, Song, Jain, Bartlett, Dhillon, 2017]... ◮ Optimization dynamics: [Tian, 2017], [Soltanolkotabi, 2017], [Li, Yuan, 2017]... ◮ Generalization: [Bartlett, Foster, Telgarsky, 2017], [Neyshabur, Bhojanapalli, McAllester, Srebro, 2017]... Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 7 / 37

  12. Related literatures (before 2018) ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ❥ ✮ ✿ ◆ ❥ ❂✶ ◮ Landscape analysis: [Soudry, Carmon, 2016], [Freeman, Bruna, 2016], [Ge, Lee, Ma, 2017], [Soltanolkotabi, Javanmard, Lee, 2017], [Zhong, Song, Jain, Bartlett, Dhillon, 2017]... ◮ Optimization dynamics: [Tian, 2017], [Soltanolkotabi, 2017], [Li, Yuan, 2017]... ◮ Generalization: [Bartlett, Foster, Telgarsky, 2017], [Neyshabur, Bhojanapalli, McAllester, Srebro, 2017]... Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 7 / 37

  13. Related literatures (before 2018) ◆ ② � ✶ ✑ ✷ ✐ ❤✏ ❳ ❘ ◆ ✭ θ ✮ ❂ E x ❀② ✛ ❄ ✭ x ❀ θ ❥ ✮ ✿ ◆ ❥ ❂✶ ◮ Landscape analysis: [Soudry, Carmon, 2016], [Freeman, Bruna, 2016], [Ge, Lee, Ma, 2017], [Soltanolkotabi, Javanmard, Lee, 2017], [Zhong, Song, Jain, Bartlett, Dhillon, 2017]... ◮ Optimization dynamics: [Tian, 2017], [Soltanolkotabi, 2017], [Li, Yuan, 2017]... ◮ Generalization: [Bartlett, Foster, Telgarsky, 2017], [Neyshabur, Bhojanapalli, McAllester, Srebro, 2017]... Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 7 / 37

  14. Overparameterization: what happens for large ◆ ? [Bengio, et. al, 2006]. Expand the square ◮ ◆ ◆ ❘ ◆ ✭ θ ✮ ❂ E ❬ ② ✷ ❪ ✰ ✷ ❱ ✭ θ ✐ ✮ ✰ ✶ ❳ ❳ ❯ ✭ θ ✐ ❀ θ ❥ ✮ ❀ ◆ ✷ ◆ ✐ ❂✶ ✐❀❥ ❂✶ where ❱ ✭ θ ✐ ✮ ❂ � E ❬ ②✛ ❄ ✭ x ❀ θ ✐ ✮❪ ❀ ❯ ✭ θ ✐ ❀ θ ❥ ✮ ❂ E ❬ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✛ ❄ ✭ x ❀ θ ❥ ✮❪ ✿ ◮ ❘ ◆ depends on ✭ θ ✐ ✮ ✐ ✔ ◆ through ✚ ◆ ❂ ✭✶ ❂◆ ✮ P ◆ ✐ ❂✶ ✍ θ ✐ . ◮ Motivate us to define ❘ ✭ ✚ ✮ , ✚ ✷ P ✭ R ❉ ✮ , ❩ ❩ ❘ ✭ ✚ ✮ ❂ E ❬ ② ✷ ❪ ✰ ✷ ❱ ✭ θ ✮ ✚ ✭❞ θ ✮ ✰ ❯ ✭ θ ✶ ❀ θ ✷ ✮ ✚ ✭❞ θ ✶ ✮ ✚ ✭❞ θ ✷ ✮ ✿ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 8 / 37

  15. Overparameterization: what happens for large ◆ ? [Bengio, et. al, 2006]. Expand the square ◮ ◆ ◆ ❘ ◆ ✭ θ ✮ ❂ E ❬ ② ✷ ❪ ✰ ✷ ❱ ✭ θ ✐ ✮ ✰ ✶ ❳ ❳ ❯ ✭ θ ✐ ❀ θ ❥ ✮ ❀ ◆ ✷ ◆ ✐ ❂✶ ✐❀❥ ❂✶ where ❱ ✭ θ ✐ ✮ ❂ � E ❬ ②✛ ❄ ✭ x ❀ θ ✐ ✮❪ ❀ ❯ ✭ θ ✐ ❀ θ ❥ ✮ ❂ E ❬ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✛ ❄ ✭ x ❀ θ ❥ ✮❪ ✿ ◮ ❘ ◆ depends on ✭ θ ✐ ✮ ✐ ✔ ◆ through ✚ ◆ ❂ ✭✶ ❂◆ ✮ P ◆ ✐ ❂✶ ✍ θ ✐ . ◮ Motivate us to define ❘ ✭ ✚ ✮ , ✚ ✷ P ✭ R ❉ ✮ , ❩ ❩ ❘ ✭ ✚ ✮ ❂ E ❬ ② ✷ ❪ ✰ ✷ ❱ ✭ θ ✮ ✚ ✭❞ θ ✮ ✰ ❯ ✭ θ ✶ ❀ θ ✷ ✮ ✚ ✭❞ θ ✶ ✮ ✚ ✭❞ θ ✷ ✮ ✿ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 8 / 37

  16. Overparameterization: what happens for large ◆ ? [Bengio, et. al, 2006]. Expand the square ◮ ◆ ◆ ❘ ◆ ✭ θ ✮ ❂ E ❬ ② ✷ ❪ ✰ ✷ ❱ ✭ θ ✐ ✮ ✰ ✶ ❳ ❳ ❯ ✭ θ ✐ ❀ θ ❥ ✮ ❀ ◆ ✷ ◆ ✐ ❂✶ ✐❀❥ ❂✶ where ❱ ✭ θ ✐ ✮ ❂ � E ❬ ②✛ ❄ ✭ x ❀ θ ✐ ✮❪ ❀ ❯ ✭ θ ✐ ❀ θ ❥ ✮ ❂ E ❬ ✛ ❄ ✭ x ❀ θ ✐ ✮ ✛ ❄ ✭ x ❀ θ ❥ ✮❪ ✿ ◮ ❘ ◆ depends on ✭ θ ✐ ✮ ✐ ✔ ◆ through ✚ ◆ ❂ ✭✶ ❂◆ ✮ P ◆ ✐ ❂✶ ✍ θ ✐ . ◮ Motivate us to define ❘ ✭ ✚ ✮ , ✚ ✷ P ✭ R ❉ ✮ , ❩ ❩ ❘ ✭ ✚ ✮ ❂ E ❬ ② ✷ ❪ ✰ ✷ ❱ ✭ θ ✮ ✚ ✭❞ θ ✮ ✰ ❯ ✭ θ ✶ ❀ θ ✷ ✮ ✚ ✭❞ θ ✶ ✮ ✚ ✭❞ θ ✷ ✮ ✿ Song Mei (Stanford University) Mean Field Dynamics for Neural Network November 14, 2018 8 / 37

Recommend

Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit Song Mei,

Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit Song Mei,

Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit Song Mei, Theodor Misiakiewicz, and Andrea Montanari Stanford University June 26, 2019 COLT 2019 Song Mei (Stanford University) Mean Field Dynamics for

1.12k views • 29 slides
Integer GEMM (under)performance Marat Dukhan Software Engineer on Ca ff e 2 GEMM in Neural

Integer GEMM (under)performance Marat Dukhan Software Engineer on Ca ff e 2 GEMM in Neural

Integer GEMM (under)performance Marat Dukhan Software Engineer on Ca ff e 2 GEMM in Neural Networks Fully-connected layers im2col+GEMM algorithm for convolution 1x1 convolutional layers Android CPU Landscape Overview of CPU

449 views • 17 slides
Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26,

Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26,

Linearized two-layers neural network in high dimensions Song Mei Stanford University May 26, 2019 Joint work with Andrea Montanari, Theodor Misiakiewicz, Behrooz Ghorbani Song Mei (Stanford University) Linearized two layers neural network

436 views • 39 slides
Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a

Perceptrons 2-29-16 What is a neural network? activation connection functions A NN is a directed acyclic graph. weights Nodes are organized into layers. -0.5 Consecutive layers are fully connected. 1.5 0.2 Edges have

147 views • 13 slides
Limitations of recurrent neural networks N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath

Limitations of recurrent neural networks N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath

Limitations of recurrent neural networks N ATURAL LAN GUAGE GEN ERATION IN P YTH ON Biswanath Halder Data Scientist Simple neural networks Nodes arranged in layers. Nodes in different layers connected by weights. Input to rst layer :

802 views • 29 slides
Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural

Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural

Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks Alexander Shevchenko Marco Mondelli Neural Network Training From theoretical perspective training of neural networks is di ffi cult

494 views • 28 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Analyzing Backprop 3-4-16 Reading Quiz Q1: If a neural network has 3 layers with 10 input, 6

Analyzing Backprop 3-4-16 Reading Quiz Q1: If a neural network has 3 layers with 10 input, 6

Analyzing Backprop 3-4-16 Reading Quiz Q1: If a neural network has 3 layers with 10 input, 6 hidden, and 8 output units, what is the dimension of backpropagations local search space? a) 10 + 6 + 8 = 24 b) 10 + 6 * 8 = 58 c) 10 * 6 + 6 *

381 views • 12 slides
8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural

8 Neural MT 2: Attentional Neural MT In the past chapter, we described a simple model for neural machine translation, which uses an encoder to encode sentences as a fixed-length vector. However, in some ways, this view is overly simplified, and

635 views • 9 slides
Deep Residual Output Layers for Neural Language Generation Nikolaos Pappas, James Henderson June

Deep Residual Output Layers for Neural Language Generation Nikolaos Pappas, James Henderson June

Deep Residual Output Layers for Neural Language Generation Nikolaos Pappas, James Henderson June 13, 2019 Neural language generation Cat sat on ? Probability distribution at time t given context vector h t R d , weights W R d |V|

474 views • 14 slides
Lecture 11: Convolutional Neural Networks 2 CS109B Data Science 2 Pavlos Protopapas, Mark

Lecture 11: Convolutional Neural Networks 2 CS109B Data Science 2 Pavlos Protopapas, Mark

Lecture 11: Convolutional Neural Networks 2 CS109B Data Science 2 Pavlos Protopapas, Mark Glickman and Chris Tanner 1 Outline Review from last lecture 1. Training CNNs 2. BackProp of MaxPooling layer 3. Layers Receptive Field 4.

2.22k views • 89 slides
Lecture 9: Convolutional Neural Networks 2 CS109B Data Science 2 Pavlos Protopapas and Mark

Lecture 9: Convolutional Neural Networks 2 CS109B Data Science 2 Pavlos Protopapas and Mark

Lecture 9: Convolutional Neural Networks 2 CS109B Data Science 2 Pavlos Protopapas and Mark Glickman 1 Outline 1. Review from last lecture 2. BackProp of MaxPooling layer 3. A bit of history 4. Layers Receptive Field 5. Saliency maps 6.

1.15k views • 68 slides
The committee machine: Computational to statistical gaps in learning a two-layers neural network

The committee machine: Computational to statistical gaps in learning a two-layers neural network

The committee machine: Computational to statistical gaps in learning a two-layers neural network Benjamin Aubin , Antoine Maillard, Jean Barbier Nicolas Macris, Florent Krzakala & Lenka Zdeborov Benjamin Aubin - Institut de Physique T

1.2k views • 32 slides
Cost function Machine Learning Neural Network (Classification) total no. of layers in network

Cost function Machine Learning Neural Network (Classification) total no. of layers in network

Neural Networks: Learning Cost function Machine Learning Neural Network (Classification) total no. of layers in network no. of units (not counting bias unit) in layer Layer 1 Layer 2 Layer 3 Layer 4 Multi-class classification (K classes)

711 views • 33 slides
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation Simon

The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation Simon

The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation Simon Jegou , Michal Drozdzal, David Vazquez, Adriana Romero Yoshua Bengio 1 Deep Neural Network use a cascade of multiple layers of units for

640 views • 29 slides
M IND Field Calculations Bob Wands April 27, 2011 1 Overview of M IND Toroids The M IND

M IND Field Calculations Bob Wands April 27, 2011 1 Overview of M IND Toroids The M IND

M IND Field Calculations Bob Wands April 27, 2011 1 Overview of M IND Toroids The M IND toroids are octagonal planes with a 14 meter span across the flats There are two layers of 1.5 cm thick iron per plane Layers are

202 views • 17 slides
Understanding Neural Networks Part II: Convolutional Layers and Collaborative Filters Nick

Understanding Neural Networks Part II: Convolutional Layers and Collaborative Filters Nick

TensorFlow Workshop 2018 Understanding Neural Networks Part II: Convolutional Layers and Collaborative Filters Nick Winovich Department of Mathematics Purdue University July 2018 SIAM@Purdue 2018 - Nick Winovich Understanding Neural Networks

1.21k views • 64 slides
Neural Architecture Search in a Proxy Validation Loss Landscape Yanxi Li 1 , Minjing Dong 1 ,

Neural Architecture Search in a Proxy Validation Loss Landscape Yanxi Li 1 , Minjing Dong 1 ,

Neural Architecture Search in a Proxy Validation Loss Landscape Yanxi Li 1 , Minjing Dong 1 , Yunhe Wang 2 , Chang Xu 1 1 University of Sydney 2 Huawei Noah's Ark Lab. Aim Improve the efficiency of Neural Architecture Search (NAS) via learning a

580 views • 26 slides
ON THE OPTIMIZATION LANDSCAPE OF NEURAL NETWORKS JOAN BRUNA , CIMS + CDS, NYU in collaboration

ON THE OPTIMIZATION LANDSCAPE OF NEURAL NETWORKS JOAN BRUNA , CIMS + CDS, NYU in collaboration

ON THE OPTIMIZATION LANDSCAPE OF NEURAL NETWORKS JOAN BRUNA , CIMS + CDS, NYU in collaboration with D.Freeman (UC Berkeley), Luca Venturi & Afonso Bandeira (NYU) MOTIVATION We consider the standard Empirical Risk Minimization setup: `

683 views • 46 slides
Bayesian neural networks: a function space view tour Yingzhen Li Microsoft Research Cambridge

Bayesian neural networks: a function space view tour Yingzhen Li Microsoft Research Cambridge

Bayesian neural networks: a function space view tour Yingzhen Li Microsoft Research Cambridge Neural networks 101 Lets say we want to classify different types of cats x : input images; y : output label "cat" build a neural

1.05k views • 45 slides
Pace Layers The social economy ecosystem has many layers, all of which change at different

Pace Layers The social economy ecosystem has many layers, all of which change at different

Pace Layers The social economy ecosystem has many layers, all of which change at different rates, the outer layers moving faster than the inner Source: Stewart Brand, The Clock of the Long Now 1 | Confidential & Proprietary 2 |

421 views • 4 slides
Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
View Volumes Canonical View Volumes Why Canonical View Volumes? University of British Columbia

View Volumes Canonical View Volumes Why Canonical View Volumes? University of British Columbia

View Volumes Canonical View Volumes Why Canonical View Volumes? University of British Columbia CPSC 314 Computer Graphics Jan-Apr 2016 standardized viewing volume representation specifies field-of-view, used for clipping permits

52 views • 3 slides
A view into the energy company of tomorrow proactive adaptation to the evolving landscape

A view into the energy company of tomorrow proactive adaptation to the evolving landscape

A view into the energy company of tomorrow proactive adaptation to the evolving landscape Sicelo Xulu Managing Director City Power, Johannesburg South Africa Contents 1 Key Global and African trends 2 Drivers of change 3 Whats in the

547 views • 26 slides

More recommend