Deep Learning Multi-Layer Perceptron Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Sigmoid Neuron Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning tanh Neuron Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning ReLU: Rectified Linear Units Nair & Hinton (2010); Maas, Hannun & Ng (2014) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Outline Antecedents Vision Case Study Antecedents 1 Building Blocks Theory Case Study: A History of Biological & Artificial Vision Neural Units Building Blocks Neural Nets Deep Neural Nets Application 2 Theory ConvNets LSTMs Biological & Artificial Neurons untapt Reinforcement Neural Networks Deep Neural Networks Contemporary Applications 3 Convolutional Neural Networks Long Short-Term Memory Recurrent Neural Networks Deep Learning at untapt Deep Reinforcement Learning
Deep Learning MNIST LeCun, Cortes & Burges Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Fully-Connected Neural Net Single Hidden Layer Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning TensorFlow Playground Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt [demo] Reinforcement
Deep Learning Outline Antecedents Vision Case Study Antecedents 1 Building Blocks Theory Case Study: A History of Biological & Artificial Vision Neural Units Building Blocks Neural Nets Deep Neural Nets Application 2 Theory ConvNets LSTMs Biological & Artificial Neurons untapt Reinforcement Neural Networks Deep Neural Networks Contemporary Applications 3 Convolutional Neural Networks Long Short-Term Memory Recurrent Neural Networks Deep Learning at untapt Deep Reinforcement Learning
Deep Learning Deep Fully-Connected Net 3 (or more) Hidden Layers Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning TFLearn Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application Caffe Torch Theano TensorFlow ConvNets Language C++, Python Lua Python Python LSTMs untapt Pretrained Yes++ Yes++ Yes (Lasagne) Inception Reinforcement Parallel GPUs: Data Yes Yes Yes Yes Parallel GPUs: Model No Yes Experimental Yes (best) Readable Source Code Yes (C++) Yes No No Good at RNN No Mediocre Yes Yes (best) Higher-Level APIs No No Keras Keras and TFLearn
Deep Learning A Simple Deep Net in TFLearn Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt [notebook] Reinforcement
Deep Learning Synaptic Pruning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning (Stochastic) Gradient Descent Adam = AdaGrad + RMSprop Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Backpropagation computes error & gradient of cost function Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Overfitting ...and avoiding it Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement • L1/L2 regularization • dropout • artificial data set expansion
Deep Learning Improving Neural Networks Mostly Hyperparameter Tuning Antecedents Vision Case Study Building Blocks Theory • problem simplification Neural Units Neural Nets Deep Neural Nets • number and width of layers Application • cost fxn: quadratic, cross-entropy, log-likelihood, &c. ConvNets LSTMs untapt • more epochs, early stopping Reinforcement • clever initialization of weights and biases • learning rate η , variable schedule • regularization parameter λ • mini-batch size • automation, e.g., with Spearmint [Summary Blog Post]
Deep Learning Universality Solve Any Continuous Function (Nielsen, 2015) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Unstable Gradient Typically Vanishes (but can Explode ) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Classic Deep Architectures ...introducing Convolutional Layers Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Outline Antecedents Vision Case Study Antecedents 1 Building Blocks Theory Case Study: A History of Biological & Artificial Vision Neural Units Building Blocks Neural Nets Deep Neural Nets Application 2 Theory ConvNets LSTMs Biological & Artificial Neurons untapt Reinforcement Neural Networks Deep Neural Networks Contemporary Applications 3 Convolutional Neural Networks Long Short-Term Memory Recurrent Neural Networks Deep Learning at untapt Deep Reinforcement Learning
Deep Learning Hubel & Wiesel (1959) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning AlexNet Krizhevsky, Sutskever & Hinton (2012) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning ConvNet Visualisation Yosinski et al. (2015) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [video]
Deep Learning Network Architectures Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning AlexNet: ILSVRC ‘12 winner Krizhevsky et al. (2012) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [TFLearn notebook]
Deep Learning VGGNet: ILSVRC ‘14 runner-up Simonyan & Zisserman (2015) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [TFLearn notebook]
Deep Learning ConvNet in TensorFlow Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application Caffe Torch Theano TensorFlow ConvNets Language C++, Python Lua Python Python LSTMs untapt Pretrained Yes++ Yes++ Yes (Lasagne) Inception Reinforcement Parallel GPUs: Data Yes Yes Yes Yes Parallel GPUs: Model No Yes Experimental Yes (best) Readable Source Code Yes (C++) Yes No No Good at RNN No Mediocre Yes Yes (best) Higher-Level APIs No No Keras Keras and TFLearn
Deep Learning ConvNet in TensorFlow Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt [notebook] Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning ConvNet in Theano Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application Caffe Torch Theano TensorFlow ConvNets Language C++, Python Lua Python Python LSTMs untapt Pretrained Yes++ Yes++ Yes (Lasagne) Inception Reinforcement Parallel GPUs: Data Yes Yes Yes Yes Parallel GPUs: Model No Yes Experimental Yes (best) Readable Source Code Yes (C++) Yes No No Good at RNN No Mediocre Yes Yes (best) Higher-Level APIs No No Keras Keras and TFLearn
Deep Learning ConvNet in Theano Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt [demo] Reinforcement
Deep Learning ConvNet in Keras calls TensorFlow or Theano Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets Caffe Torch Theano TensorFlow LSTMs untapt Language C++, Python Lua Python Python Reinforcement Pretrained Yes++ Yes++ Yes (Lasagne) Inception Parallel GPUs: Data Yes Yes Yes Yes Parallel GPUs: Model No Yes Experimental Yes (best) Readable Source Code Yes (C++) Yes No No Good at RNN No Mediocre Yes Yes (best) Higher-Level APIs No No Keras Keras and TFLearn
Deep Learning ConvNet in Keras calls TensorFlow or Theano Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [notebook]
Deep Learning “2.5-dimension” CT Scans Roth et al. (2015) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Computer-Aided Detection Shin et al. (2016); Roth et al. (2016) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Kaggle Data Science Bowl 2017 Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [link]
Deep Learning Transfer Learning Caffe Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets Caffe Torch Theano TensorFlow LSTMs untapt Language C++, Python Lua Python Python Reinforcement Pretrained Yes++ Yes++ Yes (Lasagne) Inception Parallel GPUs: Data Yes Yes Yes Yes Parallel GPUs: Model No Yes Experimental Yes (best) Readable Source Code Yes (C++) Yes No No Good at RNN No Mediocre Yes Yes (best) Higher-Level APIs No No Keras Keras and TFLearn
Deep Learning Transfer Learning Caffe Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [Model Zoo]
Deep Learning Transfer Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Video Classification Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt [video] Reinforcement
Deep Learning Outline Antecedents Vision Case Study Antecedents 1 Building Blocks Theory Case Study: A History of Biological & Artificial Vision Neural Units Building Blocks Neural Nets Deep Neural Nets Application 2 Theory ConvNets LSTMs Biological & Artificial Neurons untapt Reinforcement Neural Networks Deep Neural Networks Contemporary Applications 3 Convolutional Neural Networks Long Short-Term Memory Recurrent Neural Networks Deep Learning at untapt Deep Reinforcement Learning
Deep Learning Sunspring Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Sunspring Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning RNNs; LSTM RNNs Hochreiter & Schmidhuber (1997) Antecedents Graves, ... & Schmidhuber (2009) Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Vector Space Embedding Word2Vec: Mikolov, ... & Dean (2013) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning t-SNE Hinton & van der Maaten (2008) Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Word2Vec + t-SNE Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning Word2Vec + t-SNE Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning ‘Understand’ Language with Word2Vec features in your model Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [even with small corpora]
Deep Learning Quick, Draw! ConvNet + LSTM Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement [link]
Deep Learning Outline Antecedents Vision Case Study Antecedents 1 Building Blocks Theory Case Study: A History of Biological & Artificial Vision Neural Units Building Blocks Neural Nets Deep Neural Nets Application 2 Theory ConvNets LSTMs Biological & Artificial Neurons untapt Reinforcement Neural Networks Deep Neural Networks Contemporary Applications 3 Convolutional Neural Networks Long Short-Term Memory Recurrent Neural Networks Deep Learning at untapt Deep Reinforcement Learning
Deep Learning untapt Digital Recruitment Platform Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning untapt Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning untapt Candidate-Side Feedback Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning untapt Client-Side Feedback Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement
Deep Learning untapt Multi-Stage Bayesian Regression with PyMC3 Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement Krohn, Rives-Corbett & Donner (2016)
Deep Learning untapt Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement Krohn, Rives-Corbett & Donner (2016)
Deep Learning untapt Ensemble with Deep Neural Net Antecedents Vision Case Study Building Blocks Theory Neural Units Neural Nets Deep Neural Nets Application ConvNets LSTMs untapt Reinforcement deep-orange.untapt.com
Recommend
More recommend