andrej karpathy
play

Andrej Karpathy Bay Area Deep Learning School, 2016 So far... So - PowerPoint PPT Presentation

Mar 06, 2023 •231 likes •1.27k views

Andrej Karpathy Bay Area Deep Learning School, 2016 So far... So far... Some input vector (very few assumptions made). In many real-world applications input vectors have structure . Spectrograms Text Images Convolutional Neural Networks: A

  1. Case Study: AlexNet [Krizhevsky et al. 2012] Full (simplified) AlexNet architecture: [227x227x3] INPUT [55x55x96] CONV1: 96 11x11 filters at stride 4, pad 0 [27x27x96] MAX POOL1: 3x3 filters at stride 2 [27x27x96] NORM1: Normalization layer [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 [13x13x256] MAX POOL2: 3x3 filters at stride 2 [13x13x256] NORM2: Normalization layer [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 [6x6x256] MAX POOL3: 3x3 filters at stride 2 [4096] FC6: 4096 neurons [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons (class scores)

  2. Case Study: AlexNet [Krizhevsky et al. 2012] Full (simplified) AlexNet architecture: [227x227x3] INPUT Compared to LeCun 1998: [55x55x96] CONV1: 96 11x11 filters at stride 4, pad 0 [27x27x96] MAX POOL1: 3x3 filters at stride 2 [27x27x96] NORM1: Normalization layer 1 DATA: [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 [13x13x256] MAX POOL2: 3x3 filters at stride 2 - More data: 10^6 vs. 10^3 [13x13x256] NORM2: Normalization layer 2 COMPUTE: [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 - GPU (~20x speedup) [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 3 ALGORITHM: [6x6x256] MAX POOL3: 3x3 filters at stride 2 - Deeper: More layers (8 weight layers) [4096] FC6: 4096 neurons - Fancy regularization (dropout) [4096] FC7: 4096 neurons - Fancy non-linearity (ReLU) [1000] FC8: 1000 neurons (class scores) 4 INFRASTRUCTURE: - CUDA

  3. Case Study: AlexNet [Krizhevsky et al. 2012] Full (simplified) AlexNet architecture: [227x227x3] INPUT [55x55x96] CONV1: 96 11x11 filters at stride 4, pad 0 Details/Retrospectives: [27x27x96] MAX POOL1: 3x3 filters at stride 2 - first use of ReLU [27x27x96] NORM1: Normalization layer - used Norm layers (not common anymore) [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 - heavy data augmentation [13x13x256] MAX POOL2: 3x3 filters at stride 2 - dropout 0.5 [13x13x256] NORM2: Normalization layer - batch size 128 [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 - SGD Momentum 0.9 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 - Learning rate 1e-2, reduced by 10 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 manually when val accuracy plateaus [6x6x256] MAX POOL3: 3x3 filters at stride 2 - L2 weight decay 5e-4 [4096] FC6: 4096 neurons - 7 CNN ensemble: 18.2% -> 15.4% [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons (class scores)

  4. Case Study: ZFNet [Zeiler and Fergus, 2013] AlexNet but: CONV1: change from (11x11 stride 4) to (7x7 stride 2) CONV3,4,5: instead of 384, 384, 256 filters use 512, 1024, 512 ImageNet top 5 error: 15.4% -> 14.8%

  5. Case Study: VGGNet [Simonyan and Zisserman, 2014] Only 3x3 CONV stride 1, pad 1 and 2x2 MAX POOL stride 2 best model 11.2% top 5 error in ILSVRC 2013 -> 7.3% top 5 error

  6. (not counting biases) INPUT: [224x224x3] memory: 224*224*3=150K params: 0 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*3)*64 = 1,728 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*64)*64 = 36,864 POOL2: [112x112x64] memory: 112*112*64=800K params: 0 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*64)*128 = 73,728 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*128)*128 = 147,456 POOL2: [56x56x128] memory: 56*56*128=400K params: 0 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*128)*256 = 294,912 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 POOL2: [28x28x256] memory: 28*28*256=200K params: 0 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*256)*512 = 1,179,648 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 POOL2: [14x14x512] memory: 14*14*512=100K params: 0 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 POOL2: [7x7x512] memory: 7*7*512=25K params: 0 FC: [1x1x4096] memory: 4096 params: 7*7*512*4096 = 102,760,448 FC: [1x1x4096] memory: 4096 params: 4096*4096 = 16,777,216 FC: [1x1x1000] memory: 1000 params: 4096*1000 = 4,096,000

  7. (not counting biases) INPUT: [224x224x3] memory: 224*224*3=150K params: 0 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*3)*64 = 1,728 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*64)*64 = 36,864 POOL2: [112x112x64] memory: 112*112*64=800K params: 0 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*64)*128 = 73,728 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*128)*128 = 147,456 POOL2: [56x56x128] memory: 56*56*128=400K params: 0 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*128)*256 = 294,912 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 POOL2: [28x28x256] memory: 28*28*256=200K params: 0 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*256)*512 = 1,179,648 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 POOL2: [14x14x512] memory: 14*14*512=100K params: 0 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 POOL2: [7x7x512] memory: 7*7*512=25K params: 0 FC: [1x1x4096] memory: 4096 params: 7*7*512*4096 = 102,760,448 FC: [1x1x4096] memory: 4096 params: 4096*4096 = 16,777,216 FC: [1x1x1000] memory: 1000 params: 4096*1000 = 4,096,000 TOTAL memory: 24M * 4 bytes ~= 93MB / image (only forward! ~*2 for bwd) TOTAL params: 138M parameters

  8. (not counting biases) INPUT: [224x224x3] memory: 224*224*3=150K params: 0 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*3)*64 = 1,728 Note: CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*64)*64 = 36,864 POOL2: [112x112x64] memory: 112*112*64=800K params: 0 Most memory is in CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*64)*128 = 73,728 early CONV CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*128)*128 = 147,456 POOL2: [56x56x128] memory: 56*56*128=400K params: 0 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*128)*256 = 294,912 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 POOL2: [28x28x256] memory: 28*28*256=200K params: 0 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*256)*512 = 1,179,648 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 POOL2: [14x14x512] memory: 14*14*512=100K params: 0 Most params are CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 in late FC CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 POOL2: [7x7x512] memory: 7*7*512=25K params: 0 FC: [1x1x4096] memory: 4096 params: 7*7*512*4096 = 102,760,448 FC: [1x1x4096] memory: 4096 params: 4096*4096 = 16,777,216 FC: [1x1x1000] memory: 1000 params: 4096*1000 = 4,096,000 TOTAL memory: 24M * 4 bytes ~= 93MB / image (only forward! ~*2 for bwd) TOTAL params: 138M parameters

  9. Case Study: GoogLeNet [Szegedy et al., 2014] Inception module ILSVRC 2014 winner (6.7% top 5 error)

  10. Case Study: GoogLeNet Fun features: - Only 5 million params! (Removes FC layers completely) Compared to AlexNet: - 12X less params - 2x more compute - 6.67% (vs. 16.4%)

  11. Case Study: ResNet [He et al., 2015] ILSVRC 2015 winner (3.6% top 5 error) Slide from Kaiming He’s recent presentation https://www.youtube.com/watch?v=1PGLj-uKT1w

  12. (slide from Kaiming He’s recent presentation)

  14. Case Study: ResNet 224x224x3 spatial dimension [He et al., 2015] only 56x56!

  15. Identity Mappings in Deep Residual Networks, He et al. 2016

  16. Deep Networks with Stochastic Depth , Huang et al., 2016 “We start with very deep networks but during training, for each mini-batch, randomly drop a subset of layers and bypass them with the identity function.” x y Think of layers more like vector fields, nudging the input to the label

  17. Wide Residual Networks , Zagoruyko and Komodakis, 2016 - wide networks with only 16 layers can significantly outperform 1000-layer deep networks - main power of residual networks is in residual blocks, and not in extreme depth - wide residual networks are several times faster to train Swapout: Learning an ensemble of deep architectures , Singh et al., 2016 - 32 layer wider model performs similar to a 1001 layer ResNet model FractalNet: Ultra-Deep Neural Networks without Residuals, Larsson et al. 2016

  18. Still an active area of research... Densely Connected Convolutional Networks, Huang et al. ResNet in ResNet , Targ et al. Deeply-Fused Nets, Wang et al. Weighted Residuals for Very Deep Networks, Shen et al. Residual Networks of Residual Networks: Multilevel Residual Networks , Zhang et al. ... In large part likely due to open source code available, e.g.:

  19. ASIDE: arxiv-sanity.com plug

  20. Addressing other tasks...

  21. Addressing other tasks... CNN features image 7x7x512 224x224x3 A block of compute with a few million parameters.

  22. Addressing other tasks... predicted thing CNN features image 7x7x512 desired thing 224x224x3 A block of compute with a few million parameters.

  23. Addressing other tasks... this part changes from task to task predicted thing CNN features image 7x7x512 desired thing 224x224x3 A block of compute with a few million parameters.

  24. Image Classification thing = a vector of probabilities for different classes fully connected layer CNN image features 7x7x512 224x224x3 e.g. vector of 1000 numbers giving probabilities for different classes.

  25. Image Captioning RNN CNN image features 7x7x512 224x224x3 A sequence of 10,000-dimensional vectors giving probabilities of different words in the caption.

  26. Localization Class probabilities (as before) fully connected layer CNN image features 7x7x512 224x224x3 4 numbers: - X coord - Y coord - Width - Height

  27. Reinforcement Learning Mnih et al. 2015 fully connected CNN image features 160x210x3 e.g. vector of 8 numbers giving probability of wanting to take any of the 8 possible ATARI actions.

  28. image class “map” Segmentation deconv layers CNN image features 7x7x512 224x224x20 224x224x3 array of class probabilities at each pixel.

  29. Autoencoders deconv layers CNN image features 7x7x512 224x224x3 224x224x3 original image

  30. Variational Autoencoders reparameterization layer deconv layers CNN image features 7x7x512 224x224x3 224x224x3 original image [Kingma et al.], [Rezende et al.], [Salimans et al.]

  31. Detection 1x1 CONV CNN image features 7x7x512 7x7x(5*B+C) 224x224x3 For each of 7x7 locations: - [x,y,width,height,confidence]*B - class E.g. YOLO: You Only Look Once (Demo: http://pjreddie.com/darknet/yolo/)

  32. Dense Image Captioning 1x1 CONV CNN image features 7x7x512 7x7x(5*B+[C,..]) 224x224x3 For each of 7x7 locations: - x,y,width,height,confidence - sequence of words DenseCap: Fully Convolutional Localization Networks for Dense Captioning, Johnson et al. 2016

  33. Practical considerations when applying ConvNets

  34. What hardware do I use? Buy your own machine: - NVIDIA DIGITS DevBox (TITAN X GPUs) - NVIDIA DGX-1 (P100 GPUs) Build your own machine: https://graphific.github.io/posts/building-a-deep-learning-dream-machine/ GPUs in the cloud: - Amazon AWS (GRID K520 :( ) - Microsoft Azure (soon); 4x K80 GPUs - Cirrascale (“rent-a-box”)

  35. What framework do I use? Lasagne Caffe Torch Theano TensorFlow Keras Mxnet chainer Nervana’s Neon Microsoft’s CNTK Deeplearning4j ...

  36. What framework do I use? Lasagne Caffe Torch Theano TensorFlow Keras 1 Mxnet chainer 2,3 Nervana’s Neon Microsoft’s CNTK Deeplearning4j ...

  37. Q: How do I know what architecture to use?

  38. Q: How do I know what architecture to use? A: don’t be a hero. 1. Take whatever works best on ILSVRC (latest ResNet) 2. Download a pretrained model 3. Potentially add/delete some parts of it 4. Finetune it on your application.

  39. Q: How do I know what hyperparameters to use?

  40. Q: How do I know what hyperparameters to use? A: don’t be a hero. - Use whatever is reported to work best on ILSVRC. - Play with the regularization strength (dropout rates)

  41. ConvNets in practice: Distributed training VGG: ~ 2-3 weeks training with 4 GPUs ResNet 101: 2-3 weeks with 4 GPUs ~$1K each

Recommend

Lecture 3: Linear Classification Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej

Lecture 3: Linear Classification Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej

Lecture 3: Linear Classification Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 2 - Lecture 2 - 7 Jan 2015 7 Jan 2015 1 Last time: Image Classification assume given set of discrete labels {dog, cat, truck,

1.2k views • 47 slides
Lecture 11: CNNs in Practice Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li

Lecture 11: CNNs in Practice Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li

Lecture 11: CNNs in Practice Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 11 - Lecture 11 - 17 Feb 2016 17 Feb 2016 1 Administrative Midterms are graded! Pick

1.57k views • 112 slides
UDLS by Andrej Karpathy How to spot them Has detail on every level Is self-similar at

UDLS by Andrej Karpathy How to spot them Has detail on every level Is self-similar at

UDLS by Andrej Karpathy How to spot them Has detail on every level Is self-similar at different sizes test Not magic Julia Sets: How are they generated? Function iteration c= some_complex_number() for all x,y in window of

386 views • 21 slides
Lecture 8: Spatial Localization and Detection Fei-Fei Li & Andrej Karpathy & Justin

Lecture 8: Spatial Localization and Detection Fei-Fei Li & Andrej Karpathy & Justin

Lecture 8: Spatial Localization and Detection Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 8 - Lecture 8 - 1 Feb 2016 1 Feb 2016 1 Administrative - Project Proposals

1.17k views • 90 slides
Lecture 10: Recurrent Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 10: Recurrent Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 10: Recurrent Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 10 - Lecture 10 - 8 Feb 2016 8 Feb 2016 1 Administrative - Midterm this Wednesday!

1.22k views • 82 slides
Lecture 5: Training Neural Networks, Part I Fei-Fei Li & Andrej Karpathy & Justin

Lecture 5: Training Neural Networks, Part I Fei-Fei Li & Andrej Karpathy & Justin

Lecture 5: Training Neural Networks, Part I Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 5 - Lecture 5 - 20 Jan 2016 20 Jan 2016 1 Administrative A1 is due today

1.36k views • 102 slides
Lecture 4: Backpropagation and Neural Networks part 1 Fei-Fei Li & Andrej Karpathy &

Lecture 4: Backpropagation and Neural Networks part 1 Fei-Fei Li & Andrej Karpathy &

Lecture 4: Backpropagation and Neural Networks part 1 Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 4 - Lecture 4 - 13 Jan 2016 13 Jan 2016 1 Administrative A1 is due

1.26k views • 84 slides
Administrative - A2 has a number of corrections on Pizza. They are fixed in most recent .zip

Administrative - A2 has a number of corrections on Pizza. They are fixed in most recent .zip

Administrative - A2 has a number of corrections on Pizza. They are fixed in most recent .zip file. - Btw CNNs in Matlab: http://www.vlfeat. org/matconvnet/ Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 8 -

821 views • 66 slides
Lecture 13: Segmentation and Attention Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 13: Segmentation and Attention Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 13: Segmentation and Attention Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 13 - Lecture 13 - 24 Feb 2016 24 Feb 2016 1 Administrative Assignment 3 due

1.67k views • 133 slides
Attention, Transformers, BERT, and ViLBERT Arjun Majumdar Georgia Tech Slide Credits: Andrej

Attention, Transformers, BERT, and ViLBERT Arjun Majumdar Georgia Tech Slide Credits: Andrej

Attention, Transformers, BERT, and ViLBERT Arjun Majumdar Georgia Tech Slide Credits: Andrej Karpathy, Justin Johnson, Dhruv Batra Recall: Recurrent Neural Networks Image Credit: Andrej Karpathy Sequence-to-Sequence with RNNs Input : Sequence

2.62k views • 88 slides
Lecture 6: Training Neural Networks, Part 2 Fei-Fei Li & Andrej Karpathy & Justin

Lecture 6: Training Neural Networks, Part 2 Fei-Fei Li & Andrej Karpathy & Justin

Lecture 6: Training Neural Networks, Part 2 Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 6 - Lecture 6 - 25 Jan 2016 25 Jan 2016 1 Administrative A2 is out. Its

1.28k views • 86 slides
Object Discovery in 3D scenes via Shape Analysis Andrej Karpathy, Stephen Miller and Li Fei-Fei

Object Discovery in 3D scenes via Shape Analysis Andrej Karpathy, Stephen Miller and Li Fei-Fei

Object Discovery in 3D scenes via Shape Analysis Andrej Karpathy, Stephen Miller and Li Fei-Fei Abstract We present a method for discovering object mod- els from 3D meshes of indoor environments. Our algorithm first decomposes the scene into

342 views • 8 slides
Lecture 3: Loss functions and Optimization Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 3: Loss functions and Optimization Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 3: Loss functions and Optimization Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 3 - Lecture 3 - 11 Jan 2016 11 Jan 2016 1 Administrative A1 is due Jan 20

779 views • 76 slides
Lecture 7: Convolutional Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 7: Convolutional Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson

Lecture 7: Convolutional Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 - Lecture 7 - 27 Jan 2016 27 Jan 2016 1 Administrative A2 is due Feb 5 (next

1.29k views • 89 slides
Administrative - A2 is out. It was late 2 days so due date will be shifted by ~2 days. - we

Administrative - A2 is out. It was late 2 days so due date will be shifted by ~2 days. - we

Administrative - A2 is out. It was late 2 days so due date will be shifted by ~2 days. - we updated the project page with many pointers to datasets. Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 6 - Lecture 6 - 21

1.03k views • 47 slides
Lecture 1: Introduction 1 5-Jan-15 Lecture 1 - Fei-Fei Li & Andrej

Lecture 1: Introduction 1 5-Jan-15 Lecture 1 - Fei-Fei Li & Andrej

Lecture 1: Introduction 1 5-Jan-15 Lecture 1 - Fei-Fei Li & Andrej Karpathy Welcome to CS231n 2 5-Jan-15 Lecture 1 - Fei-Fei Li & Andrej Karpathy Biology Biology

877 views • 45 slides
Administrative - In-class midterm this Wednesday! (More on this in a bit) - Assignment #3: out

Administrative - In-class midterm this Wednesday! (More on this in a bit) - Assignment #3: out

Administrative - In-class midterm this Wednesday! (More on this in a bit) - Assignment #3: out Wed - Sample Midterm will be up in few hours Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 10 - Lecture 8 - 2 Feb 2015

1.04k views • 62 slides
Administrative - how is the assignment going? - btw, the notes get updated all the time based on

Administrative - how is the assignment going? - btw, the notes get updated all the time based on

Administrative - how is the assignment going? - btw, the notes get updated all the time based on your feedback - no lecture on Monday Fei-Fei Li & Andrej Karpathy Fei-Fei Li & Andrej Karpathy Lecture 4 - Lecture 4 - 7 Jan 2015 7

1.03k views • 66 slides
Lecture 9: Understanding and Visualizing Convolutional Neural Networks Fei-Fei Li & Andrej

Lecture 9: Understanding and Visualizing Convolutional Neural Networks Fei-Fei Li & Andrej

Lecture 9: Understanding and Visualizing Convolutional Neural Networks Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 9 - Lecture 9 - 3 Feb 2016 3 Feb 2016 1

1.04k views • 83 slides
Lecture 12: Software Packages Caffe / Torch / Theano / TensorFlow Fei-Fei Li & Andrej

Lecture 12: Software Packages Caffe / Torch / Theano / TensorFlow Fei-Fei Li & Andrej

Lecture 12: Software Packages Caffe / Torch / Theano / TensorFlow Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 12 - Lecture 12 - 22 Feb 2016 22 Feb 2016 1

2.46k views • 177 slides
Lecture 14 - Lecture 14 - 29 Feb 2016 29 Feb 2016 1 Administrative Everyone should be

Lecture 14 - Lecture 14 - 29 Feb 2016 29 Feb 2016 1 Administrative Everyone should be

Fei-Fei Li & Andrej Karpathy & Justin Johnson Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 14 - Lecture 14 - 29 Feb 2016 29 Feb 2016 1 Administrative Everyone should be done with Assignment 3 now Milestone

1.32k views • 130 slides
Connecting Images with Natural Language Andrej Karpathy CVPR 2016. Deep Vision workshop. July 1,

Connecting Images with Natural Language Andrej Karpathy CVPR 2016. Deep Vision workshop. July 1,

Connecting Images with Natural Language Andrej Karpathy CVPR 2016. Deep Vision workshop. July 1, 2016 Visual domain Domain of Natural Language [Cho et al. 2014] Connecting Images and Natural Language A man sitting on A person sitting on a

1.67k views • 119 slides
Machine Learning for NLP Sequential NN models Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Sequential NN models Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Sequential NN models Aurlie Herbelot 2019 Centre for Mind/Brain Sciences University of Trento 1 The unreasonable effectiveness... 2 Karpathy (2015) In 2015, Andrej Karpathy wrote a blog entry which became

857 views • 57 slides
Understanding How ConvNets See Springerberg et al, Striving for Simplicity: The All Convolutional

Understanding How ConvNets See Springerberg et al, Striving for Simplicity: The All Convolutional

Understanding How ConvNets See Springerberg et al, Striving for Simplicity: The All Convolutional Net (ICLR 2015 workshops) CSC321: Intro to Machine Learning and Neural Networks, Winter 2016 Slides from Andrej Karpathy Michael Guerzhoy What

248 views • 21 slides

More recommend