bbm406
play

BBM406 Fundamentals of Machine Learning Lecture 14: Deep - PowerPoint PPT Presentation

Dec 18, 2022 •161 likes •1.38k views

Illustration:detail from the visualization of ResNet-50 conv2 // Graphcore BBM406 Fundamentals of Machine Learning Lecture 14: Deep Convolutional Networks Aykut Erdem // Hacettepe University // Fall 2019 Announcement Midterm exam on Nov

  1. 51 Example: CONV layer in Caffe slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  2. 52 Example: CONV layer in Lasagne slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  3. The brain/neuron view of CONV Layer 32x32x3 image 5x5x3 filter 32 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 1 number: 32 the result of taking a dot product between the filter and this part of the image 3 (i.e. 5*5*3 = 75-dimensional dot product) 53

  4. The brain/neuron view of CONV Layer 32x32x3 image 5x5x3 filter 32 It’s just a neuron with local connectivity... slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 1 number: 32 the result of taking a dot product between the filter and this part of the image 3 (i.e. 5*5*3 = 75-dimensional dot product) 54

  5. The brain/neuron view of CONV Layer 32 28 An activation map is a 28x28 sheet of neuron outputs: 1. Each is connected to a small region in the input 2. All of them share parameters slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 32 28 “5x5 filter” -> “5x5 receptive field for each neuron” 3 55

  6. The brain/neuron view of CONV Layer 32 E.g. with 5 filters, 28 CONV layer consists of neurons arranged in a 3D grid (28x28x5) There will be 5 different slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 32 28 neurons all looking at the same region in the input volume 3 5 56

  7. 57 Activation Functions slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  8. Activation Functions Sigmoid tanh tanh(x) slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson ReLU max(0,x) 58

  9. Activation Functions - Squashes numbers to range [0,1] - Historically popular since they have nice interpretation as a saturating “firing rate” of a neuron 3 problems: 1. Saturated neurons “kill” the slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Sigmoid gradients 2. Sigmoid outputs are not zero- centered 3. exp() is a bit compute expensive 59

  10. Activation Functions - Squashes numbers to range [-1,1] - zero centered (nice) - still kills gradients when saturated :( tanh(x) slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson [LeCun et al., 1991] 60

  11. Computes f(x) = max(0,x) Activation Functions - - Does not saturate (in +region) - Very computationally efficient - Converges much faster than sigmoid/tanh in practice 
 (e.g. 6x) slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson ReLU (Rectified Linear Unit) [Krizhevsky et al., 2012] 61

  12. 62 62 two more layers to go: POOL/FC slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  13. Pooling layer - makes the representations smaller and more manageable - operates over each activation map independently: slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 63

  14. Max Pooling Single depth slice 1 1 2 4 1 1 2 4 x max pool with 2x2 filters 6 8 5 6 7 8 6 8 5 6 7 8 and stride 2 3 4 3 4 3 2 1 0 3 2 1 0 1 2 3 4 1 2 3 4 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson y 64

  15. 65 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  16. 66 Common settings: F = 2, S = 2 F = 3, S = 2 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  17. Fully Connected Layer (FC layer) - Contains neurons that connect to the entire input volume, as in ordinary Neural Networks slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 67 67

  18. [ConvNetJS demo: training on CIFAR-10] http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 68

  19. Case studies 69

  20. Case Study: LeNet-5 [LeCun et al., 1998] slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Conv filters were 5x5, applied at stride 1 Subsampling (Pooling) layers were 2x2 applied at stride 2 i.e. architecture is [CONV-POOL-CONV-POOL-CONV-FC] 70

  21. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images First layer (CONV1): 96 11x11 filters applied at stride 4 => Q: what is the output volume size? Hint: (227-11)/4+1 = 55 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 71

  22. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images First layer (CONV1): 96 11x11 filters applied at stride 4 => Output volume [55x55x96] slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Q: What is the total number of parameters in this layer? 72

  23. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images First layer (CONV1): 96 11x11 filters applied at stride 4 => Output volume [55x55x96] slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Parameters: (11*11*3)*96 = 35K 73

  24. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images After CONV1: 55x55x96 Second layer (POOL1): 3x3 filters applied at stride 2 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Q: what is the output volume size? Hint: (55-3)/2+1 = 27 74

  25. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images After CONV1: 55x55x96 Second layer (POOL1): 3x3 filters applied at stride 2 Output volume: 27x27x96 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Q: what is the number of parameters in this layer? 75

  26. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images After CONV1: 55x55x96 Second layer (POOL1): 3x3 filters applied at stride 2 Output volume: 27x27x96 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Parameters: 0! 76

  27. Case Study: AlexNet [Krizhevsky et al. 2012] Input: 227x227x3 images After CONV1: 55x55x96 After POOL1: 27x27x96 ... slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 77

  28. Case Study: AlexNet [Krizhevsky et al. 2012] Full (simplified) AlexNet architecture: [227x227x3] INPUT [55x55x96] CONV1: 96 11x11 filters at stride 4, pad 0 [27x27x96] MAX POOL1: 3x3 filters at stride 2 [27x27x96] NORM1: Normalization layer [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 [13x13x256] MAX POOL2: 3x3 filters at stride 2 [13x13x256] NORM2: Normalization layer slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 [6x6x256] MAX POOL3: 3x3 filters at stride 2 [4096] FC6: 4096 neurons [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons (class scores) 78

  29. Case Study: AlexNet [Krizhevsky et al. 2012] Full (simplified) AlexNet architecture: Details/Retrospectives: [227x227x3] INPUT - first use of ReLU [55x55x96] CONV1: 96 11x11 filters at stride 4, pad 0 - used Norm layers (not common [27x27x96] MAX POOL1: 3x3 filters at stride 2 anymore) [27x27x96] NORM1: Normalization layer - heavy data augmentation [27x27x256] CONV2: 256 5x5 filters at stride 1, pad 2 - dropout 0.5 [13x13x256] MAX POOL2: 3x3 filters at stride 2 - batch size 128 [13x13x256] NORM2: Normalization layer - SGD Momentum 0.9 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson [13x13x384] CONV3: 384 3x3 filters at stride 1, pad 1 - Learning rate 1e-2, reduced by 10 [13x13x384] CONV4: 384 3x3 filters at stride 1, pad 1 manually when val accuracy plateaus [13x13x256] CONV5: 256 3x3 filters at stride 1, pad 1 - L2 weight decay 5e-4 [6x6x256] MAX POOL3: 3x3 filters at stride 2 - 7 CNN ensemble: 18.2% -> 15.4% [4096] FC6: 4096 neurons [4096] FC7: 4096 neurons [1000] FC8: 1000 neurons (class scores) 79

  30. Case Study: ZFNet [Zeiler and Fergus, 2013] AlexNet but: slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson CONV1: change from (11x11 stride 4) to (7x7 stride 2) CONV3,4,5: instead of 384, 384, 256 filters use 512, 1024, 512 ImageNet top 5 error: 15.4% -> 14.8% 80

  31. Case Study: VGGNet [Simonyan and Zisserman, 2014] Only 3x3 CONV stride 1, pad 1 and 2x2 MAX POOL stride 2 best model slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 11.2% top 5 error in ILSVRC 2013 -> 7.3% top 5 error 81

  32. (not counting biases) INPUT: [224x224x3] memory: 224*224*3=150K params: 0 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*3)*64 = 1,728 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*64)*64 = 36,864 POOL2: [112x112x64] memory: 112*112*64=800K params: 0 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*64)*128 = 73,728 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*128)*128 = 147,456 POOL2: [56x56x128] memory: 56*56*128=400K params: 0 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*128)*256 = 294,912 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 POOL2: [28x28x256] memory: 28*28*256=200K params: 0 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*256)*512 = 1,179,648 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 POOL2: [14x14x512] memory: 14*14*512=100K params: 0 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson POOL2: [7x7x512] memory: 7*7*512=25K params: 0 FC: [1x1x4096] memory: 4096 params: 7*7*512*4096 = 102,760,448 FC: [1x1x4096] memory: 4096 params: 4096*4096 = 16,777,216 FC: [1x1x1000] memory: 1000 params: 4096*1000 = 4,096,000 82

  33. (not counting biases) INPUT: [224x224x3] memory: 224*224*3=150K params: 0 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*3)*64 = 1,728 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*64)*64 = 36,864 POOL2: [112x112x64] memory: 112*112*64=800K params: 0 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*64)*128 = 73,728 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*128)*128 = 147,456 POOL2: [56x56x128] memory: 56*56*128=400K params: 0 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*128)*256 = 294,912 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 POOL2: [28x28x256] memory: 28*28*256=200K params: 0 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*256)*512 = 1,179,648 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 POOL2: [14x14x512] memory: 14*14*512=100K params: 0 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson POOL2: [7x7x512] memory: 7*7*512=25K params: 0 FC: [1x1x4096] memory: 4096 params: 7*7*512*4096 = 102,760,448 FC: [1x1x4096] memory: 4096 params: 4096*4096 = 16,777,216 FC: [1x1x1000] memory: 1000 params: 4096*1000 = 4,096,000 TOTAL memory: 24M * 4 bytes ~= 93MB / image (only forward! ~*2 for bwd) TOTAL params: 138M parameters 83

  34. 
 
 
 (not counting biases) Note: INPUT: [224x224x3] memory: 224*224*3=150K params: 0 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*3)*64 = 1,728 CONV3-64: [224x224x64] memory: 224*224*64=3.2M params: (3*3*64)*64 = 36,864 Most memory is in POOL2: [112x112x64] memory: 112*112*64=800K params: 0 early CONV CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*64)*128 = 73,728 CONV3-128: [112x112x128] memory: 112*112*128=1.6M params: (3*3*128)*128 = 147,456 POOL2: [56x56x128] memory: 56*56*128=400K params: 0 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*128)*256 = 294,912 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 CONV3-256: [56x56x256] memory: 56*56*256=800K params: (3*3*256)*256 = 589,824 POOL2: [28x28x256] memory: 28*28*256=200K params: 0 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*256)*512 = 1,179,648 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 CONV3-512: [28x28x512] memory: 28*28*512=400K params: (3*3*512)*512 = 2,359,296 POOL2: [14x14x512] memory: 14*14*512=100K params: 0 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 Most params are CONV3-512: [14x14x512] memory: 14*14*512=100K params: (3*3*512)*512 = 2,359,296 slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson POOL2: [7x7x512] memory: 7*7*512=25K params: 0 in late FC FC: [1x1x4096] memory: 4096 params: 7*7*512*4096 = 102,760,448 FC: [1x1x4096] memory: 4096 params: 4096*4096 = 16,777,216 FC: [1x1x1000] memory: 1000 params: 4096*1000 = 4,096,000 TOTAL memory: 24M * 4 bytes ~= 93MB / image (only forward! ~*2 for bwd) TOTAL params: 138M parameters 84

  35. Case Study: GoogLeNet [Szegedy et al., 2014] Inception module slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson ILSVRC 2014 winner (6.7% top 5 error) 85

  36. Case Study: ResNet [He et al., 2015] ILSVRC 2015 winner (3.6% top 5 error) slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson Slide from Kaiming He’s recent presentation https://www.youtube.com/ watch?v=1PGLj-uKT1w 86

  37. Case Study: ResNet [He et al., 2015] ILSVRC 2015 winner (3.6% top 5 error) 2-3 weeks of training on 8 GPU machine at runtime: faster than a VGGNet! (even though it has 8x more layers) slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson (slide from Kaiming He’s recent presentation) 87

  38. Case Study: ResNet [He et al., 2015] 224x224x3 spatial dimension only 56x56! slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson 88

  39. 89 Case Study Bonus: DeepMind’s 
 AlphaGo slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson

  40. slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson policy network: [19x19x48] Input CONV1: 192 5x5 filters , stride 1, pad 2 => [19x19x192] CONV2..12: 192 3x3 filters, stride 1, pad 1 => [19x19x192] CONV: 1 1x1 filter, stride 1, pad 0 => [19x19] (probability map of promising moves) 90

  41. Summary - ConvNets stack CONV,POOL,FC layers - Trend towards smaller filters and deeper architectures - Trend towards getting rid of POOL/FC layers (just CONV) - Typical architectures look like [(CONV-RELU)*N-POOL?]*M-(FC-RELU)*K,SOFTMAX slide by Fei-Fei Li, Andrej Karpathy & Justin Johnson where N is usually up to ~5, M is large, 0 <= K <= 2. - but recent advances such as ResNet/GoogLeNet challenge this paradigm 91

  42. Understanding ConvNets 92

  43. Input Input Input Image 96 256 354 Image Image filters filters filters 7x7x3 Convolution 5x5x96 Convolution 3x3x256 Convolution RGB Input Image 3x3 Max Pooling 3x3 Max Pooling 13 x 13 x 354 224 x 224 x 3 Down Sample 4x Down Sample 4x 55 x 55 x 96 13 x 13 x 256 256 354 filters filters 3x3x354 Convolution 3x3x354 Convolution Standard Logistic Standard 3x3 Max Pooling 13 x 13 x 354 4096 Units Regression 4096 Units Down Sample 2x ≈ 1000 Classes 6 x 6 x 256 slide by Yisong Yue http://www.image-net.org/ http://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf

  44. Visualizing CNN (Layer 1) slide by Yisong Yue http://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf 94

  45. Visualizing CNN (Layer 2) Top Image Patches Part that Triggered Filter slide by Yisong Yue http://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf 95

  46. Visualizing CNN (Layer 3) Top Image Patches Part that Triggered Filter slide by Yisong Yue http://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf 96

  47. Visualizing CNN (Layer 4) Top Image Patches Part that Triggered Filter slide by Yisong Yue http://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf 97

  48. Visualizing CNN (Layer 5) Top Image Patches Part that Triggered Filter slide by Yisong Yue http://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf http://cs.nyu.edu/~fergus/presentations/nips2013_final.pdf 98

  49. 99

  50. Tips and Tricks 100

Recommend

BBM406 Fundamentals of Machine Learning Lecture 12: Computational Graph Backpropagation

BBM406 Fundamentals of Machine Learning Lecture 12: Computational Graph Backpropagation

Illustration: 3Blue1Brown BBM406 Fundamentals of Machine Learning Lecture 12: Computational Graph Backpropagation Aykut Erdem // Hacettepe University // Fall 2019 Last time Multilayer Perceptron Layer Representation y W 4 y i =

1.13k views • 69 slides
BBM406 Fundamentals of Machine Learning Lecture 9: Logistic Regression Discriminative vs.

BBM406 Fundamentals of Machine Learning Lecture 9: Logistic Regression Discriminative vs.

Illustration: Theodore Modis BBM406 Fundamentals of Machine Learning Lecture 9: Logistic Regression Discriminative vs. Generative Classification Aykut Erdem // Hacettepe University // Fall 2019 Last time Nave Bayes Classifier Given

637 views • 37 slides
BBM406 Fundamentals of Machine Learning Lecture 4: Linear Regression, Optimization,

BBM406 Fundamentals of Machine Learning Lecture 4: Linear Regression, Optimization,

illustration: detail from xkcd strip #2048 BBM406 Fundamentals of Machine Learning Lecture 4: Linear Regression, Optimization, Generalization, Model complexity, Regularization Aykut Erdem // Hacettepe University // Fall 2019 1 ,

1.09k views • 56 slides
BBM406 Fundamentals of Machine Learning Lecture 19: What is Ensemble Learning? Bagging

BBM406 Fundamentals of Machine Learning Lecture 19: What is Ensemble Learning? Bagging

Photo byUnsplash user @nathananderson BBM406 Fundamentals of Machine Learning Lecture 19: What is Ensemble Learning? Bagging Random Forests Aykut Erdem // Hacettepe University // Fall 2019 Last time Decision Trees slide by David Sontag

681 views • 38 slides
BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe

BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe

Illustration: detail from The Alchemist Discovering Phosphorus by Joseph Wright (1771) BBM406 Fundamentals of Machine Learning Lecture 5: ML Methodology Aykut Erdem // Hacettepe University // Fall 2019 About class projects This semester

1.42k views • 46 slides
BBM406 Fundamentals of Machine Learning Lecture 23: Dimensionality Reduction Aykut Erdem //

BBM406 Fundamentals of Machine Learning Lecture 23: Dimensionality Reduction Aykut Erdem //

Image credit: Matthew Turk and Alex Pentland BBM406 Fundamentals of Machine Learning Lecture 23: Dimensionality Reduction Aykut Erdem // Hacettepe University // Fall 2019 Administrative Project Presentations January 8,10, 2020 Each

1.6k views • 87 slides
BBM406 Fundamentals of Machine Learning Lecture 21: Clustering K-Means Aykut Erdem //

BBM406 Fundamentals of Machine Learning Lecture 21: Clustering K-Means Aykut Erdem //

Photo by Unsplash user @foodiesfeed BBM406 Fundamentals of Machine Learning Lecture 21: Clustering K-Means Aykut Erdem // Hacettepe University // Fall 2019 Last time Boosting Idea: given a weak learner, run it multiple times on

1.76k views • 106 slides
BBM406 Fundamentals of Machine Learning Lecture 7: Probability Review (contd.) Maximum

BBM406 Fundamentals of Machine Learning Lecture 7: Probability Review (contd.) Maximum

photo: Chessex Borealis Aquerple Polyhedral BBM406 Fundamentals of Machine Learning Lecture 7: Probability Review (contd.) Maximum Likelihood Estimation (MLE) Aykut Erdem // Hacettepe University // Fall 2019 Administrative Project

926 views • 53 slides
BBM406 Fundamentals of Machine Learning Lecture 8: Maximum a Posteriori (MAP) Nave Bayes

BBM406 Fundamentals of Machine Learning Lecture 8: Maximum a Posteriori (MAP) Nave Bayes

photo from Twilight Zone Episode The Nick of Time BBM406 Fundamentals of Machine Learning Lecture 8: Maximum a Posteriori (MAP) Nave Bayes Classifier Aykut Erdem // Hacettepe University // Fall 2019 Recap: MLE Maximum Likelihood

897 views • 63 slides
BBM406 Fundamentals of Machine Learning Lecture 10: Linear Discriminant Functions Perceptron

BBM406 Fundamentals of Machine Learning Lecture 10: Linear Discriminant Functions Perceptron

Illustration: Frank Rosenblatt's Perceptron BBM406 Fundamentals of Machine Learning Lecture 10: Linear Discriminant Functions Perceptron Aykut Erdem // Hacettepe University // Fall 2019 Assignment 2 is out! It is due November 22 (i.e.

1.57k views • 49 slides
BBM406 Fundamentals of Machine Learning Lecture 20: AdaBoost Aykut Erdem // Hacettepe

BBM406 Fundamentals of Machine Learning Lecture 20: AdaBoost Aykut Erdem // Hacettepe

Illustration adapted from Alex Rogozhnikov BBM406 Fundamentals of Machine Learning Lecture 20: AdaBoost Aykut Erdem // Hacettepe University // Fall 2019 Last time Bias/Variance Tradeo ff slide by David Sontag Graphical illustration of

725 views • 60 slides
BBM406 Fundamentals of Machine Learning Lecture 3: Kernel Regression, Distance Metrics, Curse

BBM406 Fundamentals of Machine Learning Lecture 3: Kernel Regression, Distance Metrics, Curse

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 3: Kernel Regression, Distance Metrics, Curse of Dimensionality Aykut Erdem // Hacettepe University // Fall 2019 Administrative Assignment 1 will be out

1.25k views • 32 slides
BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest

photo:@rewardyfahmi // Unsplash BBM406 Fundamentals of Machine Learning Lecture 2: Machine Learning by Examples, Nearest Neighbor Classifier Aykut Erdem // Hacettepe University // Fall 2019 When Do We Use Machine Learning? ML is

1.33k views • 110 slides
BBM406 Fundamentals of Machine Learning Lecture 13: Introduction to Deep Learning Aykut

BBM406 Fundamentals of Machine Learning Lecture 13: Introduction to Deep Learning Aykut

Illustration: Illustration: Benedetto Cristofani BBM406 Fundamentals of Machine Learning Lecture 13: Introduction to Deep Learning Aykut Erdem // Hacettepe University // Fall 2019 A reminder about course projects From now on, regular

899 views • 51 slides
BBM406 Fundamentals of Machine Learning Lecture 11: Multi-layer Perceptron Forward Pass

BBM406 Fundamentals of Machine Learning Lecture 11: Multi-layer Perceptron Forward Pass

Image: Jose-Luis Olivares BBM406 Fundamentals of Machine Learning Lecture 11: Multi-layer Perceptron Forward Pass Aykut Erdem // Hacettepe University // Fall 2019 Last time Linear Discriminant Function Linear

747 views • 46 slides
BBM406 Fundamentals of Machine Learning Lecture 1: Course outline and logistics An overview

BBM406 Fundamentals of Machine Learning Lecture 1: Course outline and logistics An overview

Illustration: Tom Gauld BBM406 Fundamentals of Machine Learning Lecture 1: Course outline and logistics An overview of Machine Learning Aykut Erdem // Hacettepe University // Fall 2019 Todays Schedule Course outline and logistics

1.05k views • 79 slides
BBM406 Fundamentals of Machine Learning Lecture 17: Kernel Trick for SVMs Risk and Loss

BBM406 Fundamentals of Machine Learning Lecture 17: Kernel Trick for SVMs Risk and Loss

Photo by Arthur Gretton, CMU Machine Learning Protestors at G20 BBM406 Fundamentals of Machine Learning Lecture 17: Kernel Trick for SVMs Risk and Loss Support Vector Regression Aykut Erdem // Hacettepe University // Fall 2019

1.58k views • 81 slides
BBM406 Fundamentals of Machine Learning Lecture 18: Decision Trees Aykut Erdem // Hacettepe

BBM406 Fundamentals of Machine Learning Lecture 18: Decision Trees Aykut Erdem // Hacettepe

Photo byUnsplash user @technobulka BBM406 Fundamentals of Machine Learning Lecture 18: Decision Trees Aykut Erdem // Hacettepe University // Fall 2019 Today Decision Trees Tree construction Overfitting Pruning

632 views • 48 slides
BBM406 Fundamentals of Machine Learning Lecture 15: Support Vector Machines Aykut Erdem //

BBM406 Fundamentals of Machine Learning Lecture 15: Support Vector Machines Aykut Erdem //

Photo by Arthur Gretton, CMU Machine Learning Protestors at G20 BBM406 Fundamentals of Machine Learning Lecture 15: Support Vector Machines Aykut Erdem // Hacettepe University // Fall 2019 Announcement Midterm exam on Nov 29 Dec 6, 2019

695 views • 35 slides
BBM406 Fundamentals of Machine Learning Lecture 6: Learning theory Probability Review Aykut

BBM406 Fundamentals of Machine Learning Lecture 6: Learning theory Probability Review Aykut

BBM406 Fundamentals of Machine Learning Lecture 6: Learning theory Probability Review Aykut Erdem // Hacettepe University // Fall 2019 Last time Regularization , Cross-Validation error Validation error the data Training error number

1.01k views • 48 slides

More recommend