Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks Convolutional neural networks One of the major kinds of ANNs in use UMaine COS 470/570 – Introduction to AI One of the reasons deep learning is so successful: addresses computational tractability addresses vanishing/exploding gradient problem Spring 2019 Start – 80s First truly successful modern version: LeNet (LeCun, 1989) LeNet-5 (LeCun, 1998): 7-layer CNN for reading numbers on checks Created: 2019-04-29 Mon 21:45 1 2 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 1 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 2 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Human visual system The problem Goal: High-accuracy image recognition Standard supervised learning with deep (fully-connected) networks: Images require connections from each pixel → each neuron E.g., 1028 × 768 image ⇒ about 789,504 weights per neuron Slow to train Vanishing/exploding gradient problem Also no spatial locality exploited Can we take inspiration from biological vision systems? Image credit: user Clock, CC BY-SA 3.0 , via Wikimedia Commons 3 . 1 4 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 3 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 4 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional layers Shared weights Instead of fully-connected layer, think of using a layer whose neurons each have a receptive fi eld : So—how to compute the layer? For a receptive fi eld : n × n weights n × n If input layer is , hidden layer is m × m m − n + 1 × m − n + 1 For hidden layer neuron at , activation is: x , y n − 1 n − 1 ∑ ∑ σ ( b + w i , j a x + i , y + j ) i =0 j =0 Overlapping receptive fi elds Slide the kernel across, down the image by some stride Neurons then learn local features, only have a few weight each weights = kernel or fi lter b Have multiple feature-detecting layers per convolutional layer Hidden layer = feature map Problem: Update weights based on entire hidden layer’s computed loss Features should be location-independent function ⇒ Weights for nodes should be shared, learned together 5 . 1 6 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 5 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 6 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Why convolutional layer? Pooling layers Learns local spatial features of input Convolutional layers are coupled with pooling layers Location-independent (location-invariant) Each node of pooling layer connected to some region of i × j Example (from Nielsen, M: Neural Networks and Deep Learning ): feature map Typically feature map/layer ⇒ learn di ff erent kinds of features > 1 7 . 1 8 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 7 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 8 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Pooling layers Using the features Pool based on some function—max, average, etc. Pooled layers’ output ⇒ fully-connected layer – e.g., for MNIST: (Aphex34 [ CC BY-SA 4.0 ], via Wikimedia Commons) Purpose(s): (From Nielson)) Reduce # weights needed Learn con fi guration of features Blur/average/smooth feature map Could have multiple fully-connected layers, too Determining if a feature is in a particular region 9 . 1 10 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 9 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 10 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Multiple convolutional layers Multiple convolutional + pooling layers: Learning in CNNs Backpropagation learning, gradient descent Equations for fully-connected nets have to be modi fi ed, though Theano, TensorFlow, PyTorch – all have support for training CNNs (Aphex34 [ CC BY-SA 4.0 ], via Wikimedia Commons) Deeper layers ⇒ more complex features 11 . 1 12 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 11 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 12 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Feature detection in CNNs Multiple convolutional layers LeNet-5: From ConvNet: 7 layers Recognize numbers on checks Recall the DQN we talked about used CNNs Many additional variants of CNNs now ResNet: 152 layers, general image recognition, lots of additions to LeNet’s basic architecture 13 . 1 14 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 13 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 14 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Progress in image recognition competition 15 . 1 (Aphex34 [ CC BY-SA 4.0 ], via Wikimedia Commons) file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 15 of 17 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 16 of 17
Convolutional Neural Networks<br/><br/> 5/4/19, 4(03 PM Your turn 1. Build a CNN Get into groups, one of whom has a laptop with Keras on it Create a simple CNN for MNIST 2. Explain a CNN Get into groups with at least 2 laptops Part of group: Look up an “inception” layer in (e.g.) GoogleNet Other part: Look up ResNet Explain them to each other after a few minutes 16 . 1 file:///Users/rmt/Classes/COS470/2019-Spring/Slides/CNET19/cnet19.html?print-pdf Page 17 of 17
Recommend
More recommend
