introduction to deep learning
play

Introduction to Deep Learning A. G. Schwing & S. Fidler - PowerPoint PPT Presentation

Dec 29, 2023 •217 likes •965 views

Introduction to Deep Learning A. G. Schwing & S. Fidler University of Toronto, 2015 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 1 / 39 Outline Universality of Neural Networks 1 Learning Neural

  1. Introduction to Deep Learning A. G. Schwing & S. Fidler University of Toronto, 2015 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 1 / 39

  2. Outline Universality of Neural Networks 1 Learning Neural Networks 2 Deep Learning 3 Applications 4 References 5 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 2 / 39

  3. What are neural networks? Let’s ask • Biological Input Hidden Output layer layer layer Input #1 Input #2 Output Input #3 • Computational Input #4 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 3 / 39

  4. What are neural networks? ... Neural networks (NNs) are computational models inspired by biological neural networks [...] and are used to estimate or approximate functions ... [Wikipedia] A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 4 / 39

  5. What are neural networks? Origins: Traced back to threshold logic [W. McCulloch and W. Pitts, 1943] Perceptron [F . Rosenblatt, 1958] A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 5 / 39

  6. What are neural networks? Use cases Classification Playing video games Captcha Neural Turing Machine (e.g., learn how to sort) Alex Graves http://www.technologyreview.com/view/532156/googles-secretive-deepmind-startup-unveils-a-neural-turing-machine/ A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 6 / 39

  7. What are neural networks? Example: input x parameters w 1 , w 2 , b A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 7 / 39

  8. What are neural networks? Example: input x parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 7 / 39

  9. How to compute the function? Forward propagation/pass, inference, prediction: Given input x and parameters w , b Compute (latent variables/) intermediate results in a feed-forward manner Until we obtain output function f x ∈ R w 1 w 2 h 1 f b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 8 / 39

  10. How to compute the function? Forward propagation/pass, inference, prediction: Given input x and parameters w , b Compute (latent variables/) intermediate results in a feed-forward manner Until we obtain output function f A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 8 / 39

  11. How to compute the function? Example: input x , parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 9 / 39

  12. How to compute the function? Example: input x , parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f = σ ( w 1 · x + b ) h 1 f = w 2 · h 1 b ∈ R Sigmoid function: σ ( z ) = 1 / ( 1 + exp ( − z )) A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 9 / 39

  13. How to compute the function? Example: input x , parameters w 1 , w 2 , b x ∈ R w 1 w 2 h 1 f = σ ( w 1 · x + b ) h 1 f = w 2 · h 1 b ∈ R Sigmoid function: σ ( z ) = 1 / ( 1 + exp ( − z )) x = ln 2, b = ln 3, w 1 = 2, w 2 = 2 h 1 =? f =? A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 9 / 39

  14. How to compute the function? Given parameters, what is f for x = 0, x = 1, x = 2, ... f = w 2 σ ( w 1 · x + b ) A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 10 / 39

  15. How to compute the function? Given parameters, what is f for x = 0, x = 1, x = 2, ... f = w 2 σ ( w 1 · x + b ) 2 1.5 1 f 0.5 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 10 / 39

  16. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  17. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  18. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes 1 0.8 0.6 f 0.4 b = −2 0.2 b = 0 b = 2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  19. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes b = 0 , w 1 changes 1 0.8 0.6 f 0.4 b = −2 0.2 b = 0 b = 2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  20. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes b = 0 , w 1 changes 1 1 0.8 0.8 0.6 0.6 w 1 = 0 f f 0.4 0.4 w 1 = 0.5 b = −2 w 1 = 1.0 0.2 0.2 b = 0 w 1 = 100 b = 2 0 0 −5 0 5 −5 0 5 x x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  21. Let’s mess with parameters: x ∈ R w 1 w 2 h 1 f h 1 = σ ( w 1 · x + b ) f = w 2 · h 1 σ ( z ) = 1 / ( 1 + exp ( − z )) b ∈ R w 1 = 1 . 0 , b changes b = 0 , w 1 changes 1 1 0.8 0.8 0.6 0.6 w 1 = 0 f f 0.4 0.4 w 1 = 0.5 b = −2 w 1 = 1.0 0.2 0.2 b = 0 w 1 = 100 b = 2 0 0 −5 0 5 −5 0 5 x x Keep in mind the step function. A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 11 / 39

  22. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 y 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  23. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  24. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  25. How to use Neural Networks for binary classification? Feature/Measurement: x Output: How likely is the input to be a cat? 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 12 / 39

  26. How to use Neural Networks for binary classification? Shifted feature/measurement: x Output: How likely is the input to be a cat? Previous features 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 13 / 39

  27. How to use Neural Networks for binary classification? Shifted feature/measurement: x Output: How likely is the input to be a cat? Previous features Shifted features 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 13 / 39

  28. How to use Neural Networks for binary classification? Shifted feature/measurement: x Output: How likely is the input to be a cat? Previous features Shifted features 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x Learning/Training means finding the right parameters. A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 13 / 39

  29. So far we are able to scale and translate sigmoids. How well can we approximate an arbitrary function? With the simple model we are obviously not going very far. Features are good Simple classifier 1 0.8 0.6 f 0.4 0.2 0 −5 0 5 x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 14 / 39

  30. So far we are able to scale and translate sigmoids. How well can we approximate an arbitrary function? With the simple model we are obviously not going very far. Features are good Features are noisy Simple classifier More complex classifier 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 14 / 39

  31. So far we are able to scale and translate sigmoids. How well can we approximate an arbitrary function? With the simple model we are obviously not going very far. Features are good Features are noisy Simple classifier More complex classifier 1 1 0.8 0.8 0.6 0.6 f f 0.4 0.4 0.2 0.2 0 0 −5 0 5 −5 0 5 x x How can we generalize? A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 14 / 39

  32. Let’s use more hidden variables: b 1 h 1 = σ ( w 1 · x + b 1 ) h 1 h 2 = σ ( w 3 · x + b 2 ) w 1 w 2 x ∈ R f = w 2 · h 1 + w 4 · h 2 f w 3 w 4 h 2 b 2 A. G. Schwing & S. Fidler (UofT) CSC420: Intro to Image Understanding 2015 15 / 39

Recommend

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention

Introduction to Deep Learning Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples RNNs Some slides borrowed from Fei-Fei Li & Justin Johnson &

1.1k views • 55 slides
Presentation about Deep Learning --- Zhongwu xie Contents 1.Brief introduction of Deep learning.

Presentation about Deep Learning --- Zhongwu xie Contents 1.Brief introduction of Deep learning.

Presentation about Deep Learning --- Zhongwu xie Contents 1.Brief introduction of Deep learning. 2.Brief introduction of Backpropagation. 3.Brief introduction of Convolutional Neural Networks. Deep learning I . Introduction to Deep Learning

609 views • 22 slides
Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical

Deep learning for NLP: Introduction CS 6956: Deep Learning for NLP Words are a very fantastical banquet, just so many strange dishes And yet, we seem to do fine We can understand and generate language effortlessly. Almost 1 Wouldnt it be

1.62k views • 52 slides
Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang What is deep

Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang What is deep

Introduction to Deep Learning Princeton University COS 495 Instructor: Yingyu Liang What is deep learning? Short answer: recent buzz word Industry Google Facebook Microsoft Musk Toyota Drug Finance

282 views • 26 slides
Tsing nghua hua University versity Introduction Deep learning has widely used in lots of

Tsing nghua hua University versity Introduction Deep learning has widely used in lots of

Deep ep500 500 BOF 2018 Jidong ong Zhai Tsing nghua hua University versity Introduction Deep learning has widely used in lots of areas Introduction A lot of deep learning frameworks, compute libraries and acceleration devices

371 views • 15 slides
Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning

Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning

Introduction to Deep Learning Georgia Tech CS 4650/7650 Fall 2020 Outline Deep Learning CNN RNN Attention Transformer Pytorch Introduction Basics Examples CNNs Some slides borrowed from

575 views • 56 slides
BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem Hacettepe University Computer Vision Lab (HUCVL) What is deep learning? Y. LeCun, Y. Bengio, G. Hinton, "Deep Learning" , Nature,

1.12k views • 84 slides
Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief

Deep Learning on Massively Parallel Processing Databases Frank McQuillan Feb 2019 2 A Brief Introduction to Deep Learning Artificial Intelligence Landscape Deep Learning 4 Example Deep Learning Algorithms Multilayer Recurrent

651 views • 44 slides
Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Deep Learning (CNNs) Deep Learning Readings: Matt Gormley Murphy 28 Bishop

985 views • 86 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides
ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image

308 views • 26 slides
CS535: Deep Learning 1. Introduction Winter 2018 Fuxin Li With materials from Pierre Baldi,

CS535: Deep Learning 1. Introduction Winter 2018 Fuxin Li With materials from Pierre Baldi,

CS535: Deep Learning 1. Introduction Winter 2018 Fuxin Li With materials from Pierre Baldi, Geoffrey Hinton, Andrew Ng, Honglak Lee, Aditya Khosla, Joseph Lim 1 Cutting Edge of Machine Learning: Deep Learning in Neural Networks Engineering

732 views • 48 slides
Advanced Machine Learning - Exercise 3 Deep learning essentials Introduction Whats the plan?

Advanced Machine Learning - Exercise 3 Deep learning essentials Introduction Whats the plan?

Advanced Machine Learning - Exercise 3 Deep learning essentials Introduction Whats the plan? Exercise overview Deep learning in a nutshell Backprop in (painful) detail 2 of 13 Introduction Exercise overview Goal: implement a simple DL

407 views • 22 slides
Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice Scaling up DL 2 What is Deep Learning? 3 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY SECURITY & DEFENSE MEDIA &

1.45k views • 39 slides
Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of

Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of

Introduction to Deep Learning M S Ram Dept. of Computer Science & Engg. Indian Institute of Technology Kanpur Reading of Chap. 1 from Learning Deep Architectures for AI; Yoshua Bengio; FTML Vol. 2, No. 1 (2009) 1 127 Date: 12 Nov,

570 views • 19 slides
Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides
EE-559 Deep learning 1a. Introduction Fran cois Fleuret https://fleuret.org/dlc/

EE-559 Deep learning 1a. Introduction Fran cois Fleuret https://fleuret.org/dlc/

EE-559 Deep learning 1a. Introduction Fran cois Fleuret https://fleuret.org/dlc/ [version of: June 5, 2018] COLE POLYTECHNIQUE FDRALE DE LAUSANNE Why learning Fran cois Fleuret EE-559 Deep learning / 1a. Introduction 2

1.13k views • 87 slides
9.1 Overview 9 Deep Learning Alexander Smola Introduction to Machine Learning 10-701

9.1 Overview 9 Deep Learning Alexander Smola Introduction to Machine Learning 10-701

9.1 Overview 9 Deep Learning Alexander Smola Introduction to Machine Learning 10-701 http://alex.smola.org/teaching/10-701-15 A brief history of computers 1970s 1980s 1990s 2000s 2010s Data 10 5 10 8 10 2 10 3 10 11 RAM ? 1MB 100MB

932 views • 57 slides
Deep Learning for Natural Language Processing (in 2 hours) Eneko Agirre

Deep Learning for Natural Language Processing (in 2 hours) Eneko Agirre

Deep Learning for Natural Language Processing (in 2 hours) Eneko Agirre http://ixa2.si.ehu.eus/eneko IXA NLP group http://ixa.eus @eagirre Contents Introduction to Deep learning Deep Learning ~ Learning Representations Text as

1.58k views • 156 slides
Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI Mission Helping people solve challenging problems using AI and deep learning. Developers, data scientists and engineers Self-driving cars,

1.44k views • 72 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Introduction to Machine Learning Deep Learning Barnabs Pczos Credits Many of the pictures,

Introduction to Machine Learning Deep Learning Barnabs Pczos Credits Many of the pictures,

Introduction to Machine Learning Deep Learning Barnabs Pczos Credits Many of the pictures, results, and other materials are taken from: Ruslan Salakhutdinov Joshua Bengio Geoffrey Hinton Yann LeCun 2 Contents Definition

599 views • 44 slides
Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin Clark Deep Learning for NLP 5 years ago No Seq2Seq No Attention No large-scale QA/reading comprehension datasets No TensorFlow or

1.41k views • 92 slides
Introduction to Deep Learning Milan Straka February 24, 2020 Charles University in Prague

Introduction to Deep Learning Milan Straka February 24, 2020 Charles University in Prague

NPFL114, Lecture 1 Introduction to Deep Learning Milan Straka February 24, 2020 Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise stated Deep Learning Highlights

347 views • 34 slides

More recommend