deep learning for vision
play

Deep Learning for Vision Presented by Kevin Matzen Wednesday, - PowerPoint PPT Presentation

Sep 15, 2022 •290 likes •717 views

Deep Learning for Vision Presented by Kevin Matzen Wednesday, April 9, 14 Quick Intro - DNN Feed-forward Sparse connectivity (layer to layer) Different layer types Recently popularized for vision [Krizhevsky, et. al. NIPS 2012]

  1. Deep Learning for Vision Presented by Kevin Matzen Wednesday, April 9, 14

  2. Quick Intro - DNN • Feed-forward • Sparse connectivity (layer to layer) • Different layer types • Recently popularized for vision [Krizhevsky, et. al. NIPS 2012] Wednesday, April 9, 14

  3. The Layers • Convolution • Loss functions • Fully connected • Image processing • Pooling • Neuron activation function • Normalization Wednesday, April 9, 14

  4. deeplearning.net/tutorial/lenet.html Wednesday, April 9, 14

  5. [Krizhevsky, NIPS 2012] Wednesday, April 9, 14

  6. Software • code.google.com/p/cuda-convnet/ [nvidia gpu] • github.com/UCB-ICSI-Vision-Group/decaf-release/ [deprecated; cpu-only] • caffe.berkeleyvision.org [cpu; nvidia gpu] • research.google.com/archive/ large_deep_networks_nips2012.html [proprietary; distributed system] Wednesday, April 9, 14

  7. DeepPose: Human Pose Estimation via Deep Neural Networks Alexander Toshev, Christian Szegedy - CVPR 2014 DeepFace: Closing the Gap to Human-Level Performance in Face Verification Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, Lior Wolf - CVPR 2014 Wednesday, April 9, 14

  8. DeepPose: Human Pose Estimation via Deep Neural Networks Alexander Toshev, Christian Szegedy - CVPR 2014 DeepFace: Closing the Gap to Human-Level Performance in Face Verification Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, Lior Wolf - CVPR 2014 Wednesday, April 9, 14

  9. Input: Uncropped photo Output: Joint locations Wednesday, April 9, 14

  10. Pipeline 1. Person detection 2. Joint position regression 3. Joint refinement Wednesday, April 9, 14

  11. Datasets Leeds Sports Pose (LSP) [Johnson, et. al. BMVC 2010] 14 joint locations 2000 main person - 150 px Frames Labeled in Cinema (FLIC) [Sapp, et. al. CVPR 2013] 5003 person detector every 10 frames of 30 movies 20k candidates Image Parse [Ramanan NIPS 2006] mturk 305 images 10 upperbody joints similar to leeds includes casual photos Buffy Stickmen 748 frames Wednesday, April 9, 14

  12. Person Detection • Input: Uncropped image • Output: Cropped image • LSP dataset - No person detector • FLIC dataset - Enlarged face detector Wednesday, April 9, 14

  13. Wednesday, April 9, 14

  14. Main difference Wednesday, April 9, 14

  15. Wednesday, April 9, 14

  16. Runtime • 0.1s per image - 12 cores (SotA - 1.5s, 4s) • Training stage 0 - 3 days • Training refinement - 7 days each Wednesday, April 9, 14

  17. Evaluation • Percentage of Correct Parts (PCP) • Correct if predicted limb is within 1/2 of correct limb length • Percentage of Detected Joints (PDJ) • Predicted and correct joints are within some factor of torso diameter Wednesday, April 9, 14

  18. Wednesday, April 9, 14

  19. Wednesday, April 9, 14

  20. Wednesday, April 9, 14

  21. Wednesday, April 9, 14

  22. DeepPose: Human Pose Estimation via Deep Neural Networks Alexander Toshev, Christian Szegedy - CVPR 2014 DeepFace: Closing the Gap to Human-Level Performance in Face Verification Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, Lior Wolf - CVPR 2014 Wednesday, April 9, 14

  23. Pipeline • Detect faces • Correct out-of-plane rotation • Generate features via CNN • Classify Wednesday, April 9, 14

  24. Alignment Wednesday, April 9, 14

  25. Fiducial Detection • LBP histograms • Support Vector Regressor • Iteratively transform and predict • 6 fiducial points for 2D alignment • 67 fiducial points for 3D alignment Wednesday, April 9, 14

  26. 3D Alignment • Iterative affine camera PnP • 3D reference - Average mesh of USF Human-ID dataset • Considers fiducial covariance • Residuals applied to reference mesh • Affine warp texture Wednesday, April 9, 14

  27. CNN Architecture Wednesday, April 9, 14

  28. CNN Architecture Features Wednesday, April 9, 14

  29. CNN Architecture weight sharing no weight sharing Wednesday, April 9, 14

  30. Training softmax cross-entropy loss -log p k Wednesday, April 9, 14

  31. Sparsity • ReLU nonlinearly - rectified linear unit max(0, x) • 75% model parameters = 0 • Dropout - first fully connected layer Wednesday, April 9, 14

  32. Normalization • ReLU - unbounded • Normalize features to [0, 1] based on holdout Wednesday, April 9, 14

  33. Verification Metrics • Unsupervised - dot product • χ 2 similarity • Siamese network Wednesday, April 9, 14

  34. Χ 2 Similarity • Χ 2 (f 1 ,f 2 ) = Σ i w i (f 1 [i] - f 2 [i]) 2 /(f 1 [i] + f 2 [i]) • weights learned via svm Wednesday, April 9, 14

  35. Siamese Network FC 4096-to-1 - Wednesday, April 9, 14

  36. Datasets • Social Face Classification (SFC) • Presumably Facebook photos • 4.4 mil faces; 4,030 people • No overlap with other datasets Wednesday, April 9, 14

  37. Datasets • Labeled Faces in the Wild (LFW) • 13,323 faces; 5,749 celebs • 6,000 pairs • Restricted protocol - same/not same labels at training • Unrestricted protocol - identities during training • Unsupervised - no training on LFW Wednesday, April 9, 14

  38. Datasets • YouTube Faces (YTF) • 3,425 videos of 1,595 subjects • Subset of celebs from LFW Wednesday, April 9, 14

  39. SFC Training Perf Reduce data by Reduce data by Remove layers omitting people omitting examples from network Wednesday, April 9, 14

  40. LFW Perf Wednesday, April 9, 14

  41. Runtime • 0.18 s - feature extraction (1 core; 2.2 GHz) • 0.05 s - alignment • 0.33 s - total Wednesday, April 9, 14

  42. Questions? Wednesday, April 9, 14

Recommend

Deep Learning in Computer Vision Caner Hazrba Deep Learning in Action 24. June 15

Deep Learning in Computer Vision Caner Hazrba Deep Learning in Action 24. June 15

Deep Learning in Computer Vision Caner Hazrba Deep Learning in Action 24. June 15 Computer Vision Group 6 Postdocs, 16 PhD students Caner Hazrba | vision.in.tum.de Deep Learning in Computer Vision 2 Research in Computer Vision

960 views • 30 slides
FUNDAMENTALS OF DEEP LEARNING FOR COMPUTER VISION Twin Karmakharm DLI Certified Instructor

FUNDAMENTALS OF DEEP LEARNING FOR COMPUTER VISION Twin Karmakharm DLI Certified Instructor

FUNDAMENTALS OF DEEP LEARNING FOR COMPUTER VISION Twin Karmakharm DLI Certified Instructor This event is organised and run by... What we do: Project work and consultancy Training Deep Learning, HPC, GPU Deep Learning

651 views • 32 slides
Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision : Image Classification,

1.53k views • 23 slides
Deep Learning and its Applications in Vision and Robotics

Deep Learning and its Applications in Vision and Robotics

#dlvr Deep Learning and its Applications in Vision and Robotics http://juxi.net/workshop/deeplearning-applications-vision-robotics-2015/ Deep Learning and its Applications in Vision and Robotics #dlvr Programme 09:15-09:30 Welcome by Juxi,

388 views • 6 slides
APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston Practical DEEP

APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston Practical DEEP

APPLICATIONS OF DEEP LEARNING TO COMPUTER VISION AND COMPUTER GRAPHICS Mike Houston Practical DEEP LEARNING Examples Image Classification, Object Detection, Localization, Speech Recognition, Speech Translation, Action Recognition, Scene

885 views • 40 slides
EE-559 Deep learning 7. Networks for computer vision Fran cois Fleuret

EE-559 Deep learning 7. Networks for computer vision Fran cois Fleuret

EE-559 Deep learning 7. Networks for computer vision Fran cois Fleuret https://fleuret.org/dlc/ [version of: June 8, 2018] COLE POLYTECHNIQUE FDRALE DE LAUSANNE Tasks and data-sets Fran cois Fleuret EE-559 Deep learning

1.93k views • 170 slides
Deep Learning in Computer Vision Yikang Li MMLab, The Chinese University of Hong Kong Sep 22nd,

Deep Learning in Computer Vision Yikang Li MMLab, The Chinese University of Hong Kong Sep 22nd,

Deep Learning in Computer Vision Yikang Li MMLab, The Chinese University of Hong Kong Sep 22nd, 2017 @Microsoft Research Asia, China Outline 1. Introduction 2. Roadmap of Deep Learning 3. DL in CV: Object detection 4. DL in CV: Image

630 views • 45 slides
E9 205 Machine Learning for Signal Procesing Deep Learning for Audio and Vision 20-11-2019

E9 205 Machine Learning for Signal Procesing Deep Learning for Audio and Vision 20-11-2019

E9 205 Machine Learning for Signal Procesing Deep Learning for Audio and Vision 20-11-2019 Speech Recognition Noise Channel Automatic Speech Systems Courtesy Google Images Signal Modeling Short-term spectra integrated in mel frequency

555 views • 32 slides
Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Lecture 6: CNNs and Deep Q Learning 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2019 1 With many slides for DQN from David Silver and Ruslan Salakhutdinov and some vision slides from Gianni Di Caro and images from Stanford CS231n,

1.17k views • 68 slides
Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Table of Contents Convolutional Neural Nets (CNNs) 1 Deep Q Learning 2 Lecture 6: CNNs and Deep

Lecture 6: CNNs and Deep Q Learning 2 Emma Brunskill CS234 Reinforcement Learning. Winter 2018 2 With many slides for DQN from David Silver and Ruslan Salakhutdinov and some vision slides from Gianni Di Caro and images from Stanford CS231n,

850 views • 67 slides
Delving Deep into Computer Vision Caner Hazirbas Machine Learning Meetup #1 Delving Deep into

Delving Deep into Computer Vision Caner Hazirbas Machine Learning Meetup #1 Delving Deep into

Delving Deep into Computer Vision Caner Hazirbas Machine Learning Meetup #1 Delving Deep into Computer Vision FlowNet FuseNet PoseLSTM DDFF Caner Hazirbas | hazirbas@cs.tum.edu Delving Deep into Computer Vision 2 Delving Deep into

1.14k views • 45 slides
Deep multi-task learning with evolving weights Machine learning - computer vision published in

Deep multi-task learning with evolving weights Machine learning - computer vision published in

Deep multi-task learning with evolving weights Machine learning - computer vision published in European Symposium on Artificial Neural Networks (ESANN 2016) Soufiane Belharbi Romain Hrault Clment Chatelain Sbastien Adam

631 views • 44 slides
Impact of Deep Learning Speech Recogni4on Computer Vision

Impact of Deep Learning Speech Recogni4on Computer Vision

Impact of Deep Learning Speech Recogni4on Computer Vision Recommender Systems Language Understanding Drug Discovery and Medical Image

526 views • 21 slides
Deep learning 8.1. Computer vision tasks Fran cois Fleuret https://fleuret.org/dlc/ Dec 20,

Deep learning 8.1. Computer vision tasks Fran cois Fleuret https://fleuret.org/dlc/ Dec 20,

Deep learning 8.1. Computer vision tasks Fran cois Fleuret https://fleuret.org/dlc/ Dec 20, 2020 Computer vision tasks: classification, object detection, semantic or instance segmentation, Fran cois Fleuret Deep learning /

426 views • 20 slides
BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem

BBM413 Fundamentals of Image Processing Introduction to Deep Learning Erkut Erdem Hacettepe University Computer Vision Lab (HUCVL) What is deep learning? Y. LeCun, Y. Bengio, G. Hinton, "Deep Learning" , Nature,

1.12k views • 84 slides
Recent Trends in Computer Vision and Deep Learning Systems Yangqing Jia Lead Researcher and

Recent Trends in Computer Vision and Deep Learning Systems Yangqing Jia Lead Researcher and

Recent Trends in Computer Vision and Deep Learning Systems Yangqing Jia Lead Researcher and Manager of AI Platform, Facebook Computer Vision AlexNet So it begins. VGGNet Punch it. GoogLeNet We must go deeper. ResNet And we took the word

424 views • 40 slides
Distributed Deep Learning with Horovod Alex Sergeev, Machine Learning Platform, Uber Engineering

Distributed Deep Learning with Horovod Alex Sergeev, Machine Learning Platform, Uber Engineering

Distributed Deep Learning with Horovod Alex Sergeev, Machine Learning Platform, Uber Engineering @alsrgv Deep Learning Continues to improve accuracy long after older algorithms reach data saturation State of the art for vision,

866 views • 41 slides
Introduction to Deep Learning Principles and applications in vision and natural language

Introduction to Deep Learning Principles and applications in vision and natural language

Introduction to Deep Learning Principles and applications in vision and natural language processing Jakob Verbeek (INRIA) Slides in collaboration with Laurent Besacier (Univ. Grenoble Alpes) 2018 Introduction Convolutional Neural Networks

517 views • 28 slides
Deep Learning in the Connected Kitchen or Launching a Computer Vision program in a new

Deep Learning in the Connected Kitchen or Launching a Computer Vision program in a new

Deep Learning in the Connected Kitchen or Launching a Computer Vision program in a new vertical Hristo Bojinov, CTO Company Vision The Problem Food People disconnect Not-so-smart smart kitchen Food info not available, not

491 views • 28 slides
Compositional Methods for Learning and Inference in Deep Probabilistic Programs Jan-Willem

Compositional Methods for Learning and Inference in Deep Probabilistic Programs Jan-Willem

Compositional Methods for Learning and Inference in Deep Probabilistic Programs Jan-Willem van de Meent Eli Sennesh Sam Stites Hao Wu Heiko Zimmermann Deep Learning Success Stories Computer Vision Natural Language Reinforcement

710 views • 40 slides
CSE 802 Spring 2017 Deep Learning Inci M. Baytas Michigan State University February 13-15,

CSE 802 Spring 2017 Deep Learning Inci M. Baytas Michigan State University February 13-15,

CSE 802 Spring 2017 Deep Learning Inci M. Baytas Michigan State University February 13-15, 2017 1 Deep Learning in Computer Vision Large-scale Video Classification with Convolutional Neural Networks, CVPR 2014 2 Deep Learning in Computer

918 views • 32 slides
Recognition and Classification of Radioactive Waste using Computer Vision-based Deep Learning

Recognition and Classification of Radioactive Waste using Computer Vision-based Deep Learning

Transactions of the Korean Nuclear Society Virtual Spring Meeting July 9-10, 2020 Recognition and Classification of Radioactive Waste using Computer Vision-based Deep Learning Technology Sung-Chan Jang a , Dong-ju Lee a , Il-Sik Kang a ,

52 views • 3 slides
PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING ARJUN

PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING ARJUN

PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING ARJUN CHANDRASEKARAN DEEP LEARNING AND PERCEPTION (ECE 6504) PLAYING ATARI WITH DEEP REINFORCEMENT LEARNING NEURAL NETWORK VISION FOR ROBOT DRIVING

1.07k views • 62 slides
Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides

More recommend