rich feature hierarchies for accurate object detection
play

Rich feature hierarchies for accurate object detection and semantic - PowerPoint PPT Presentation

Sep 18, 2023 •329 likes •704 views

Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Je ff Donahue, Trevor Darrell, Jitendra Malik UC Berkeley Tech Report @ http://arxiv.org/abs/1311.2524 Detection & Segmentation input

  1. Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Je ff Donahue, Trevor Darrell, Jitendra Malik UC Berkeley � Tech Report @ http://arxiv.org/abs/1311.2524

  2. Detection & Segmentation input person background motorbike person motorbike

  3. PASCAL VOC Example PASCAL VOC images

  4. Dominant detection methods 1. Part-based sliding window methods (HOG) DPM Poselets 2. Region-proposal classifiers (SIFT++ BoW) Russell et al. 2006 Gu et al. 2009 van de Sande et al. 2011 > “selective search”

  5. PASCAL VOC epochs (detection) 2007-2010 The Moore’s law years � 2010-2011 The year of kitchen sinks (or the all-too-soon end of Moore’s law) � 2011-2012 Stagnation (no new features le fu , juice all squeezed from context) � 2013– Learning rich features?

  6. ImageNet LSVRC’12 winner UToronto “SuperVision” CNN Krizhevsky, Sutskever, and Hinton. ImageNet Classification with Deep Convolutional Neural Networks. NIPS 2012. � cf. LeCun et al. Neural Comp. ’89 & Proc. of the IEEE ‘98

  7. Impressive ImageNet results! Task: 1000-way whole-image classification metric: classification error rate (lower is better) But... does it generalize to other datasets and tasks? See: Donahue, Jia, et al. DeCAF Tech Report. � Much debate at ECCV’12

  8. Objective Understand if the SuperVision CNN can be made to work as an object detector.

  9. Object detection system R-CNN: “Regions with CNN features” warped region aeroplane ? no. . . . person? yes. . . . CNN tvmonitor? no. 1 . Input 2 . Extract region 3 . Compute 4 . Classify image proposals (~2k) CNN features regions (With a few minor tweaks: semantic segmentation) (e.g. selective search)

  10. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet)

  11. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet)

  12. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet) 2. Fine-tune CNN on target dataset and task fine-tune CNN (optional) small target dataset (PASCAL VOC)

  13. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet) 2. Fine-tune CNN on target dataset and task fine-tune CNN (optional) small target dataset (PASCAL VOC)

  14. Training 1. Pre-train CNN for image classification train CNN large auxiliary dataset (ImageNet) 2. Fine-tune CNN on target dataset and task fine-tune CNN (optional) small target dataset (PASCAL VOC) 3. Train linear predictor for detection CNN features region proposals per class ~2000 warped SVM windows / image small target training labels dataset (PASCAL VOC)

  15. Training labels 3. Train linear predictor for detection CNN features region proposals per class ~2000 warped SVM windows / image training labels small target dataset (PASCAL VOC) labeling protocol positives = ground truth negatives = max IoU < 0.3

  16. CNN features for detection warped region region pool 5 : 6 x 6 x 256 = 9216-dim 6.4% / 15% non-zero � fc 6 : 4096-dimensional 71.2% / 20% nz � fc 7 : 4096-dimensional 100% / 20% nz

  17. Results VOC 2007 VOC 2010 reference DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% R-CNN pool 5 40.1% R-CNN fc 6 43.4% R-CNN fc 7 42.6% R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  18. Results VOC 2007 VOC 2010 DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% pre-trained R-CNN pool 5 40.1% only R-CNN fc 6 43.4% R-CNN fc 7 42.6% R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  19. Results VOC 2007 VOC 2010 DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% R-CNN pool 5 40.1% R-CNN fc 6 43.4% R-CNN fc 7 42.6% fine-tuned R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  20. Results — update VOC 2007 VOC 2010 DPM v5 (Girshick et al. 2011) 33.7% 29.6% UVA sel. search (Uijlings et al. 2012) 35.1% Regionlets (Wang et al. 2013) 41.7% 39.7% pre-trained R-CNN pool 5 40.1% 44.0% only R-CNN fc 6 43.4% 46.2% R-CNN fc 7 42.6% 43.5% R-CNN FT pool 5 42.1% R-CNN FT fc 6 47.2% R-CNN FT fc 7 48% 43.5% metric: mean average precision (higher is better)

  21. CV and DL together Good features are not enough! warped region aeroplane ? no. . . . person? yes. . . . CNN tvmonitor? no. 1 . Input 2 . Extract region 3 . Compute 4 . Classify image proposals (~2k) CNN features regions Computer Deep Computer Vision Learning Vision

  22. Top bicycle FPs (AP 62.5%)

  23. Top bird FPs (AP 41.4%)

  24. False positive types: cat DPM voc − release5: cat CNN FT fc7: cat 100 100 percentage of each type percentage of each type 80 80 60 60 40 40 Loc Loc Sim Sim 20 20 Oth Oth BG BG 0 0 25 100 400 1600 6400 25 100 400 1600 6400 total false positives total false positives AP 56.3% AP 23.0% Analysis so fu ware from: D. Hoiem, Y. Chodpathumwan, and Q. Dai. “Diagnosing Error in Object Detectors.” ECCV , 2012.

  25. Visualizing features > What does pool 5 learn? > Recap: > pool 5 : max-pooled output of last conv. layer > 6 x 6 spatial structure (with 256 channels) > receptive field size 163 x 163 (of 224 x 224) 6 6 256 unit position receptive field

  26. Visualization method > Select a unit in pool 5 > Run it as a detector > Show top-scoring regions > Non-parametric, lets unit “speak for itself” � � � (Used ~10 million held-out regions.)

  27. pool5 feature: (3,3,42) (top 1 − 96) 0.9 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

  28. pool5 feature: (3,4,80) (top 1 − 96) 0.9 0.8 0.8 0.8 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4

  29. pool5 feature: (4,5,110) (top 1 − 96) 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.4 0.3 0.3 0.3

  30. pool5 feature: (3,5,129) (top 1 − 96) 0.9 0.9 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6

  31. pool5 feature: (4,2,26) (top 1 − 96) 0.8 0.8 0.8 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

  32. pool5 feature: (3,3,39) (top 1 − 96) 0.8 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

Recommend

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Authors: Ross

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Authors: Ross

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Authors: Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik Presented by Huihuang Zheng Problem: Object Detection Regionlets SegDPM (2013) Selective

499 views • 19 slides
Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection

Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection

Convolutional Feature Maps Elements of efficient (and accurate) CNN-based object detection Kaiming He Microsoft Research Asia (MSRA) Overview of this section Quick introduction to convolutional feature maps Intuitions: into the black

786 views • 41 slides
3D Object Proposals using Stereo Imagery for Accurate Object Class Detection Xiaozhi Chen,

3D Object Proposals using Stereo Imagery for Accurate Object Class Detection Xiaozhi Chen,

3D Object Proposals using Stereo Imagery for Accurate Object Class Detection Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Huimin Ma, Sanja Fidler and Raquel Urtasun Presentation by Jungwook Lee Why use proposals? - Smart proposal generation methods

287 views • 25 slides
Bounding Box Regression With Uncertainty for Accurate Object Detection 1 Carnegie Mellon

Bounding Box Regression With Uncertainty for Accurate Object Detection 1 Carnegie Mellon

Bounding Box Regression With Uncertainty for Accurate Object Detection 1 Carnegie Mellon University 2 Megvii Yihui He 1 , Chenchen Zhu 1 , Jianren Wang 1 , Marios Savvides, 2 Xiangyu Zhang Ambiguity: inaccurate labelling MS-COCO Ambiguity:

713 views • 32 slides
S9243 Fast and Accurate Object Detection Floris Chabert , Solutions Architect with PyTorch and

S9243 Fast and Accurate Object Detection Floris Chabert , Solutions Architect with PyTorch and

S9243 Fast and Accurate Object Detection Floris Chabert , Solutions Architect with PyTorch and TensorRT Prethvi Kashinkunti , Solutions Architect March 19 2019 OVERVIEW Topics What &amp; Why? Problem Our solution How? Architecture

1.78k views • 34 slides
Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW APPLICATIONS - Object detection

Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW APPLICATIONS - Object detection

Feature Extraction Manos Baltsavias, Haris Papasaika OVERVIEW APPLICATIONS - Object detection and localisation (2D measurement) - Matching (3D information) - Image analysis and scene interpretation , e.g. Edgels -&gt; edge segments -&gt;

1.6k views • 84 slides
Feature Selection Matters for Anchor-Free Object Detection Chenchen Zhu Carnegie Mellon

Feature Selection Matters for Anchor-Free Object Detection Chenchen Zhu Carnegie Mellon

Carnegie Mellon Feature Selection Matters for Anchor-Free Object Detection Chenchen Zhu Carnegie Mellon University 04/29/2020 Carnegie Mellon Overview Background Motivation Feature Selection in Anchor-Free Detection General

466 views • 42 slides
Feature-level Phase Detection for Execution Trace Using Object Cache Yui Watanabe, Takashi Ishio,

Feature-level Phase Detection for Execution Trace Using Object Cache Yui Watanabe, Takashi Ishio,

Feature-level Phase Detection for Execution Trace Using Object Cache Yui Watanabe, Takashi Ishio, Katsuro Inoue Osaka University, Japan July 21, 2008 WODA 2008 1 Software Engineering Laboratory, Department of Computer Science, Graduate School

202 views • 19 slides
Object Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 48 Object Detection The

Object Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 48 Object Detection The

Object Detection Sanja Fidler CSC420: Intro to Image Understanding 1 / 48 Object Detection The goal of object detection is to localize objects in an image and tell their class Localization: place a tight bounding box around object Most

1.17k views • 91 slides
Automated Feature Extraction Automated Feature Extraction for Object Recognition for Object

Automated Feature Extraction Automated Feature Extraction for Object Recognition for Object

Automated Feature Extraction Automated Feature Extraction for Object Recognition for Object Recognition I.Levner and V.Bulitko http://ircl.cs.ualberta.ca Outline Outline Motivation System Overview Feature Extraction Problem

601 views • 15 slides
A Review on Salient Object Detection Feng Lin Salient Object Detection Target Detect and

A Review on Salient Object Detection Feng Lin Salient Object Detection Target Detect and

A Review on Salient Object Detection Feng Lin Salient Object Detection Target Detect and segment salient objects in natural scenes a) good detection b) high resolution c) computational efficiency Metric F-score MAE (mean

1.12k views • 39 slides
Detection, Segmentation Overview Object Detection deer cat Object Detection as Classification

Detection, Segmentation Overview Object Detection deer cat Object Detection as Classification

CS6501: Deep Learning for Visual Recognition Detection, Segmentation Overview Object Detection deer cat Object Detection as Classification deer? cat? CNN background? Object Detection as Classification deer? cat? CNN background?

577 views • 28 slides
Lecture 11: Object detection Contains slides from S. Lazebnik, R. Girshick, B. Hariharan 1

Lecture 11: Object detection Contains slides from S. Lazebnik, R. Girshick, B. Hariharan 1

Lecture 11: Object detection Contains slides from S. Lazebnik, R. Girshick, B. Hariharan 1 Object detection with bounding boxes What? Where? Object detection Source: R. Girshick 2 Evaluating an object detector At test time,

2.51k views • 56 slides
From image classification to object detection Image classification Object detection Image source

From image classification to object detection Image classification Object detection Image source

From image classification to object detection Image classification Object detection Image source Slides from L. Lazebnik What are the challenges of object detection? Images may contain more than one class, multiple instances from the same

1.11k views • 57 slides
Making Object Detection Work in Medical Imaging Idan Bassuk Segmentation Detection

Making Object Detection Work in Medical Imaging Idan Bassuk Segmentation Detection

Making Object Detection Work in Medical Imaging Idan Bassuk Segmentation Detection Classification 4 5 Mature technology 6 Mature, Accurate and Fast 7 Changing the world, one killer app at a time 8 Changing the world, one killer app at

570 views • 39 slides
Object based feature extraction of Google based feature extraction of Google Object Earth

Object based feature extraction of Google based feature extraction of Google Object Earth

International Electronic Conference on Sensors and Applications 1 16 June 2014 Object based feature extraction of Google based feature extraction of Google Object Earth i imagery for mapping termite mounds in magery for mapping termite

279 views • 9 slides
Previously Instance recognition Local features: detection and description Window-based

Previously Instance recognition Local features: detection and description Window-based

4/11/2011 Previously Instance recognition Local features: detection and description Window-based models for Local feature matching, scalable indexing Spatial verification generic object detection Intro to generic object

429 views • 11 slides
Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representation Eliyahu

Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representation Eliyahu

Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representation Eliyahu Kiperwasser &amp; Yoav Goldberg 2016 Presented by: Yaoyang Zhang Outline Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature

344 views • 22 slides
Fast and Adaptive Online Training of Feature-Rich Translation Models Spence Green Sida Wang

Fast and Adaptive Online Training of Feature-Rich Translation Models Spence Green Sida Wang

Fast and Adaptive Online Training of Feature-Rich Translation Models Spence Green Sida Wang Daniel Cer Christopher D. Manning Stanford University ACL 2013 Feature-Rich Research Industry/Evaluations Liang et al. 2006 Tillmann and Zhang

1.24k views • 93 slides
Fast and Adaptive Online Training of Feature-Rich Translation Models Spence Green Sida Wang

Fast and Adaptive Online Training of Feature-Rich Translation Models Spence Green Sida Wang

Fast and Adaptive Online Training of Feature-Rich Translation Models Spence Green Sida Wang Daniel Cer Christopher D. Manning Stanford University ACL 2013 Feature-Rich Research Industry/Evaluations Liang et al. 2006 Tillmann and Zhang

899 views • 65 slides
The Status of Phoneme Inventories: The Role of Contrastive Feature Hierarchies B. Elan Dresher

The Status of Phoneme Inventories: The Role of Contrastive Feature Hierarchies B. Elan Dresher

Congrs de lACL 2020 | 2020 CLA meeting May 30June 1, 2020 The Status of Phoneme Inventories: The Role of Contrastive Feature Hierarchies B. Elan Dresher Daniel Currie Hall Sara Mackenzie Toronto Saint Marys Memorial 1 1.

687 views • 68 slides
Object Detection Ujjwal Post-Doc, STARS Team INRIA Sophia Antipolis Outline What is Object

Object Detection Ujjwal Post-Doc, STARS Team INRIA Sophia Antipolis Outline What is Object

Object Detection Ujjwal Post-Doc, STARS Team INRIA Sophia Antipolis Outline What is Object Detection ? Qualitative Definition. Machine Learning Definition. Ingredients of Object Detection. Components of a typical deep

683 views • 55 slides
Rapid Facial Feature Detection in iOS Instructor - Simon Lucey 16-423 - Designing Computer Vision

Rapid Facial Feature Detection in iOS Instructor - Simon Lucey 16-423 - Designing Computer Vision

Rapid Facial Feature Detection in iOS Instructor - Simon Lucey 16-423 - Designing Computer Vision Apps Today Facial Feature Detection in iOS. Active Appearance Models. Face Detection Face Detection in iOS In iOS face detection

1.05k views • 101 slides
CS488 More Geometric Transformations, Object Hierarchies, and some Fonts Luc R ENAMBOT 1

CS488 More Geometric Transformations, Object Hierarchies, and some Fonts Luc R ENAMBOT 1

CS488 More Geometric Transformations, Object Hierarchies, and some Fonts Luc R ENAMBOT 1 Previous Lectures Frame buffers Drawing a line (Midpoint Line Algorithm) Polygon Filling (Edge-table algorithm) Line Clipping

867 views • 38 slides

More recommend