Large-Scale R-CNN with Classifier Adaptive Quantization Redmon et - PowerPoint PPT Presentation

Large-Scale R-CNN with Classifier Adaptive Quantization Redmon et al., ECCV 2016 Mincheul Kang 1

Review • Deepfashion: Powering robust clothes recognition and retrieval with rich annotations – CVPR 16 • Try to solve this problem with rich annotations • Category and Attribute Annotation • Landmarks Annotation • Pair annotation 2 From Joongun’s slide

Contents 1. Introduction 2. Background 3. Related work 4. Approach 5. Result 6. Conclusion 3

Introduction • Object detection • Image retrieval, Robotics, Self-driving car Large-scale Fast and Accurate Fast R-CNN slides : Ross Girshick 4 http://www.nvidia.com/object/drive-px.html http://kitschthingoftheday.blogspot.com/2011/06/breakfast-making-robots-at-tum.html

Introduction • Real-time, Large-scale object detection • Retrieve objects for a certain category from large image collections immediately and accurately • Problem : huge costs Large-scale R-CNN with Classifier Adaptive Quantization, 5 R Hinami et al., ECCV 2016

Introduction • Extend R-CNN for large scale • Classify millions/billions of features in real- time • Use the techniques of nearest neighbor search • Collaboration of two fields Large-scale R-CNN with Classifier Adaptive Quantization, 6 R Hinami et al., ECCV 2016

Background • Inverted index • Generate a codebook by quantization • e.g. k-means clustering • Given a query, • Find its K closest words • Retrieve all the data in the K lists corresponding to the words 7 From lecture notes

Background • K-means Clustering • A method of vector quantization • Minimize the within-cluster sum of squares 8 http://sanghyukchun.github.io/69/

Background • Product Quantization (PQ) • Use separate small codebook for each chunk • Low memory/time cost • Vector split into m subvectors • Subvectors are quantized separately by quantizers => 64-bit quantization index 9 http://nick0702.blogspot.kr/2013/03/aggregating-local-descriptors-into.html

Related work • YOLO : Real-time object detection • Predict all bounding boxes across all classes for an image simultaneously • Pascal 2007 (5k) -> 20 classes • R-CNN, Fast R-CNN, Faster R-CNN 69.0 You only look once: Unified, real-time object detection, 10 J Redmon et al., CVPR 2016

Related work • R-CNN (Region proposals + CNN) • Selective search • CNN that extracts a fixed-length feature vector from each region • Binary linear SVMs Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, 11 Ross Girshick et al., CVPR 2014

Approach • Framework of Large-scale R-CNN Large-scale R-CNN with Classifier Adaptive Quantization, 12 R Hinami et al., ECCV 2016

Approach • Offline procedure • Detect proposal objects and extract features • Similar to R-CNN Large-scale R-CNN with Classifier Adaptive Quantization, 13 R Hinami et al., ECCV 2016

Approach • Construct an inverted index • Avoid exhaustive search by an inverted index • Use the classifier adaptive quantization(CAQ) to learn codebook instead of k-means Large-scale R-CNN with Classifier Adaptive Quantization, 14 R Hinami et al., ECCV 2016

Approach • Limitation of k-means • It is suitable for nearest neighbor search • But, It is not optimal for linear classification • Instead of K-means -> CAQ Large-scale R-CNN with Classifier Adaptive Quantization, 15 R Hinami et al., ECCV 2016

Approach • Classifier Adaptive Quantization (CAQ) • CAQ is based on the k-means algorithm • Linear SVM classifier Large-scale R-CNN with Classifier Adaptive Quantization, 16 R Hinami et al., ECCV 2016

Approach • Compress data with residual vector quantization Large-scale R-CNN with Classifier Adaptive Quantization, 17 R Hinami et al., ECCV 2016

Approach • Residual vector quantization (RVQ) • PQ-based quantization • Reduce the quantization error • Learn multiple sub-codebooks one by one by minimizing the error greedily. Residual vectors Learning codebooks Quantizing a vector 18 Approximate nearest neighbor search by residual vector quantization, Chen et al., Sensors 10.12 2010

Approach • Comparison of PQ, OPQ and RVQ • Data compression methods on PASCAL VOC 2007 Large-scale R-CNN with Classifier Adaptive Quantization, 19 R Hinami et al., ECCV 2016

Approach • Online procedure Large-scale R-CNN with Classifier Adaptive Quantization, 20 R Hinami et al., ECCV 2016

Results • Efficiency over R-CNN • PASCAL dataset(5K images, 10M features) • 250x faster, 106x memory reduction with comparable accuracy 69.0 GoogLeNet VGGNet-16 You only look once: Unified, real-time object detection, J Redmon et al., CVPR 2016 21 Large-scale R-CNN with Classifier Adaptive Quantization, R Hinami et al., ECCV 2016

Results • Large-scale dataset PASCAL+Imagenet (~105K images) • 105K search in 130ms Large-scale R-CNN with Classifier Adaptive Quantization, 22 R Hinami et al., ECCV 2016

Results • CAQ vs K-means • CAQ improves by ~10% mAP over K-means The number of category Inverted multi index Large-scale R-CNN with Classifier Adaptive Quantization, 23 R Hinami et al., ECCV 2016

Conclusion • Pros • Compared to R-CNN, 250x speed-up and 106x memory reduction • Quickly detect the object on large-scale data. • Present classifier adaptive quantization instead of k- means • Cons • The accuracy is not high yet. 24

Q & A 25