hierarchical convolutional features
play

Hierarchical Convolutional Features for Visual Tracking Chao Ma - PowerPoint PPT Presentation

Sep 13, 2023 •192 likes •509 views

Hierarchical Convolutional Features for Visual Tracking Chao Ma Jia-Bin Huang Xiaokang Yang Ming-Husan Yang SJTU UIUC SJTU UC Merced ICCV 2015 Background Given the initial state

  1. Hierarchical Convolutional Features for Visual Tracking Chao Ma Jia-Bin Huang Xiaokang Yang Ming-Husan Yang SJTU UIUC SJTU UC Merced ICCV 2015

  2. Background • Given the initial state (position and scale), estimate the unknown states in the subsequence frames ˗ Model-free ˗ Single target visual tracking 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 2

  3. Real-Applications with Tracking Images from Google Search 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 3

  4. Challenges I 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 4

  5. Challenges II • Challenges = significant appearance variations over time!!! 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 5

  6. Convolutional Neural Networks • Show significant advantages on a wide range of computer vision problems: image classification, object detection, object recognition et al. AlexNet (NIPS’12) 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 6

  7. Typical Tracking Framework • Incrementally learn classifiers to separate targets from background (online learning to adapt to appearance changes) ˗ MIL (CVPR’09), Struck (ICCV’11), CT (ECCV’12), ASLA (CVPR’12), MEEM (ECCV’14), etc. 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 7

  8. Existing CNN Trackers • DLT (NIPS'13), LHF (TIP'15), DeepTrack (BMVC'14), CNN-SVM (ICML'15), MDNet (CVPR’16) This figure credits to Li et al. in the DeepTrack (BMVC’ 14) 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 8

  9. Issues of Existing CNN Trackers • Only use the last (fully-connected) layer of the CNN network for classification ˗ Too coarse to localize target precisely • Sample target states with binary labels (positive and negative) ˗ Ambiguity in labeling the spatially over-correlated samples • MDNet (CVPR’16): negative mining • Struck (ICCV’11): structure output 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 9

  10. Issues of Existing CNN Trackers • Only use the last (fully-connected) layer of the CNN network for classification ˗ Too coarse to localize target precisely • Sample target states with binary labels (positive and negative) ˗ Ambiguity in labeling the spatially over-correlated samples • MDNet (CVPR’16): negative mining • Struck (ICCV’11): structure output 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 10

  11. Our Observations • Earlier layers retain higher spatial resolution for precise localization. • Latter layers capture more semantic information and are robust to appearance changes. • Exploit the rich hierarchies for robust visual tracking. 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 11

  12. Toy Example • Layer conv5 robust to appearance change: insensitive to the sharp step edge • Layer conv3 is useful for precise localization: sensitive to the edge position 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 12

  13. Feature Visualization using VGG-Net-19 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 13

  14. Flowchart of Our Approach 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 14

  15. Issues of Existing CNN Trackers • Only use the last (fully-connected) layer of the CNN network for classification ˗ Too coarse to localize target precisely • Sample target states with binary labels (positive and negative) ˗ Ambiguity in labeling the spatially over-correlated samples • MDNet (CVPR’16): negative mining • Struck (ICCV’11): structure output 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 15

  16. Alleviating Sampling Ambiguity • Adaptive correlation filters regress the deep features with soft labels decaying from 1 to 0 ˗ Computational efficiency using FFT • Convolutional theorem: convolutional filter? correlation filter? ˗ Best exploit the contextual cues • K. Zhang et al, Fast Visual Tracking via Dense Spatio-Temporal Context Learning, in ECCV’14 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 16

  17. Correlation Filters • Correlation filters learning in the spatial domain: Vertical circular shifts of input x with corresponding soft labels generated by a Gaussian function. The first five figures credit to the KCF tracker by Henrisque et al. • Use FFT to learn correlation filter in the frequency domain as 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 17

  18. Implementation Details: Feature Interpolation • Problem: deeper layers with lower spatial resolution due to the pooling ˗ pool5-4 in VGG-Net is of spatial size 7 x 7, which is 1/32 of the input image 224 x 224 • Solution: resize each CNN layers with bilinear interpolation ˗ Affirm that deconvolution is usually helpful for finer position inference ˗ Different conclusion without feature interpolation • M. Danelljan et al. Convolutional Features for Correlation Filter Based Visual Tracking. In ICCV 2015 workshop 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 18

  19. Coarse-to-Fine Inference • For the l-th CNN layer with channel D , the response map is: • Given the location , locate the target in the ( l-1 )-th layer: 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 19

  20. Model Update • Use a moving average scheme to update the numerator and denominator of separately as: 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 20

  21. Experimental Setting • Datasets: OTB-50, and OTB-100 ˗ Yi Wu et al, Online Object Tracking: A Benchmark, in CVPR, 2013 ˗ Yi Wu et al, Object Tracking Benchmark, TPAMI, 2015 • Metrics: ˗ Distance precision rate ˗ Overlap success (intersection of union) rate • Validation schemes: ˗ OPE: one-pass evaluation ˗ TRE: temporal robustness evaluation ˗ SRE: spatial robustness evaluation • Fix parameters for all sequences 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 21

  22. Overall Results on OTB-50 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 22

  23. Overall Results on OTB-100 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 23

  24. Attribute Evaluation on OTB-50 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 24

  25. Attribute Evaluation on OTB-100 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 25

  26. Ablation Studies • Single layer (c5,c4 and c3), combination of the conv5-4 and conv4-4 layers (c5-c4), and concatenation of three layers (c543) 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 26

  27. Qualitative Results I 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 27

  28. Qualitative Results II 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 28

  29. Failure Cases 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 29

  30. Public Sources on This Work • Project webpage ˗ https://sites.google.com/site/chaoma99/iccv15_tracking • Source code ˗ https://github.com/jbhuang0604/CF2 • Further release the results of nine baseline trackers on OTB- 100 ˗ https://sites.google.com/site/chaoma99/iccv15_tracking 5/6/2016 Hierarchical Convolutional Features for Visual Tracking 30

  31. Thanks

Recommend

Learning Spatiotemporal Features with 3D Convolutional Networks Du Tran, Lubomir Bourdev, Rob

Learning Spatiotemporal Features with 3D Convolutional Networks Du Tran, Lubomir Bourdev, Rob

Learning Spatiotemporal Features with 3D Convolutional Networks Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri ada BAK 29.03.16 Effective Video Descriptor Generic Can represent different types Compact

355 views • 20 slides
Convolutional Neural Networks for Language CS 6956: Deep Learning for NLP Features from text

Convolutional Neural Networks for Language CS 6956: Deep Learning for NLP Features from text

Convolutional Neural Networks for Language CS 6956: Deep Learning for NLP Features from text Example: Sentiment classification The goal: Is the sentiment of a sentence positive, negative or neutral? The film is fun and is host to some truly

1.37k views • 88 slides
Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron

Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron

CS4501: Introduction to Computer Vision Neural Networks + Convolutional Neural Networks Last Class Global Features The perceptron model Neural Networks multilayer perceptron model (MLP) Backpropagation Todays Class

789 views • 34 slides
Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder

Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder

Convolutional Autoencoder (CAE) Prof. Seungchul Lee Industrial AI Lab. Convolutional Autoencoder Motivation: image to autoencoder ? Convolutional autoencoder extends the basic structure of the simple autoencoder by changing the fully

349 views • 15 slides
Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural

Transfer Learning with Convolutional Neural Networks ---- Off the shelf top notch performances Convolutional Neural Networks A breakthough Convolutional Neural Networks VGG-16 example Layers of Convolutional filters Bottleneck

625 views • 23 slides
A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette

A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette Phil Blunsom Department of Computer Science, Oxford University Overview of Model Represent sentences by extracting more abstract features Input:

195 views • 16 slides
Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN)

Convolutional Kuan-Ting Lai 2020/3/31 Neural Network Convolutional Neural Networks (CNN) A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital Images Input array: an images

1.42k views • 72 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides
15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter

15-780 Graduate Artificial Intelligence: Convolutional and recurrent networks J. Zico Kolter (this lecture) and Ariel Procaccia Carnegie Mellon University Spring 2017 1 Outline Convolutional neural networks Applications of convolutional

1.32k views • 32 slides
UT DA Hierarchical and Analytical Hierarchical and Analytical Pla Placement T cement

UT DA Hierarchical and Analytical Hierarchical and Analytical Pla Placement T cement

UT DA Hierarchical and Analytical Hierarchical and Analytical Pla Placement T cement Technique echniques for s for High High- Performa Performance A nce Analog C nalog Circuit ircuits Biying Xu, Shaolan Li, Xiaoqing Xu, Nan Sun, and

506 views • 29 slides
Proposal of a Hierarchical Proposal of a Hierarchical Architecture for Multimodal Architecture for

Proposal of a Hierarchical Proposal of a Hierarchical Architecture for Multimodal Architecture for

Proposal of a Hierarchical Proposal of a Hierarchical Architecture for Multimodal Architecture for Multimodal Interactive Systems y Masahiro Araki* 1 Tsuneo Nitta* 2 Kouichi Katsurada* 2 Takuya Nishimoto* 3 Tetsuo Amakasu* 4 Shinnichi Kawamoto* 5

546 views • 35 slides
and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast

and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast

Band-limited Training and Inference for Convolutional Neural Networks 1 2 FFT IFFT 3 4 Mathieu et al.: Fast Training of Convolutional Networks through FFTs Fast Convolutional Nets With fbfft: A GPU Performance Evaluation Data: x

648 views • 46 slides
Classification Based on Missing Features in Deep Convolutional Neural Networks Nemanja Milo

Classification Based on Missing Features in Deep Convolutional Neural Networks Nemanja Milo

Classification Based on Missing Features in Deep Convolutional Neural Networks Nemanja Milo sevi c UNSPMF EuroPython 2019 Nemanja Milo sevi c ( UNSPMF ) EuroPython 2019 1 / 19 Hello! Nemanja Milo sevi c PhD Student,

682 views • 19 slides
Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image

Convolutional Neural Networks (Part III) 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional

444 views • 27 slides
Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi

Convolutional Neural Networks in Speech Lecture 20 CS 753 Instructor: Preethi Jyothi Convolutional Neural Networks (CNNs) Fully connected (dense) layers have no awareness of spatial information Key concept behind convolutional layers is

983 views • 41 slides
Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Lecture 8: Convolutional Neural Nets Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier) https://courses.grainger.illinois.edu/cs447/ 1 Convolutional Neural Nets (ConvNets, CNNs) [4 parameters, applied 3

615 views • 24 slides
Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Separation and convexity properties of hierarchical and non hierarchical clustering Patrice

Plan Separation and convexity properties of hierarchical and non hierarchical clustering Patrice Bertrand 1 1 CEREMADE, Universit e Paris-Dauphine, Paris, France Joint work with Jean Diatta 2 2 LIM, Universit e de La R eunion,

401 views • 29 slides
Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing

Convolutional Neural Networks 08, 10 & 17 Nov, 2016 J. Ezequiel Soto S. Image Processing 2016 Prof. Luiz Velho Convolutional Neural Networks 1 Summary & References 08/11 ImageNet Classification with Deep Convolutional Neural Networks

648 views • 26 slides
HIERARCHICAL DESIGN IN Victor Khomenko, Danil Sokolov, Alex Yakovlev Why hierarchical? The

HIERARCHICAL DESIGN IN Victor Khomenko, Danil Sokolov, Alex Yakovlev Why hierarchical? The

HIERARCHICAL DESIGN IN Victor Khomenko, Danil Sokolov, Alex Yakovlev Why hierarchical? The only way to design large systems Requests from the engineers Natural development of Workcraft Automation of repetitive tasks by tracking

612 views • 13 slides
FeUdal Networks for Hierarchical Reinforcement Learning Alexander Sasha Vezhnevets, Simon

FeUdal Networks for Hierarchical Reinforcement Learning Alexander Sasha Vezhnevets, Simon

FeUdal Networks for Hierarchical Reinforcement Learning Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, Koray Kavukcuoglu Topic: Hierarchical RL Presenter: Thophile Gaudin Why Hierarchical

921 views • 22 slides
Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural

Semantic Segmentation of the sekleton in bone scintigraphy images with convolutional neural networks Problem Description Convolutional Neural Networks Convolutional Layer Max Pooling Transforming Classification networks to segmentation

157 views • 12 slides
Hierarchical Bounding Volume October 11, 2005 () Hierarchical Bounding Volume October 11, 2005

Hierarchical Bounding Volume October 11, 2005 () Hierarchical Bounding Volume October 11, 2005

Hierarchical Bounding Volume October 11, 2005 () Hierarchical Bounding Volume October 11, 2005 1 / 15 Outline Introduction to hierarchical bounding volume (HBV) Tree generation Other optimization issues () Hierarchical Bounding Volume

350 views • 16 slides
Lecture 6: Convolutional NN Princeton University COS 495 Instructor: Yingyu Liang Review:

Lecture 6: Convolutional NN Princeton University COS 495 Instructor: Yingyu Liang Review:

Deep Learning Basics Lecture 6: Convolutional NN Princeton University COS 495 Instructor: Yingyu Liang Review: convolutional layers Convolution: two dimensional case Input Kernel/filter a b c d w x e f g h y z i j k l wa + bx

847 views • 36 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides

More recommend