free 18-May-17 Towards Weakly Supervised Image Understanding 1/50
Towards Weakly Supervised Image Understanding (WSIU) 报告人：程明明 南开大学、计算机与控制工程学院 http://mmcheng.net/ 18-May-17 Towards Weakly Supervised Image Understanding 2/50
Understanding Visual Information Image by kirkh.deviantart.com 18-May-17 Towards Weakly Supervised Image Understanding 3/50
Dataset Annotation 18-May-17 Towards Weakly Supervised Image Understanding 4/50
Dataset Annotation • PASCAL 11: • 10? workers • 27.374 bounding boxes • ImageNet: • 25.000 workers • 11.231.732 images labeled with one word • My mother: • 213.841 segmented objects • Job offer: I am looking for more parents CVML 2012, Antonio Torralba 18-May-17 Towards Weakly Supervised Image Understanding 5/50
Dataset Annotation 18-May-17 Towards Weakly Supervised Image Understanding 6/50
Towards WSIU Editing Synthesis Web images Graphics/vision applications Semantic segmentation Interaction Light weighted semantic parsing Attention Boundary Segmentation Low level vision 18-May-17 Towards Weakly Supervised Image Understanding 7/50
Outline of the survey • Low level vision • Attention: CVPR 11, CVPR 14, TPAMI 15, IEEE TIP 15, IJCV 17, CVPR 17 • Boundary: CVPR 17 • Segmentation: TVC 14, CGF 15, ECCV 15, TPAMI 15 • Light weighted semantic parsing • Semantic segmentation: TPAMI 17, CVPR (oral) 17 • Interaction: TOG 15, TOG 10, TOG 12 • Graphics/vision applications • Editing: TOG 14 • Synthesis: TOG 09 • Web images: CVPR 12, TOG 11, CVPR 12, CVPR 13, TOG 12 18-May-17 Towards Weakly Supervised Image Understanding 8/50
Visual attention: motivation 18-May-17 Towards Weakly Supervised Image Understanding 9/50
Visual attention fixation Salient object Objectness prediction detection proposals Deeply supervised salient object detection with short connections, IEE EEE CVP CVPR 2017 2017. Salient Object Detection: A Discriminative Regional Feature Integration Approach, IJCV JCV 2017 2017. Salient Object Detection: A Benchmark, IEE EEE TIP TIP 2015 2015. Global Contrast based Salient Region Detection, IEE EEE TP TPAMI 2015 2015. BING: Binarized Normed Gradients for Objectness Estimation at 300fps, IEE EEE CVP CVPR 2014 2014. 18-May-17 Towards Weakly Supervised Image Understanding 10/50
Global Contrast based Salient Region Detection, IEEE TPAMI, 2014, MM Cheng, et. al. (2nd nd most cited pap paper in in CVP CVPR 2011 2011) 18-May-17 Towards Weakly Supervised Image Understanding 11/50
Core idea: region contrast (RC) 2 → ∞ 2 → 0.4 Image Segmentation 𝜏 𝑡 𝜏 𝑡 Spatial weighting Region size 𝐸 𝑡 𝑠 𝑙 ,𝑠 𝑗 𝑙 = 𝑠 𝑙 ≠𝑠 𝑗 exp − 𝑇 𝑠 𝜕 𝑠 𝑗 𝐸 𝑠 (𝑠 𝑙 , 𝑠 𝑗 ) 2 𝜏 𝑡 Region contrast by sparse histogram comparison. 18-May-17 Towards Weakly Supervised Image Understanding 12/50
Experimental results • Dataset: MSRA1000 [Achanta09] • Precision vs. recall 18-May-17 Towards Weakly Supervised Image Understanding 13/50
Supervised feature integration Salient Object Detection: A Discriminative Regional Feature Integration Approach, IJCV JCV 2017 2017. 18-May-17 Towards Weakly Supervised Image Understanding 14/50
Benchmarking 40+ methods Salient Object Detection: A Benchmark, IEE EEE TIP TIP 2015 2015. 18-May-17 Towards Weakly Supervised Image Understanding 15/50
Going with deep models Deeply supervised salient object detection with short connections, IEE EEE CVP CVPR 2017 2017. 18-May-17 Towards Weakly Supervised Image Understanding 16/50
Bridging between multi-levels 18-May-17 Towards Weakly Supervised Image Understanding 17/50
Messages from numbers 18-May-17 Towards Weakly Supervised Image Understanding 18/50
Methodology: observation • Objects are stand-alone things with well defined closed boundaries and centers. Finding pictures of objects in large collections of images. Springer Berlin Heidelberg, 1996, Forsyth et. al. Using stuff to find things. ECCV 2008, Heitz et. al. Measuring the objectness of image window, IEEE TPAMI 2012, Alexe et. al. • Little variations could present in such abstracted view. 18-May-17 Towards Weakly Supervised Image Understanding 19/50
Experimental results • Proposal quality on PASCAL VOC 2007 18-May-17 Towards Weakly Supervised Image Understanding 20/50
Experimental results • Computational time • A laptop with an Intel i7-3940XM CPU • 20 seconds for training on the PASCAL 2007 training set!! • Testing time 300fps on VOC 2007 images Method [1] OBN [2] CSVM [3] SEL [4] Our BING Time (seconds) 89.2 3.14 1.32 11.2 0.003 Category-Independent Object Proposals With Diverse Ranking, PAMI 2014, Endres et. al. Measuring the objectness of image windows. PAMI 2012, Alexe, et. al. Proposal Generation for Object Detection using Cascaded Ranking SVMs. CVPR 2011, Zhang et. al. Selective Search for Object Recognition, IJCV 2013, Uijlings et. al. 18-May-17 Towards Weakly Supervised Image Understanding 21/50
Towards WSIU Editing Synthesis Web images Graphics/vision applications Semantic segmentation Interaction Light weighted semantic parsing Attention Boundary Segmentation Low level vision 18-May-17 Towards Weakly Supervised Image Understanding 22/50
Boundary Richer Convolutional Features for Edge Detection, IEE EEE CVP CVPR 2017 2017. 18-May-17 Towards Weakly Supervised Image Understanding 23/50
18-May-17 Towards Weakly Supervised Image Understanding 24/50
Sate of the arts 18-May-17 Towards Weakly Supervised Image Understanding 25/50
Towards WSIU Editing Synthesis Web images Graphics/vision applications Semantic segmentation Interaction Light weighted semantic parsing Attention Boundary Segmentation Low level vision 18-May-17 Towards Weakly Supervised Image Understanding 26/50
SaliencyCut • Iterative refine: iteratively run GrabCut to refine segmentation • Adaptive fitting: adaptively fit with newly segmented salient region Enables automatic initialization provided by salient object detection. Global Contrast based Salient Region Detection, IEE EEE TP TPAMI 2015 2015. 18-May-17 Towards Weakly Supervised Image Understanding 27/50
Salient shape • Is salient object detection for ‘simple’ images useful? SalientShape: Group Saliency in Image Collections, TV TVC 2016 2016. 18-May-17 Towards Weakly Supervised Image Understanding 28/50
Segmentation DenseCut: Densely Connected CRFs for Realtime GrabCut, CGF GF 2015 2015. 18-May-17 Towards Weakly Supervised Image Understanding 29/50
Segmentation HFS: Hierarchical Feature Selection for Efficient Image Segmentation, ECC CCV 2015 2015. 18-May-17 Towards Weakly Supervised Image Understanding 30/50
Towards WSIU Editing Synthesis Web images Graphics/vision applications Semantic segmentation Interaction Light weighted semantic parsing Attention Boundary Segmentation Low level vision 18-May-17 Towards Weakly Supervised Image Understanding 31/50
STC 10% improvement over state of the art! STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation, arXiv 2015, Wei et al. 18-May-17 Towards Weakly Supervised Image Understanding 32/50
Interaction Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach, IEE EEE CVP CVPR (or (oral) al) 2017 2017. 18-May-17 Towards Weakly Supervised Image Understanding 33/50
Results 18-May-17 Towards Weakly Supervised Image Understanding 34/50
Towards WSIU Editing Synthesis Web images Graphics/vision applications Semantic segmentation Interaction Light weighted semantic parsing Attention Boundary Segmentation Low level vision 18-May-17 Towards Weakly Supervised Image Understanding 35/50
Motivation Objects Scene Pixels/patches Layering … 18-May-17 Towards Weakly Supervised Image Understanding 36/50
Motivation • Rough depth ordering is possible for a single image with repeated elements RepFinder: Finding Approximately Repeated Scene Elements for Image Editing, ACM CM TOG OG 2010 2010. 18-May-17 Towards Weakly Supervised Image Understanding 37/50
Image rearrangement Source image 18-May-17 Towards Weakly Supervised Image Understanding 38/50
18-May-17 Towards Weakly Supervised Image Understanding 39/50
18-May-17 Towards Weakly Supervised Image Understanding 40/50
Towards WSIU Editing Synthesis Web images Graphics/vision applications Semantic segmentation Interaction Light weighted semantic parsing Attention Boundary Segmentation Low level vision 18-May-17 Towards Weakly Supervised Image Understanding 41/50
ImageSpirit: Verbal Guided Image Parsing, ACM TOG, 2014 18-May-17 Towards Weakly Supervised Image Understanding 42/50
Motivations 18-May-17 Towards Weakly Supervised Image Understanding 43/50
Verbal guided image parsing Make the wood cabinet in bottom-middle lower nouns Adjective Verb/Adverb Object Attributes Commands Multi label CRF 18-May-17 Towards Weakly Supervised Image Understanding 44/50
Recommend
More recommend
