Semantic segmentation Image classification Object detection - PowerPoint PPT Presentation

Aug 13, 2022 •712 likes •857 views

Accel : A Corrective Fusion Network for Efficient Semantic Segmentation on Video Samvit Jain , Xin Wang , Joseph Gonzalez RISE Lab, UC Berkeley Semantic segmentation Image classification Object detection Semantic segmentation Evolution

Accel : A Corrective Fusion Network for Efficient Semantic Segmentation on Video Samvit Jain , Xin Wang , Joseph Gonzalez RISE Lab, UC Berkeley
Semantic segmentation Image classification Object detection Semantic segmentation
Evolution … Multi-Scale Aggregation by Efficient Graph-Based Fully Convolutional Dilated Convolutions Image Segmentation Networks for SS (2015) (2004) (2014) DeepLab-v2 PSPNet DeepLab-v3 (2016) (2017) (2017)
Evolution Fully Convolutional DeepLab-v3 Networks (2014) (2017) Dataset Pascal VOC 2012 Accuracy (mIoU) 62.2 85.7 Inference Time 175 ms 750 ms
Motivation ● Image models don’t translate to video ○ High frame rates (e.g. 30 fps) ○ High resolution (e.g. full-HD, 1920 x 1080 p) ○ Scene complexity (e.g. ego motion, urban streets) Cityscapes dataset : Frankfurt
Deep Feature Flow ● Idea: run feature net on keyframes , warp features to intermediate frames
Problems ● Accuracy degradation ○ Warping with a flow field is a coarse operation ○ Non-translational temporal change (e.g. new objects, occlusions, lighting) ignored (a) k (b) k+2 (c) k+4 (d) k+6
Accel ResNet-101 keyframe N R I k feat ... W optical flow reference ... branch score fusion N R optical flow W ... task S k+i warp SF N U I k+i N U feat task update branch segmentation current frame ResNet-{18,34,51,101} Accel : a family of corrective, two-stream fusion networks combining: N R ( reference branch ) – optical flow-based keyframe feature warping (1) N U ( update branch ) – per-frame correction with residual segmentation network (2)
Accel N R + N U N Rfeat N Ufeat (reference branch) (update branch) (full network) ResNet-101 ResNet-18 Accel-18 ResNet-101 ResNet-34 Accel-34 ResNet-101 ResNet-51 Accel-51 ResNet-101 ResNet-101 Accel-101
Results Cityscapes CamVid Accuracy (mIoU) vs. inference time (s/frame)
Results Accuracy (mIoU) vs. keyframe interval
Visualizations DFF (reference branch) DeepLab-18 (update branch) Accel-18
Thank you! Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Video S. Jain, X. Wang, J. Gonzalez In: CVPR 2019 (oral) https://arxiv.org/abs/1807.06667

Recommend

Pixel-Level Im Image Understanding wit ith Semantic Segmentation and Panoptic Segmentation

Pixel-Level Im Image Understanding wit ith Semantic Segmentation and Panoptic Segmentation Hengshuang Zhao The Chinese University of Hong Kong May 29, 2019 Part I: I: Semantic Segmentation Semantic Segmentation background car person

510 views • 39 slides

Segmentation Bottom-up Segmentation Semantic / instance segmentation Many Slides from L.

Segmentation Bottom-up Segmentation Semantic / instance segmentation Many Slides from L. Lazebnik. Outline Bottom-up segmentation Superpixel segmentation Semantic segmentation Metrics Architectures

1.28k views • 49 slides

Semantic Segmentation / Instance Segmentation Based on Deep learning Yiding Liu 2018.12.08

Semantic Segmentation / Instance Segmentation Based on Deep learning Yiding Liu 2018.12.08 Outline Overview of segmentation problem Semantic segmentation Instance Segmentation Our work Definition of segmentation problem Image

900 views • 54 slides

Context For Semantic Segmentation Gang Yu Collaborators Changqian Yu

Context For Semantic Segmentation Gang Yu Collaborators Changqian Yu Jingbo Wang Chao Peng Xiangyu Zhang Changxin Gao Nong Sang Gang Yu Jian Sun Outline Revisit Semantic Segmentation Context for Semantic

774 views • 43 slides

Learning Deep Structured Models for Semantic Segmentation Guosheng Lin Semantic Segmentation

Learning Deep Structured Models for Semantic Segmentation Guosheng Lin Semantic Segmentation Outline Exploring Context with Deep Structured Models Guosheng Lin, Chunhua Shen, Ian Reid, Anton van dan Hengel; Efficient Piecewise Training

992 views • 45 slides

An Overview of Semantic Image Segmentation with Deep Learning Simone Bonechi Outline

An Overview of Semantic Image Segmentation with Deep Learning Simone Bonechi Outline Semantic Image Segmentation Deep Network for Semantic Segmentation FCN (Fully Convolutional Neural Network) DeconvNet PSPNet (Pyramid Scene

746 views • 22 slides

Budget-aware Semi-Supervised Semantic and Instance Segmentation Miriam Bellver, Amaia Salvador,

Budget-aware Semi-Supervised Semantic and Instance Segmentation Miriam Bellver, Amaia Salvador, Jordi Torres, Xavier Giro-i-Nieto Women In Computer Vision - CVPR 2019 Motivation Semantic segmentation Instance segmentation Pixel-level

282 views • 15 slides

LID Challenge: Weakly Supervised Semantic Segmentation 3d place solution NoPeopleAllowed: The 3

LID Challenge: Weakly Supervised Semantic Segmentation 3d place solution NoPeopleAllowed: The 3 step approach to weakly supervised semantic segmentation Mariia Dobko, Ostap Viniavskyi, Oles Dobosevych UCU & SoftServe team The Machine

771 views • 20 slides

A new metric for evaluating semantic segmentation: leveraging global and contour accuracy Eduardo

Introduction Semantic segmentation Accuracy evaluation Conclusions A new metric for evaluating semantic segmentation: leveraging global and contour accuracy Eduardo Fernandez-Moral 1 , Renato Martins 1 , Denis Wolf 2 , and Patrick Rives 1 1

786 views • 40 slides

Temporally Distributed Networks for Fast Video Semantic Segmentation Ping Hu 1 Fabian Caba

Temporally Distributed Networks for Fast Video Semantic Segmentation Ping Hu 1 Fabian Caba Heilbron 2 Oliver Wang 2 Zhe Lin 2 Stan Sclaroff 1 Federico Perazzi 2 1 Boston University 2 Adobe Research Challenge Video Semantic Segmentation frame

984 views • 11 slides

Semantic Image Segmentation and Web-Supervised Visual Learning Florian Schroff Andrew Zisserman

Semantic Image Segmentation and Web-Supervised Visual Learning Florian Schroff Andrew Zisserman University of Oxford, UK Antonio Criminisi Microsoft Research Ltd, Cambridge, UK Outline Part I: Semantic Image Segmentation Goal:

794 views • 63 slides

Learning Deconvolution Network for Semantic Segmentation Hyeonwoo Noh, Seunghoon Hong, Bohyung

Learning Deconvolution Network for Semantic Segmentation Hyeonwoo Noh, Seunghoon Hong, Bohyung Han Mehmet Gnel What is this paper about? A novel semantic segmentation algorithm Convolution & Deconvolution layers Fully

1.03k views • 37 slides

Lecture: Segmentation Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab

Semantic Segmentation Lecture: Segmentation Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 17-Oct-2019 1 St Stanfor ord University CS 131 Roadmap Semantic Segmentation Pixels Segments Images Videos Web Neural

744 views • 59 slides

Deep learning 8.4. Networks for semantic segmentation Fran cois Fleuret

Deep learning 8.4. Networks for semantic segmentation Fran cois Fleuret https://fleuret.org/ee559/ Nov 2, 2020 The historical approach to image segmentation was to define a measure of similarity between pixels, and to cluster groups of

222 views • 18 slides

SEMANTIC IMAGE SEGMENTATION WITH DEEP CONVOLUTIONAL NETS AND FULLY CONNECTED CRFS Paper by Chen,

SEMANTIC IMAGE SEGMENTATION WITH DEEP CONVOLUTIONAL NETS AND FULLY CONNECTED CRFS Paper by Chen, Papandreou, Kokkinos, Murphy, Yuille Slides by Josh Kelle (with graphics from the paper) Semantic Segmentation Goal: Partition the image into

520 views • 22 slides

Deep Watershed Transform for Instance Segmentation Min Bai & Raquel Urtasun To appear at

Deep Watershed Transform for Instance Segmentation Min Bai & Raquel Urtasun To appear at IEEE CVPR 2017 in Hawaii Presented at NVIDIA GTC 2017 Semantic Segmentation Input: RGB Image Output at each pixel: Semantic label

465 views • 28 slides

Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds Francis Engelmann*

Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds Francis Engelmann* Theodora Kontogianni* Alexander Hermans Bastian Leibe Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds Problem Statement Input

140 views • 10 slides

GCN INTRODUCTION AND ITS APPLICATION IN 3D POINT CLOUD SEMANTIC SEGMENTATION Yisong Li (NVIDIA),

GCN INTRODUCTION AND ITS APPLICATION IN 3D POINT CLOUD SEMANTIC SEGMENTATION Yisong Li (NVIDIA), Guohao Li (KAUST) Grid Data vs General Graphs CNN vs GCN ResGCN OUTLINE Experiments on 3D Cloud Point Segmentation Sequential

1.31k views • 102 slides

Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Je

Rich feature hierarchies for accurate object detection and semantic segmentation Ross Girshick, Je ff Donahue, Trevor Darrell, Jitendra Malik UC Berkeley Tech Report @ http://arxiv.org/abs/1311.2524 Detection & Segmentation input

704 views • 37 slides

Image Segmentation Machine Learning Study Group Presented by Yaochen Xie Jan 25, 2018 Outline

Image Segmentation Machine Learning Study Group Presented by Yaochen Xie Jan 25, 2018 Outline Overview Three Levels of Segmentation Basic Segmentation Semantic Segmentation (FCN, DeepLab) Instance Segmentation (Mask-RCNN)

449 views • 28 slides

and Background for Semantic Segmentation Yu Liu and Michael S. Lew Leiden Institute of Advanced

IEEE International Conference on Image Processing (ICIP 2017), Beijing, China Improving the Discrimination Between Foreground and Background for Semantic Segmentation Yu Liu and Michael S. Lew Leiden Institute of Advanced Computer Science,

470 views • 34 slides

Efficient Semantic Segmentation using Gradual Grouping Nikitha Vallurupalli 1 , Sriharsha

Efficient Semantic Segmentation using Gradual Grouping Nikitha Vallurupalli 1 , Sriharsha Annamaneni 1 , Girish Varma 1 , C V Jawahar 1 , Manu Mathew 2 , Soyeb Nagori 2 IIIT Hyderabad 1 , TI Bangalore 2 Image / annotation from cityscapes dataset

309 views • 13 slides

Optimizing the Relevance-Redundancy Tradeoff for Efficient Semantic Segmentation Caner Hazrba

Optimizing the Relevance-Redundancy Tradeoff for Efficient Semantic Segmentation Caner Hazrba Joint work with Julia Diebold and Daniel Cremers Optimizing the Relevance-Redundancy Tradeoff for Efficient Semantic Segmentation Caner Hazrba

794 views • 21 slides

Semantic (less) Motion and Video Segmentation Ren Vidal Johns Hopkins University Talk

Semantic (less) Motion and Video Segmentation Ren Vidal Johns Hopkins University Talk Outline Semantic-less Motion Segmentation (Vidal et al., ECCV02, IJCV06; Vidal, Ma and Sastry CVPR03, PAMI05; Vidal and Sastry CVPR03; Vidal and

885 views • 31 slides