lecture segmentation
play

Lecture: Segmentation Juan Carlos Niebles and Ranjay Krishna - PowerPoint PPT Presentation

Sep 26, 2023 •147 likes •744 views

Semantic Segmentation Lecture: Segmentation Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 17-Oct-2019 1 St Stanfor ord University CS 131 Roadmap Semantic Segmentation Pixels Segments Images Videos Web Neural

  1. Semantic Segmentation Lecture: Segmentation Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 17-Oct-2019 1 St Stanfor ord University

  2. CS 131 Roadmap Semantic Segmentation Pixels Segments Images Videos Web Neural networks Convolutions Recognition Resizing Motion Convolutional Edges Detection Segmentation Tracking neural networks Descriptors Machine learning Clustering 17-Oct-2019 2 St Stanfor ord University

  3. What we will learn today • Introduction to segmentation and clustering Semantic Segmentation • Gestalt theory for perceptual grouping • Agglomerative clustering • Oversegmentation 17-Oct-2019 3 Reading: [FP] Chapters: 14.2, 14.4 St Stanfor ord University

  4. Semantic Segmentation 17-Oct-2019 4 ord University Stanfor St

  5. Image Segmentation • Goal: identify groups of pixels that go together Semantic Segmentation 17-Oct-2019 5 St Stanfor ord University Slide credit: Steve Seitz, Kristen Grauman

  6. The Goals of Segmentation • Separate image into coherent “objects” Semantic Segmentation Image Human segmentation 17-Oct-2019 6 St Stanfor ord University Slide credit: Svetlana Lazebnik

  7. The Goals of Segmentation • Separate image into coherent “objects” Semantic Segmentation • Group together similar-looking pixels for efficiency of further processing Slide credit: Svetlana Lazebnik 17-Oct-2019 “superpixels” 7 X. Ren and J. Malik. Learning a classification model for segmentation. ICCV 2003. St Stanfor ord University

  8. Segmentation for feature support Semantic Segmentation 50x50 Patch 50x50 Patch 17-Oct-2019 8 St Stanfor ord University Slide: Derek Hoiem

  9. Segmentation for efficiency Semantic Segmentation [Felzenszwalb and Huttenlocher 2004] 17-Oct-2019 [Hoiem et al. 2005, Mori 2005] [Shi and Malik 2001] 9 St Stanfor ord University Slide: Derek Hoiem

  10. Segmentation as a result Semantic Segmentation 17-Oct-2019 10 Rother et al. 2004 St Stanfor ord University

  11. Types of segmentations Semantic Segmentation Oversegmentation Undersegmentation 17-Oct-2019 Multiple Segmentations 11 St Stanfor ord University

  12. One way to think about “segmentation” is Clustering Semantic Segmentation Clustering: group together similar data points and represent them with a single token Key Challenges: 1) What makes two points/images/patches similar? 17-Oct-2019 2) How do we compute an overall grouping from pairwise similarities? 12 St Stanfor ord University Slide: Derek Hoiem

  13. Why do we cluster? • Summarizing data – Look at large amounts of data Semantic Segmentation – Patch-based compression or denoising – Represent a large continuous vector with the cluster number • Counting – Histograms of texture, color, SIFT vectors • Segmentation 17-Oct-2019 – Separate the image into different regions • Prediction – Images in the same cluster may have the same labels 13 Stanfor St ord University Slide: Derek Hoiem

  14. How do we cluster? • Agglomerative clustering Semantic Segmentation – Start with each point as its own cluster and iteratively merge the closest clusters • K-means (next lecture) – Iteratively re-assign points to the nearest cluster center • Mean-shift clustering (next lecture) – Estimate modes of pdf 17-Oct-2019 14 St Stanfor ord University

  15. General ideas • Tokens Semantic Segmentation – whatever we need to group (pixels, points, surface elements, etc., etc.) • Bottom up clustering – tokens belong together because they are locally coherent • Top down clustering – tokens belong together because they lie on the same visual entity 17-Oct-2019 (object, scene…) > These two are not mutually exclusive 15 St Stanfor ord University

  16. Examples of Grouping in Vision Semantic Segmentation Grouping video frames into shots Determining image regions 17-Oct-2019 What things should Figure-ground be grouped? What cues indicate groups? Object-level grouping 16 St Stanfor ord University Slide credit: Kristen Grauman

  17. Similarity Semantic Segmentation 17-Oct-2019 17 St Stanfor ord University Slide credit: Kristen Grauman

  18. Symmetry Semantic Segmentation 17-Oct-2019 18 Slide credit: Kristen Grauman St Stanfor ord University

  19. Common Fate Semantic Segmentation 17-Oct-2019 Image credit: Arthus-Bertrand (via F. Durand) 19 Slide credit: Kristen Grauman St Stanfor ord University

  20. Proximity Semantic Segmentation 17-Oct-2019 Slide credit: Kristen Grauman 20 St Stanfor ord University

  21. Muller-Lyer Illusion Semantic Segmentation • What makes the bottom line look longer than the top line? 17-Oct-2019 21 St Stanfor ord University

  22. What we will learn today • Introduction to segmentation and clustering Semantic Segmentation • Gestalt theory for perceptual grouping • Agglomerative clustering • Oversegmentation 17-Oct-2019 22 St Stanfor ord University

  23. The Gestalt School • Grouping is key to visual perception • Elements in a collection can have properties that Semantic Segmentation result from relationships – “The whole is greater than the sum of its parts” Illusory/subjective Occlusion contours Slide credit: Svetlana Lazebnik 17-Oct-2019 Familiar configuration 23 http://en.wikipedia.org/wiki/Gestalt_psychology Stanfor St ord University

  24. Gestalt Theory • Gestalt: whole or group Semantic Segmentation – Whole is greater than sum of its parts – Relationships among parts can yield new properties/features • Psychologists identified series of factors that predispose set of elements to be grouped (by human visual system) “I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees.” 17-Oct-2019 Max Wertheimer (1880-1943) Untersuchungen zur Lehre von der Gestalt, Psychologische Forschung , Vol. 4, pp. 301-350, 1923 http://psy.ed.asu.edu/~classics/Wertheimer/Forms/forms.htm 24 Stanfor St ord University

  25. Gestalt Factors Semantic Segmentation 17-Oct-2019 Image source: Forsyth & Ponce • These factors make intuitive sense, but are very difficult to translate into algorithms. 25 St Stanfor ord University

  26. Continuity through Occlusion Cues Semantic Segmentation 17-Oct-2019 26 St Stanfor ord University

  27. Continuity through Occlusion Cues Semantic Segmentation 17-Oct-2019 Continuity, explanation by occlusion 27 St Stanfor ord University

  28. Continuity through Occlusion Cues Semantic Segmentation Image source: Forsyth & Ponce 17-Oct-2019 28 St Stanfor ord University

  29. Continuity through Occlusion Cues Semantic Segmentation Image source: Forsyth & Ponce 17-Oct-2019 29 St Stanfor ord University

  30. Figure-Ground Discrimination Semantic Segmentation 17-Oct-2019 30 St Stanfor ord University

  31. The Ultimate Gestalt? Semantic Segmentation 17-Oct-2019 31 St Stanfor ord University

  32. What we will learn today • Introduction to segmentation and clustering Semantic Segmentation • Gestalt theory for perceptual grouping • Agglomerative clustering • Oversegmentation 17-Oct-2019 32 St Stanfor ord University

  33. Clustering: distance measure Clustering is an unsupervised learning method. Given items Semantic Segmentation , the goal is to group them into clusters. We need a pairwise distance/similarity function between items, and sometimes the desired number of clusters. When data (e.g. images, objects, documents) are represented by feature vectors, commonly used measures are: - Euclidean distance. 17-Oct-2019 - cosine similarity. 34 St Stanfor ord University

  34. Defining Distance Measures Let x and x’ be two objects from the universe of possible objects. Semantic Segmentation The distance (similarity) between x and x’ is a real number denoted by sim(x, x’). • The Euclidean distance is defined as 𝑒𝑗𝑡𝑢 𝑦, 𝑦 ' = ∑ 𝑦 * − 𝑦′ * - • In contrast, the cosine similarity measure would be 17-Oct-2019 35 St Stanfor ord University

  35. Desirable Properties of a Clustering Algorithms • Scalability (in terms of both time and space) Semantic Segmentation • Ability to deal with different data types • Minimal requirements for domain knowledge to determine input parameters • Interpretability and usability Optional – Incorporation of user-specified constraints 17-Oct-2019 36 St Stanfor ord University

  36. Animated example Semantic Segmentation 17-Oct-2019 source 37 St Stanfor ord University

  37. Animated example Semantic Segmentation 17-Oct-2019 source 38 St Stanfor ord University

  38. Animated example Semantic Segmentation 17-Oct-2019 source 39 St Stanfor ord University

  39. Agglomerative clustering Semantic Segmentation 17-Oct-2019 Slide credit: Andrew Moore 40 St Stanfor ord University

  40. Agglomerative clustering Semantic Segmentation 17-Oct-2019 Slide credit: Andrew Moore 41 St Stanfor ord University

  41. Agglomerative clustering Semantic Segmentation 17-Oct-2019 Slide credit: Andrew Moore 42 St Stanfor ord University

  42. Agglomerative clustering Semantic Segmentation 17-Oct-2019 Slide credit: Andrew Moore 43 St Stanfor ord University

Recommend

Segmentation Professor Fei Fei Li Stanford Vision Lab 1 19 Apr 11 Lecture 8 -

Segmentation Professor Fei Fei Li Stanford Vision Lab 1 19 Apr 11 Lecture 8 -

Segmentation Professor Fei Fei Li Stanford Vision Lab 1 19 Apr 11 Lecture 8 - Fei-Fei Li Image Segmentation Goal: identify groups of pixels that go together 2 19 Apr 11 Lecture 8 - Fei-Fei Li Success Story 3 19 Apr

1.12k views • 64 slides
Segmentation Segmentation Segmentation Define the accurate boundaries of all objects in an image

Segmentation Segmentation Segmentation Define the accurate boundaries of all objects in an image

Day 4 Lecture 2 Segmentation Segmentation Segmentation Define the accurate boundaries of all objects in an image Segmentation: Datasets Pascal Visual Object Classes Microsoft COCO 20 Classes 80 Classes ~ 5.000 images ~ 300.000 images

995 views • 26 slides
Lecture 8: Image Segmentation Peng Chao Face++ Researcher pengchao@megvii.com Nov. 2017

Lecture 8: Image Segmentation Peng Chao Face++ Researcher pengchao@megvii.com Nov. 2017

Lecture 8: Image Segmentation Peng Chao Face++ Researcher pengchao@megvii.com Nov. 2017 Image Segmentation Semantic Segmentation Instance Segmentation Scene Parsing Human Parsing Stuff Segmentation New Track in COCO 2017

1.04k views • 103 slides
VIDEO SIGNALS Segmentation WHAT IS SEGMENTATION WHAT IS SEGMENTATION Segmentation is a

VIDEO SIGNALS Segmentation WHAT IS SEGMENTATION WHAT IS SEGMENTATION Segmentation is a

VIDEO SIGNALS Segmentation WHAT IS SEGMENTATION WHAT IS SEGMENTATION Segmentation is a fundamental low level operation on Segmentation is a fundamental low-level operation on images. If an image is already partitioned into segments

639 views • 51 slides
Segmentation Bottom-up Segmentation Semantic / instance segmentation Many Slides from L.

Segmentation Bottom-up Segmentation Semantic / instance segmentation Many Slides from L.

Segmentation Bottom-up Segmentation Semantic / instance segmentation Many Slides from L. Lazebnik. Outline Bottom-up segmentation Superpixel segmentation Semantic segmentation Metrics Architectures

1.28k views • 49 slides
1 Mean Shift Algorithm Graph-Theoretic Image Segmentation Mean Shift Algorithm 1. Choose a

1 Mean Shift Algorithm Graph-Theoretic Image Segmentation Mean Shift Algorithm 1. Choose a

6.891 Last time: Segmentation and Clustering (Ch. 14) Computer Vision and Applications Supervised->Unsupervised Category Learning needs segmentation Prof. Trevor. Darrell K-Means Mean Shift Graph cuts Lecture 15:

531 views • 19 slides
Lecture 2: Introduction to Segmentation Jonathan Krause Fei-Fei Li, Jonathan Krause Lecture 2 -

Lecture 2: Introduction to Segmentation Jonathan Krause Fei-Fei Li, Jonathan Krause Lecture 2 -

Lecture 2: Introduction to Segmentation Jonathan Krause Fei-Fei Li, Jonathan Krause Lecture 2 - 1 Goal Goal: Identify groups of pixels that go together image credit: Steve Seitz, Kristen Grauman Fei-Fei Li, Jonathan Krause Lecture 2 - 2

1.12k views • 70 slides
Advanced 3D segmentation Sigmund Rolfsjord Todays lecture Different ways to work with 3D

Advanced 3D segmentation Sigmund Rolfsjord Todays lecture Different ways to work with 3D

Advanced 3D segmentation Sigmund Rolfsjord Todays lecture Different ways to work with 3D data: Curriculum: - Point clouds - Grids SEGCloud: Semantic Segmentation of 3D Point Clouds - Graphs Multi-view Convolutional Neural Networks for

1.07k views • 70 slides
Image Analysis The topics of this lecture are Segmentation Thresholding Region based

Image Analysis The topics of this lecture are Segmentation Thresholding Region based

Image Analysis The topics of this lecture are Segmentation Thresholding Region based segmentation Christina Olsn colsen@cs.umu.se The Watershed-algorithm Department of Computing Science Ume University February 17, 2009 Christina Olsn

372 views • 12 slides
Segmentation 2014-11-14 Robin Strand Centre for Image Analysis Dept. of IT Uppsala University

Segmentation 2014-11-14 Robin Strand Centre for Image Analysis Dept. of IT Uppsala University

Lecture 6: Segmentation 2014-11-14 Robin Strand Centre for Image Analysis Dept. of IT Uppsala University Today What is image segmentation? A smrgsbord of methods for image segmentation: Thresholding Edge-based segmentation

830 views • 51 slides
Pixel-Level Im Image Understanding wit ith Semantic Segmentation and Panoptic Segmentation

Pixel-Level Im Image Understanding wit ith Semantic Segmentation and Panoptic Segmentation

Pixel-Level Im Image Understanding wit ith Semantic Segmentation and Panoptic Segmentation Hengshuang Zhao The Chinese University of Hong Kong May 29, 2019 Part I: I: Semantic Segmentation Semantic Segmentation background car person

510 views • 39 slides
Semantic Segmentation / Instance Segmentation Based on Deep learning Yiding Liu 2018.12.08

Semantic Segmentation / Instance Segmentation Based on Deep learning Yiding Liu 2018.12.08

Semantic Segmentation / Instance Segmentation Based on Deep learning Yiding Liu 2018.12.08 Outline Overview of segmentation problem Semantic segmentation Instance Segmentation Our work Definition of segmentation problem Image

900 views • 54 slides
Segmentation using Segmentation using Bayesian Decision Theory Bayesian Decision Theory

Segmentation using Segmentation using Bayesian Decision Theory Bayesian Decision Theory

Segmentation Segmentation Segmentation using Segmentation using Bayesian Decision Theory Bayesian Decision Theory Segmentation of images is the separation of pixels into different categories depending upon their intensities and/or other

109 views • 8 slides
Image Segmentation using Seg3D Segmentation From Clinical Scans RA RA LA RV LA LV RV LV

Image Segmentation using Seg3D Segmentation From Clinical Scans RA RA LA RV LA LV RV LV

Image Segmentation using Seg3D Segmentation From Clinical Scans RA RA LA RV LA LV RV LV Tools Layers Viewers Final Segmenting Original Scan Segmentation using Seg3D 2 3 Tools Menu Interactive tools for operating on volume

448 views • 10 slides
Image Segmentation Machine Learning Study Group Presented by Yaochen Xie Jan 25, 2018 Outline

Image Segmentation Machine Learning Study Group Presented by Yaochen Xie Jan 25, 2018 Outline

Image Segmentation Machine Learning Study Group Presented by Yaochen Xie Jan 25, 2018 Outline Overview Three Levels of Segmentation Basic Segmentation Semantic Segmentation (FCN, DeepLab) Instance Segmentation (Mask-RCNN)

449 views • 28 slides
Segmentation Shao-Yi Chien Department of Electrical Engineering National Taiwan

Segmentation Shao-Yi Chien Department of Electrical Engineering National Taiwan

Segmentation Shao-Yi Chien Department of Electrical Engineering National Taiwan University Fall 2018 1 Outline Segmentation Image segmentation Object selection with interactive segmentation Super-pixel methods

1.23k views • 87 slides
Segmentation & Grouping Kristen Grauman UT Austin Tues Feb 7 Announcements A0 on

Segmentation & Grouping Kristen Grauman UT Austin Tues Feb 7 Announcements A0 on

2/6/2017 Segmentation & Grouping Kristen Grauman UT Austin Tues Feb 7 Announcements A0 on Canvas No office hours today TA office hours this week as usual Guest lecture Thursday by Suyog Jain Interactive segmentation

604 views • 45 slides
6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 14: Unsupervised

6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 14: Unsupervised

6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 14: Unsupervised Category Learning Gestalt Principles Segmentation by Clustering K-Means Graph cuts Segmentation by Fitting Hough transform

1.14k views • 112 slides
Semantic segmentation Image classification Object detection Semantic segmentation Evolution

Semantic segmentation Image classification Object detection Semantic segmentation Evolution

Accel : A Corrective Fusion Network for Efficient Semantic Segmentation on Video Samvit Jain , Xin Wang , Joseph Gonzalez RISE Lab, UC Berkeley Semantic segmentation Image classification Object detection Semantic segmentation Evolution

857 views • 13 slides
LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

DATA MINING LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

998 views • 78 slides
Lecture 2 Annotation tools & Segmentation Summary of Part 1 Annotation theory

Lecture 2 Annotation tools & Segmentation Summary of Part 1 Annotation theory

Center for Reflected Text Analytics Lecture 2 Annotation tools & Segmentation Summary of Part 1 Annotation theory Guidelines Inter-Annotator agreement Inter-subjective annotations Annotation exercise Discuss

671 views • 22 slides
Image Segmentation Image Segmentation: Definitions How do we know which groups of pixels in a

Image Segmentation Image Segmentation: Definitions How do we know which groups of pixels in a

Image Segmentation Image Segmentation: Definitions How do we know which groups of pixels in a Segmentation is the process of partitioning an digital image correspond to the objects to be image into semantically interpretable

766 views • 24 slides
6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 15: Fitting and

6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 15: Fitting and

6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 15: Fitting and Segmentation Readings: F&P Ch 15.3-15.5,16 1 Last time: Segmentation and Clustering (Ch. 14) Supervised->Unsupervised Category Learning

1.41k views • 111 slides
Algorithms for Geographic Data Spring 2016 Lecture 4: Segmentation Motivation: Geese Migration

Algorithms for Geographic Data Spring 2016 Lecture 4: Segmentation Motivation: Geese Migration

2IMA20 Algorithms for Geographic Data Spring 2016 Lecture 4: Segmentation Motivation: Geese Migration Two behavioural types stopover migration flight Input: GPS tracks expert description of behaviour Goal: Delineate

1.01k views • 53 slides

More recommend