CS 376: Computer Vision - lecture 10 2/20/2018 Last time • Fitting an arbitrary shape with “active” deformable contours Segmentation & Grouping Kristen Grauman, UT Austin Last time: deformable contours Review questions • How does Hough fitting and deformable • Representation of the contours contour fitting differ? How are they alike? • Defining the energy functions – External • What is the influence of the number of the – Internal vertices in an active contour? • Minimizing the energy function • What is the influence in the number of • Extensions: candidate states (m) when fitting the active – Tracking contour with DP? – Interactive segmentation 3D active contours Tracking via deformable contours 1. Use final contour/model extracted at frame t as an initial solution for frame t+1 2. Evolve initial contour to fit exact object boundary http://www.cvl.isy.liu.se/ScOut/Masters/Papers/Ex1708.pdf at frame t+1 3. Repeat, initializing with most recent frame. Tracking Heart Ventricles (multiple frames) Jörgen Ahlberg 1
CS 376: Computer Vision - lecture 10 2/20/2018 Limitations Limitations • External energy: snake does not really “see” object • May over-smooth the boundary boundaries in the image unless it gets very close to it. • Cannot follow topological changes of objects I image gradients are large only directly on the boundary Interactive forces Distance transform • External image can instead be taken from the distance transform of the edge image. original -gradient distance transform How can we implement such an interactive Value at (x,y) tells how far that position is from the force with deformable contours? nearest edge point (or other binary mage structure) >> help bwdist Slide credit: Kristen Grauman edges Slide credit: Kristen Grauman Interactive forces Intelligent scissors • An energy function can be altered online based Another form of on user input – use the cursor to push or pull the interactive initial snake away from a point. segmentation: • Modify external energy term to include: Compute optimal paths from every point to 2 n 1 r the seed based on E push 2 | p | edge-related costs. i 0 i Nearby points get pushed hardest VIDEO [Mortensen & Barrett, SIGGRAPH 1995, CVPR 1999] 2
CS 376: Computer Vision - lecture 10 2/20/2018 Intelligent scissors Intelligent scissors • http://rivit.cs.byu.edu/Eric/Eric.html • http://rivit.cs.byu.edu/Eric/Eric.html Recap: deformable contours Deformable contours: pros and cons • Deformable shapes and active contours are useful for Pros: – Segmentation: fit or “snap” to boundary in image • Useful to track and fit non-rigid shapes – Tracking: previous frame’s estimate serves to initialize the next • Contour remains connected • Fitting active contours: • Possible to fill in “subjective” contours – Define terms to encourage certain shapes, smoothness, low • Flexibility in how energy function is defined, weighted. curvature, push/pulls, … Cons: – Use weights to control relative influence of each component cost • Must have decent initialization near true boundary, may – Can optimize 2d snakes with Viterbi algorithm. get stuck in local minimum • Image structure (esp. gradients) can act as attraction • Parameters of energy function must be set well based on force for interactive segmentation methods. prior information Slide credit: Kristen Grauman Grouping in vision Outline • Goals: • What are grouping problems in vision? – Gather features that belong together – Obtain an intermediate representation that compactly • Inspiration from human perception describes key image or video parts – Gestalt properties • Bottom-up segmentation via clustering – Algorithms: • Mode finding and mean shift: k-means, mean-shift • Graph-based: normalized cuts – Features: color, texture, … • Quantization for texture summaries 3
CS 376: Computer Vision - lecture 10 2/20/2018 Examples of grouping in vision Grouping in vision • Goals: – Gather features that belong together – Obtain an intermediate representation that compactly describes key image (video) parts [http://poseidon.csd.auth.gr/LAB_RESEARCH/Latest/imgs/S peakDepVidIndex_img2.jpg] • Top down vs. bottom up segmentation Group video frames into shots [Figure by J. Shi] – Top down: pixels belong together because they are Determine image regions Fg / Bg from the same object – Bottom up: pixels belong together because they look [Figure by Wang & Suter] Figure-ground similar • Hard to measure success – What is interesting depends on the app. [Figure by Grauman & Darrell] Object-level grouping Slide credit: Kristen Grauman Muller-Lyer illusion What are meta-cues for grouping? Gestalt • Gestalt: whole or group – Whole is greater than sum of its parts What things should be grouped? – Relationships among parts can yield new properties/features What cues indicate groups? • Psychologists identified series of factors that predispose set of elements to be grouped (by human visual system) 4
CS 376: Computer Vision - lecture 10 2/20/2018 Similarity Symmetry Slide credit: Kristen Grauman Slide credit: Kristen Grauman http://chicagoist.com/attachments/chicagoist_alicia/GEESE.jpg, http://wwwdelivery.superstock.com/WI/223/1532/PreviewComp/SuperStock_1532R-0831.jpg http://seedmagazine.com/news/2006/10/beauty_is_in_the_processingtim.php Common fate Proximity Image credit: Arthus-Bertrand (via F. Durand) Slide credit: Kristen Grauman http://www.capital.edu/Resources/Images/outside6_035.jpg Illusory/subjective contours Interesting tendency to explain by occlusion In Vision , D. Marr, 1982 5
CS 376: Computer Vision - lecture 10 2/20/2018 Continuity, explanation by occlusion D. Forsyth Continuity, explanation by occlusion Slide credit: Kristen Grauman Figure-ground http://entertainthis.usatoday.com/2015/09/09/how-tom-hardys-legend- poster-hid-this-hilariously-bad-review/ Slide credit: Kristen Grauman 6
CS 376: Computer Vision - lecture 10 2/20/2018 In Vision , D. Marr, 1982; from J. L. Marroquin, “Human visual perception of structure”, 1976. Grouping phenomena in real life Grouping phenomena in real life Forsyth & Ponce, Figure 14.7 Forsyth & Ponce, Figure 14.7 Gestalt Outline • What are grouping problems in vision? • Gestalt: whole or group – Whole is greater than sum of its parts • Inspiration from human perception – Relationships among parts can yield new properties/features – Gestalt properties • Psychologists identified series of factors that • Bottom-up segmentation via clustering predispose set of elements to be grouped (by – Algorithms: human visual system) • Mode finding and mean shift: k-means, EM, mean-shift • Graph-based: normalized cuts • Inspiring observations/explanations; challenge – Features: color, texture, … remains how to best map to algorithms. • Quantization for texture summaries 7
CS 376: Computer Vision - lecture 10 2/20/2018 The goals of segmentation The goals of segmentation Separate image into coherent “objects” Separate image into coherent “objects” image human segmentation Group together similar-looking pixels for efficiency of further processing “superpixels” X. Ren and J. Malik. Learning a classification model for segmentation. ICCV 2003. Source: Lana Lazebnik Source: Lana Lazebnik Image segmentation: toy example Clustering white pixels 3 pixel count black pixels gray Clustering algorithms: 2 1 pixels Unsupervised learning Detect patterns in unlabeled data E.g. group emails or search results input image E.g. find categories of customers E.g. group pixels into regions intensity • These intensities define the three groups. Useful when don’t know what you’re looking for • We could label every pixel in the image according to Requires data, but no labels which of these primary intensities it is. • i.e., segment the image based on the intensity feature. • What if the image isn’t quite so simple? Slide credit: Kristen Grauman Slide credit: Dan Klein pixel count pixel count input image input image intensity intensity • Now how to determine the three main intensities that define our groups? pixel count • We need to cluster. input image intensity Slide credit: Kristen Grauman Slide credit: Kristen Grauman 8
Recommend
More recommend
