Announcements Reminder: Pset 2 due Wed March 2 Fitting a - PDF document

2/23/2011 Announcements • Reminder: Pset 2 due Wed March 2 Fitting a transformation: • Midterm exam is Wed March 9 (2 weeks from now) feature-based alignment Wed, Feb 23 Prof. Kristen Grauman UT ‐ Austin Last time: Deformable contours Last time: Deformable contours a.k.a. active contours, snakes Given : initial contour (model) near desired object Goal : evolve the contour to fit exact object boundary Main idea : elastic band is iteratively adjusted so as to • be near image positions with high gradients, and • satisfy shape “preferences” or contour priors Kristen Grauman [Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987] Figure credit: Yuri Boykov Image from http://www.healthline.com/blogs/exercise_fitness/uploaded_images/HandBand2-795868.JPG Last time: Deformable contours Today Pros: • Interactive segmentation • Useful to track and fit non-rigid shapes • Feature-based alignment • Contour remains connected – 2D transformations • Possible to fill in “subjective” contours – Affine fit • Flexibility in how energy function is defined, weighted. Cons: – RANSAC • Must have decent initialization near true boundary, may get stuck in local minimum • Parameters of energy function must be set well based on prior information Kristen Grauman CS 376 Lecture 11 1

2/23/2011 Interactive forces Interactive forces • An energy function can be altered online based on user input – use the cursor to push or pull the initial snake away from a point. • Modify external energy term to include:  2 n 1  r  E   push 2 | p |  0 i i How can we implement such an interactive Nearby points get pushed hardest force with deformable contours? Kristen Grauman Kristen Grauman Intelligent scissors Intelligent scissors Another form of interactive segmentation: Compute optimal paths from every point to the seed based on edge-related costs. [Mortensen & Barrett, SIGGRAPH 1995, CVPR 1999] • http://rivit.cs.byu.edu/Eric/Eric.html Intelligent scissors Beyond boundary snapping… • Another form of interactive guidance: specify regions • Usually taken to suggest foreground/background color distributions User Input Result How to use this information? Boykov and Jolly (2001) • http://rivit.cs.byu.edu/Eric/Eric.html Kristen Grauman CS 376 Lecture 11 2

2/23/2011 Recall: Images as graphs Recall: Segmentation by Graph Cuts q w pq w w p A B C Fully-connected graph Break graph into segments • node for every pixel • Delete links that cross between segments • link between every pair of pixels, p , q • Easiest to break links that have low similarity • similarity w pq for each link – similar pixels should be in the same segments » similarity is inversely proportional to difference in color and position – dissimilar pixels should be in different segments Steve Seitz Steve Seitz Recall: Segmentation by Graph Cuts Graph cuts for interactive segmentation B A Link Cut • set of links whose removal makes a graph disconnected  • cost of a cut:  ( , ) cut A B w p , q   p A , q B Adding hard constraints: Find minimum cut • gives you a segmentation Add two additional nodes, object and background “terminals” • fast algorithms exist for doing this Link each pixel • To both terminals • To its neighboring pixels Yuri Boykov Source: Steve Seitz Graph cuts for interactive Graph cuts for interactive segmentation segmentation Adding hard constraints: Let the edge weight to object or background terminal reflect similarity to the respective seed pixels. Boykov and Jolly (2001) Yuri Boykov Yuri Boykov CS 376 Lecture 11 3

2/23/2011 Graph cuts for interactive “Grab Cut” segmentation • Loosely specify foreground region • Iterated graph cut ? User initialization User Initialisation Another interaction modality: specify bounding box Graph cuts to K-means for learning infer the colour distributions Intelligent Scissors Graph Cuts GrabCut segmentation Mortensen and Boykov and Rother et al. Barrett (1995) Jolly (2001) (2004) Rother et al (2004) “Grab Cut” “Grab Cut” • Loosely specify foreground region • Iterated graph cut R R Iterated Foreground & Background Foreground graph cut Background G Background G Gaussian Mixture Model (typically 5-8 components) Rother et al (2004) Rother et al (2004) Today Motivation: Recognition • Interactive segmentation • Feature-based alignment – 2D transformations – Affine fit – RANSAC Figures from David Lowe CS 376 Lecture 11 4

2/23/2011 Motivation: medical image Motivation: mosaics registration (In detail next week) Image from http://graphics.cs.cmu.edu/courses/15-463/2010_fal Parametric (global) warping Alignment problem Examples of parametric warps: • We have previously considered how to fit a model to image evidence – e.g., a line to edge points, or a snake to a deforming contour • In alignment, we will fit the parameters of some aspect rotation translation transformation according to a set of matching feature pairs (“correspondences”). x i ' x i T perspective affine Source: Alyosha Efros Parametric (global) warping Scaling Scaling a coordinate means multiplying each of its components by a scalar T Uniform scaling means this scalar is the same for all components: p = (x,y) p’ = (x’,y’) Transformation T is a coordinate-changing machine: p’ = T (p) What does it mean that T is global ? • Is the same for any point p • can be described by just a few numbers (parameters)  2 Let’s represent T as a matrix: p’ = M p     ' x x    M    '    y y Source: Alyosha Efros Source: Alyosha Efros CS 376 Lecture 11 5

2/23/2011 Scaling Scaling Non-uniform scaling : different scalars per component:  ' x ax Scaling operation:  ' y by Or, in matrix form:       ' 0 x a x        X  2,  '   0    y b y Y  0.5 scaling matrix S Source: Alyosha Efros Source: Alyosha Efros What transformations can be What transformations can be represented with a 2x2 matrix? represented with a 2x2 matrix? 2D Scaling? 2D Mirror about Y axis?        ' * ' s 0 x s x x x  x x               ' 1 0 x x x ' x    '  0 s   y   y       y ' s * y   '   0 1    y ' y y y y y 2D Rotate around (0,0)? 2D Mirror over (0,0)?              ' cos * sin * x ' cos sin x x x y                     ' sin * cos * x ' 1 0 x y x y   x ' x   y '   sin cos   y           '  '   0 1    y y y y 2D Shear? 2D Translation?         1 sh x ' x sh * y x ' x x  x         x ' x t   y ' sh * x y   sh 1   y '   y x NO! y y   ' y y t y Source: Alyosha Efros Source: Alyosha Efros 2D Linear Transformations Homogeneous coordinates       ' x a b x  To convert to homogeneous coordinates:        '      y c d y Only linear 2D transformations can be represented with a 2x2 matrix. homogeneous image Linear transformations are combinations of … coordinates • Scale, • Rotation, Converting from homogeneous coordinates • Shear, and • Mirror Source: Alyosha Efros CS 376 Lecture 11 6