Fitting: Deformable contours Goal: move from array of pixel values - PDF document

2/21/2011 Recap so far: Grouping and Fitting Fitting: Deformable contours Goal: move from array of pixel values (or Monday, Feb 21 filter outputs) to a collection of regions, Prof. Kristen Grauman objects, and shapes. UT-Austin Fitting: Edges vs. boundaries Grouping: Pixels vs. regions By grouping pixels based on Gestalt- inspired attributes, we can map the pixels into a set of image clusters on intensity regions. Each region is consistent according to the Edges useful signal to features and indicate occluding similarity metric we boundaries, shape. used to do the …but quite often boundaries of interest clustering . Here the raw edge are fragmented, and we have extra output is not so bad… image clusters on color “clutter” edge points. Kristen Grauman Images from D. Jacobs Kristen Grauman Fitting: Edges vs. boundaries Voting with Hough transform Given a model of • Hough transform for fitting lines, circles, arbitrary interest, we can shapes overcome some of the missing and noisy b edges using fitting y ( x 1 , y 1 ) techniques. y 0 ( x 0 , y 0 ) With voting methods x 0 x m like the Hough Hough space image space transform , detected points vote on possible model parameters. In all cases, we knew the explicit model to fit. Kristen Grauman Kristen Grauman CS376 Lecture 10 K. Grauman 1

2/21/2011 Today Deformable contours a.k.a. active contours, snakes • Fitting an arbitrary shape with “active” Given : initial contour (model) near desired object deformable contours [Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987] Figure credit: Yuri Boykov Deformable contours Deformable contours: intuition a.k.a. active contours, snakes Given : initial contour (model) near desired object Goal : evolve the contour to fit exact object boundary Main idea : elastic band is iteratively adjusted so as to • be near image positions with high gradients, and • satisfy shape “preferences” or contour priors [Snakes: Active contour models, Kass, Witkin, & Terzopoulos, ICCV1987] Figure credit: Yuri Boykov Image from http://www.healthline.com/blogs/exercise_fitness/uploaded_images/HandBand2-795868.JPG Kristen Grauman Why do we want to fit Deformable contours vs. Hough deformable shapes? Like generalized Hough transform, useful for shape fitting; but initial intermediate final Hough Deformable contours • Some objects have similar basic form but Rigid model shape Prior on shape types, but shape some variety in the contour shape. iteratively adjusted ( deforms ) Single voting pass can detect multiple instances Requires initialization nearby One optimization “pass” to fit a single contour Kristen Grauman Kristen Grauman CS376 Lecture 10 K. Grauman 2

2/21/2011 Why do we want to fit Why do we want to fit deformable shapes? deformable shapes? • Non-rigid, • Non-rigid, deformable deformable objects can objects can change their change their shape over shape over time, e.g. lips, time, e.g. lips, hands… hands… Figure from Kass et al. 1987 Kristen Grauman Kristen Grauman Why do we want to fit Aspects we need to consider deformable shapes? • Representation of the contours • Defining the energy functions – External – Internal • Minimizing the energy function • Extensions: – Tracking – Interactive segmentation • Non-rigid, deformable objects can change their shape over time. Figure credit: Julien Jomier Kristen Grauman Kristen Grauman Fitting deformable contours Representation How should we adjust the current contour to form the new • We’ll consider a discrete representation of the contour, contour at each iteration? consisting of a list of 2d point positions (“vertices”). • Define a cost function (“energy” function) that says how   ( , ), x y good a candidate configuration is. ( , ) x 0 y i i i 0 • Seek next configuration that minimizes that cost function.   0 , 1 ,  , 1 i n for ( x 19 y , ) 19 • At each iteration, we’ll have the option to move each vertex to initial intermediate another nearby location (“state”). final Kristen Grauman CS376 Lecture 10 K. Grauman 3

2/21/2011 Energy function External energy: intuition The total energy (cost) of the current snake is • Measure how well the curve matches the image data defined as: • “Attract” the curve toward different image features   – Edges, lines, texture gradient, etc. E E E total internal external Internal energy: encourage prior shape preferences: e.g., smoothness, elasticity, particular known shape. External energy (“image” energy): encourage contour to fit on places where image structures exist, e.g., edges. A good fit between the current deformable contour and the target shape in the image will yield a low value for this cost function. External image energy External image energy ( , ) ( , ) • Gradient images and G x x y G y x y How do edges affect “snap” of rubber band? Think of external energy from image as gravitational pull towards areas of high contrast • External energy at a point on the curve is:       2 2 ( ) ( | ( ) | | ( ) | ) E G G external x y • External energy for the whole curve:  n 1 Magnitude of gradient     2 2 | ( , ) | | ( , ) | E G x y G x y - (Magnitude of gradient)  2 2 ( ) ( )   external x i i y i i G I G I x y    2 2 i 0 G ( I ) G ( I ) x y Kristen Grauman Kristen Grauman Internal energy: intuition Internal energy: intuition A priori , we want to favor smooth shapes, contours with low curvature , contours similar to a known shape , etc. to balance what is actually observed (i.e., in the gradient image). What are the underlying And in this one? boundaries in this fragmented edge image? Kristen Grauman Kristen Grauman CS376 Lecture 10 K. Grauman 4

2/21/2011 Internal energy Internal energy For a continuous curve, a common internal energy term • For our discrete representation, is the “bending energy”.      ( x , y ) i 0 n 1 … At some point v(s) on the curve, this is: i i i  2  2 d d  2  2                  ( ) ( ) 2 d d v      i 1 i i i 1 i 1 i i 1 i 1 i 2 ds ds      E internal ( ( s )) 2 ds d s Note these are derivatives relative to position- --not spatial • Internal energy for the whole curve: image gradients.  n 1  Tension, Stiffness,      2        2 2 E Elasticity Curvature    internal i 1 i i 1 i i 1  i 0 Why do these reflect tension and curvature ? Kristen Grauman Kristen Grauman Example: compare curvature Penalizing elasticity       2 • Current elastic energy definition uses a discrete estimate E curvature v ( ) 2   i i 1 i i 1 of the derivative:        2 2 n 1 ( x 2 x x ) ( y 2 y y )        2     i 1 i i 1 i 1 i i 1 E elastic i 1 i  0 i (2,5)  1 n        2 2 ( ) ( ) x x y y   i 1 i i 1 i (2,2)  i 0 (3,1) What is the possible problem (1,1) (3,1) (1,1) with this definition? 3 � 2 2 � 1 2 � 1 � 2 5 � 1 2 3 � 2 2 � 1 2 � 1 � 2 2 � 1 2 � �2 2 � 4 � �8 2 � 64 Kristen Grauman Kristen Grauman Dealing with missing data Penalizing elasticity • The preferences for low-curvature, smoothness help • Current elastic energy definition uses a discrete estimate deal with missing data: of the derivative:  1 n  2       E elastic i 1 i  i 0    2 Instead: n 1         2 2 ( x x ) ( y y ) d   1 1 i i i i  i 0 where d is the average distance between pairs of points – updated at each iteration. Illusory contours found! [Figure from Kass et al. 1987] Kristen Grauman CS376 Lecture 10 K. Grauman 5