BIL 717 Image Processing Active Contours Variational - PowerPoint PPT Presentation

Today BIL 717 Image Processing Active Contours • • Variational Segmentation Models Mar. 7, 2016 Active Contours/Snakes Variational Segmentation Models Erkut Erdem Acknowledgement: The slides on active contours are adapted from the slides Hacettepe University prepared by K. Grauman of University of Texas at Austin Computer Vision Lab (HUCVL) Fitting: Edges vs. boundaries Today • Active Contours Variational Segmentation Models • Edges useful signal to indicate occluding boundaries, shape. Here the raw edge …but quite often boundaries of interest are output is not so bad… fragmented, and we have extra “clutter” edge points. Images from D. Jacobs

Fitting: Edges vs. boundaries Deformable contours a.k.a. active contours, snakes Given a model of interest, we can overcome some of Given: initial contour (model) near desired object the missing and noisy edges Goal: evolve the contour to fit exact object boundary using fitting techniques. Main idea: elastic band is iteratively adjusted so as to With voting methods like the Hough transform , detected • be near image positions with points vote on possible high gradients, and model parameters. • satisfy shape “preferences” or contour priors [Snakes: Active contour models, Kass, Witkin, & T erzopoulos, ICCV1987] Figure credit: Yuri Boykov Deformable contours: intuition Deformable contours vs. Hough Like generalized Hough transform, useful for shape fitting; but initial intermediate final Hough Deformable contours Rigid model shape Prior on shape types, but shape iteratively adjusted (deforms) Single voting pass can detect multiple instances Requires initialization nearby One optimization “pass” to fit a single contour Image from http://www.healthline.com/blogs/exercise_fitness/up loaded_images/H and Band 2-795868.JPG

Why do we want to fit deformable Why do we want to fit deformable shapes? shapes? Some objects have similar basic form but • some variety in the contour shape. • Non-rigid, deformable objects can change their shape over time. Figure credit: Julien Jomier Representation Aspects we need to consider • We’ll consider a discrete representation of the contour, • Representation of the contours consisting of a list of 2d point positions (“vertices”). Defining the energy functions • – External ( x , y ), ν = ( x 0 y , ) – Internal i i i 0 • Minimizing the energy function i 0 , 1 , ! , n 1 = − for ( x 19 y , ) 19 • At each iteration, we’ll have the option to move each vertex to another nearby location (“state”).

Fitting deformable contours Energy function How should we adjust the current contour to form the new contour at each The total energy (cost) of the current snake is iteration? defined as: • Define a cost function (“energy” function) that says how good a E E E candidate configuration is. = + total internal external • Seek next configuration that minimizes that cost function. Internal energy: encourage prior shape preferences: e.g., smoothness, elasticity, particular known shape. External energy (“image” energy): encourage contour to fit on places where image structures exist, e.g., edges. A good fit between the current deformable contour and initial intermediate final the target shape in the image will yield a low value for this cost function. External energy: intuition External image energy How do edges affect “snap” of rubber Measure how well the curve matches the image data • band? “Attract” the curve toward different image features • Think of external – Edges, lines, texture gradient, etc. energy from image as gravitational pull towards areas of high contrast Magnitude of gradient - (Magnitude of gradient) 2 2 G ( I ) G ( I ) + x y ( ) 2 2 G ( I ) G ( I ) − + x y

External image energy Internal energy: intuition G x ( x , y ) G y ( x , y ) • Gradient images and External energy at a point on the curve is: • 2 2 E ( ) ( | G ( ) | | G ( ) | ) What are the underlying ν = − ν + ν And in this one? external x y boundaries in this fragmented edge image? • External energy for the whole curve: n 1 − 2 2 E | G ( x , y ) | | G ( x , y ) | ∑ = − + external x i i y i i i 0 = Internal energy: intuition Internal energy For a continuous curve, a common internal energy term is A priori , we want to favor smooth shapes, contours with the “bending energy”. low curvature , contours similar to a known shape , etc. to balance what is actually observed (i.e., in the gradient image). At some point v(s) on the curve, this is: 2 2 2 d d ν ν E internal ( ( s )) ν = α + β 2 ds d s Tension, Stiffness, Elasticity Curvature

Internal energy Example: compare curvature 2 For our discrete representation, • E curvature v ( ) 2 = ν − ν + ν i i 1 i i 1 + − 2 2 ( x 2 x x ) ( y 2 y y ) ( x , y ) i 0 ! n 1 = − + + − + ν = = − … i 1 i i 1 i 1 i i 1 + − + − i i i d 2 d ν (2,5) ν v ( ) ( ) 2 ≈ − ν ≈ ν − ν − ν − ν = ν − ν + ν i 1 i + ds i 1 i i i 1 i 1 i i 1 2 + − + − ds (2,2) Note these are derivatives relative to position---not spatial image gradients. • Internal energy for the whole curve: (3,1) (1,1) (3,1) (1,1) n 1 − 2 2 E 2 ∑ = α ν − ν + β ν − ν + ν internal i 1 i i 1 i i 1 + + − i 0 = Why do these reflect tension and curvature? Penalizing elasticity Penalizing elasticity Current elastic energy definition uses a discrete estimate Current elastic energy definition uses a discrete estimate • • of the derivative: of the derivative: n 1 n 1 − − 2 2 E E ∑ ∑ = α ν + − ν = α ν + − ν elastic i 1 i elastic i 1 i i 0 i 0 = = n 1 − 2 Instead: n 1 2 2 − ( x x ) ( y y ) ∑ ( ) = α ⋅ − + − 2 2 ( x x ) ( y y ) d ∑ i 1 i i 1 i = α ⋅ − + − − + + i 1 i i 1 i + + i 0 = i 0 = where d is the average distance between pairs What is the possible problem of points – updated at each iteration. with this definition?

Extending the internal energy: Dealing with missing data capture shape prior The preferences for low-curvature, smoothness help deal • with missing data: If object is some smooth variation on a • known shape, we can use a term that will penalize deviation from that shape: n 1 − ˆ 2 E ( ) ∑ + = α ⋅ ν − ν internal i i i 0 = where are the points of the known shape. Illusory contours found! ˆ { } ν i [Figure from Kass et al. 1987] Fig from Y. Boykov Total energy: function of the weights Total energy: function of the weights • e.g., weight controls the penalty for internal elasticity α E E E = + γ total internal external n 1 − 2 2 E | G ( x , y ) | | G ( x , y ) | ∑ = − + external x i i y i i small α large α medium α i 0 = n 1 − 2 ( ) 2 E d 2 ∑ = α − ν − ν + β ν − ν + ν internal i 1 i i 1 i i 1 + + − i 0 = Fig from Y. Boykov

Energy minimization Recap: deformable contour • A simple elastic snake is defined by: • Several algorithms have been proposed to fit – A set of n points, deformable contours: – An internal energy term (tension, bending, – Greedy search plus optional shape prior) – Dynamic programming (for 2d snakes) – An external energy term (gradient-based) – etc. • To use to segment an object: – Initialize in the vicinity of the object – Modify the points to minimize the total energy Energy minimization: greedy Energy minimization: dynamic programming For each point, search window around it • and move to where energy function is v v minimal 2 3 v – Typical window size, e.g., 5 x 5 pixels 1 v v 4 6 v Stop when predefined number of points • 5 have not changed in last iteration, or after max number of iterations With this form of the energy function, we can minimize using dynamic programming, with the Viterbi algorithm. • Note: – Convergence not guaranteed Iterate until optimal position for each point is the center of – Need decent initialization the box, i.e., the snake is optimal in the local search space constrained by boxes. Fig from Y. Boykov [Amini, Weymouth, Jain, 1990]