Final Presentation Image Segmentation Tamir Segev Chen Shapira - PowerPoint PPT Presentation

Final Presentation Image Segmentation Tamir Segev Chen Shapira

Project Goal To implement a multi-region image segmentation algorithm, with active contours and single level set function.

Introduction • In our work, we’ll focus on the Multi-region Active Contours with a single Level-Set function method proposed in [1]. • This method allows quick & accurate image segmentation on 2d and 3d images. • This is done by dividing the image into multiple regions, and iteratively calculating a distance function, which is extracted using the Voronoi Implicit Interface Method. [1] - Anastasia Dubrovina, Guy Rosman, and Ron Kimmel . “Multi-Region Active Contours with a Single Level Set Function”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, No. 8, August 2015.

Introduction • We’ve used the algorithm proposed in article [1]. • This algorithm was previously implemented in Matlab. • Here, we implemented the proposed method in C++ using the commonly used library - OpenCV. • This made the implementation portable across multiple platforms, and easily used. • This will allow using this segmentation method in a relatively fast and agile environment.

The Algorithm Initialization: C 0 Start with an initial contour , compute the distance level φ set function from that initial contour. Perform until convergence: 1. Calculate the extension velocity function , and F ext φ t calculate from it. φ 2. Evolve using the LOD scheme φ 3. Extract the epsilon level-sets of the evolved 4. Calculate the Voronoi Regions of the epsilon level-sets 5. Reconstruct the evolved contour C as the boundaries between the regions φ 6. Perform re-distancing and calculate the current using the contour C and the Fast Marching Method on it

Active Contours • In our project, we used the approach of Multi-Region Active Contours with a Single Level Set Function. • In the active contours method, we model the region boundaries as curves (contours), while choosing a segmentation criteria to minimize. • This segmentation criteria is the energy functional: E ( C ) = E data ( C ) + µ E reg ( C )

Active Contours • We minimize the energy functional by using the steepest-descent method, with respect to a virtual parameter ’t’: + µ δ E reg ( C ) C t = − δ E ( C ) = − δ E data ( C ) ⎡ ⎤ ⎥ = Fn ⎢ δ C δ C δ C ⎣ ⎦

+ µ δ E reg ( C ) C t = − δ E ( C ) = − δ E data ( C ) ⎡ ⎤ ⎥ = Fn ⎢ δ C δ C δ C ⎣ ⎦ Geodesic Active Contours • The regularization term used for Multi-Region Active Contours is the regularization term: µ ∑ ∫ ! g ( C i ( s )) ds 2 i C i • This term is defined by g(x): g ( x ) = (1 + | ∇ ( G ∗ I )| 2 ) − 1

+ µ δ E reg ( C ) C t = − δ E ( C ) = − δ E data ( C ) ⎡ ⎤ ⎥ = Fn ⎢ δ C δ C δ C ⎣ ⎦ Piecewise-Constant Model • The data term we used for Multi-Region Active Contours is the data term: ∑ ( I ( x ) − c i ) 2 dx ∫ M E data ( C ,{ c i }) = i = 1 Ω i ∫ I ( x ) dx Ω i where : c i = ∫ dx Ω i • This is known as the Piecewise-Constant Model. • Together they form the new evolution rule: ( I ( x ) − c i ) 2 + µ data ( x ) + µ ⎡ ⎤ ⎡ ⎤ ∑ ∑ C t = 2 ( κ i g − 〈∇ g , n i 〉 ) n i = gac ( x ) F 2 F n i ⎢ ⎥ ⎢ ⎥ i i ⎣ ⎦ ⎣ ⎦ i ∈ N ( x ) i ∈ N ( x )

Implicit Method C t • We don’t calculate directly. Rather, we get φ C from the zero level-set of the function φ • is defined as the Level-Set function, which at each iteration represents the distances of each pixel to the nearest contour. φ • is calculated in 2 steps: ! φ t = F ext | ∇ φ | φ • We calculate and evolve from it using semi-implicit LOD scheme. • The Voronoi Implicit Interface Method is used ! φ φ to calculate C from , and the next is calculated to be the distance from C.

Extended Velocity Function φ • To calculate over the whole image, we need to extend the velocity function from the contour to the whole image • Here we get: ⎛ ⎞ ( ) gac ( x ) = div g ( x ) ∇ φ data ( x ) = F i ( x ) − F j ( x ) = ( I ( x ) − c i ) 2 − ( I ( x ) − c j ) 2 F ext F ext ⎜ ⎟ | ∇ φ | ⎝ ⎠ φ • From this, we derive the evolution of : ⎡ ⎤ ⎛ ⎞ φ t ( x ) = F ext | ∇ φ | = F i ( x ) − F j ( x ) + µ div g ( x ) ∇ φ ⎥ | ∇ φ | ⎢ ⎜ ⎟ | ∇ φ | ⎝ ⎠ ⎣ ⎦

Voronoi Implicit Interface Method • The Voronoi Implicit Interface Method is a framework for numerical solution of interface propagation problems, allowing numerous regions. • This method arose from the field of computational fluid dynamics.

Voronoi Implicit Interface Method • This method allows the evolution of the level set function by: • Finding the epsilon-level sets of the level set function phi. • Computing the Voronoi regions of each epsilon- level set • Reconstructing the contour C as the intersection of the Voronoi regions

Fast Marching Method • This method is a numerical method for solving boundary value problems.

The Algorithm Initialization: C 0 Start with an initial contour , compute the distance level φ set function from that initial contour. Perform until convergence: 1. Calculate the extension velocity function , and F ext φ t calculate from it. φ 2. Evolve using the LOD scheme φ 3. Extract the epsilon level-sets of the evolved 4. Calculate the Voronoi Regions of the epsilon level-sets 5. Reconstruct the evolved contour C as the boundaries between the regions φ 6. Perform re-distancing and calculate the current using the contour C and the Fast Marching Method on it

Results Original Segmentation Level-sets

Results Original Segmentation Level-sets

Results Original Segmentation Level-sets

Results Original Segmentation Initial Contour

Results Original with custom contour Iteration 1 Segmentation Iteration 2 I t e r a t i o n 5

Performance Time analysis for different modules 4% 2% 3% 91% Redistancing Evolution LOD Force Misc

Performance Time analysis for segmentation of the same image in different sizes Iteration \ Image Size 200x200 400x400 720x720 1200x1200 1 2.55385 8.84214 19.4748 40.2629 2 1.85931 5.66848 17.1476 43.5463 3 1.62422 5.4637 16.3112 34.4325 4 1.3332 5.3024 15.0653 33.546 5 1.34812 4.84125 13.9251 33.37 6 1.29236 4.46471 14.3317 33.8212 7 1.28993 4.03684 13.7124 31.649 8 1.35421 4.24658 12.8794 31.8627 9 1.33509 4.4917 13.5365 31.6875 10 1.28264 4.49044 12.1101 33.1953 11 1.3427 4.39936 12.5372 35.0835 12 1.32035 4.3532 12.7796 32.6506 13 1.25789 4.84434 12.1706 35.5387 14 1.07976 4.11092 11.4616 35.3814 15 1.07421 4.37707 13.2785 32.7203 Elapsed 21.5636 74.311 211.56 520.628

End Notes • Overall, we’ve seen good results for segmenting many types of images. • In many cases we tested, we saw that the segmentation and level-set function that evolved during the iterations were as expected, and very close to the contours perceived by the naked eye. • We saw the limitations of our algorithm when we tried segmenting images with a mild gradient (such as the aura of the moon). • We also noticed that we can improve the segmentation of some images by choosing a custom initial contour for them, and offered a few ideas on how to expand our implementation.