Graph-based Segmentation Computer Vision CS 543 / ECE 549 - PowerPoint PPT Presentation

02/25/10 Graph-based Segmentation Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem

Images as graphs i w ij c j • Fully-connected graph – node for every pixel – link between every pair of pixels, p , q – similarity w ij for each link Source: Seitz

Segmentation by Graph Cuts w A B C • Break Graph into Segments – Delete links that cross between segments – Easiest to break links that have low cost (low similarity) • similar pixels should be in the same segments • dissimilar pixels should be in different segments Source: Seitz

Graph cuts segmentation

Markov Random Fields Node y i : pixel label Edge: constrained pairs Cost to assign a label to Cost to assign a pair of labels to each pixel connected pixels å å q = y q y q Energy ( y ; , data ) ( y ; , data ) ( y , y ; , data ) 1 i 2 i j Î i i , j edges

Markov Random Fields Unary potential • Example: “label smoothing” grid 0: -logP(y i = 0 ; data) 1: -logP(y i = 1 ; data) Pairwise Potential 0 1 0 0 K 1 K 0 å å q = y q y q Energy ( y ; , data ) ( y ; , data ) ( y , y ; , data ) 1 i 2 i j Î i i , j edges

Solving MRFs with graph cuts Source (Label 0) Cost to assign to 0 Cost to split nodes Cost to assign to 1 Sink (Label 1) å å q = y q y q Energy ( y ; , data ) ( y ; , data ) ( y , y ; , data ) 1 i 2 i j Î i i , j edges

Solving MRFs with graph cuts Source (Label 0) Cost to assign to 0 Cost to split nodes Cost to assign to 1 Sink (Label 1) å å q = y q y q Energy ( y ; , data ) ( y ; , data ) ( y , y ; , data ) 1 i 2 i j Î i i , j edges

Graph cuts segmentation 1. Define graph usually 4-connected or 8-connected – 2. Define unary potentials Color histogram or mixture of Gaussians for – background and foreground æ ö q P ( c ( x ); ) ç ÷ = - foreground unary _ potential ( x ) log ç ÷ q P ( c ( x ); ) è ø background 3. Define pairwise potentials ì ü 2 - - c ( x ) c ( y ) ï ï = + edge _ potential ( x , y ) k k exp í ý 1 2 s 2 2 ï ï 4. Apply graph cuts î þ 5. Return to 2, using current labels to compute foreground, background models

Moderately straightforward examples … GrabCut completes automatically GrabCut – Interactive Foreground Extraction 10

Difficult Examples Camouflage & Fine structure Harder Case Low Contrast Initial Rectangle Initial Result GrabCut – Interactive Foreground Extraction 11