Interactive Graph Cuts for Optimal Boundary & Region - PowerPoint PPT Presentation

Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images Yuri Y. Boykov and Marie-Pierre Jolly 1

From [3] 2

Vision problems as image labeling • depth (stereo) • object index (segmention) • original intensity (image restoration) 3

Labeling problems can be cast in terms of energy minimization � � E ( L ) = D p ( L p ) + V p,q ( L p , L q ) p ∈ P p,q ∈ N Labeling of pixels L : P → L p Penalty for pixel labeling D p : L p → � Interaction between V p,q : P x P → � neighboring pixels. Smoothing term. 4

Energy minimization can be solved with graph cuts S terminal S terminal T terminal T terminal 5

• Energy function and graph construction • Min-cut of graph minimizes energy • Summar max-flow/min-cut algorithms • Rest of bone segmentation example 6

Some Notation P = set of pixels p N = set of unordered pairs ( p, q ) of neighbors in P Binary vector representing L = ( L 1 , . . . , L p , . . . , L | P | ) a binary segmentation G = ( V, E ) graph with nodes, V, and edges, E B = set of user defined background pixels O = set of user defined object pixels 7

Labeling problems can be cast in terms of energy minimization � � E ( L ) = D p ( L p ) + V p,q ( L p , L q ) p ∈ P p,q ∈ N Labeling of pixels L : P → L p Penalty for pixel labeling D p : L p → � Interaction between V p,q : P x P → � neighboring pixels. Smoothing term. 8

Their Energy Function � � E ( L ) = λ · R p ( L p ) + B ( p, q ) · δ ( L p , L q ) p ∈ P ( p,q ) ∈ N D p ( L p ) becomes regional term R p ( L p ) V p,q becomes boundary term B ( p, q ) · δ ( L p , L q ) 9

Regional term � � E ( L ) = λ · R p ( L p ) + B ( p, q ) · δ ( L p , L q ) p ∈ P ( p,q ) ∈ N Penalize pixel label based on local properties Negative log-likelihood of intensity R p ( obj ) = − ln Pr ( I p |O ) R p ( bkq ) = − ln Pr ( I p |B ) 10

Boundary term: penalize dissimilar neighbors � � E ( L ) = λ · R p ( L p ) + B ( p, q ) · δ ( L p , L q ) p ∈ P ( p,q ) ∈ N p r q s B ( p, q ) ∝ exp( − ( I p − I q ) 2 1 ) · 2 σ 2 dist ( p, q ) 11

Boundary term: penalize dissimilar neighbors � � E ( L ) = λ · R p ( L p ) + B ( p, q ) · | L p − L q | p ∈ P ( p,q ) ∈ N p r q s B ( p, q ) ∝ exp( − ( I p − I q ) 2 1 ) · 2 σ 2 dist ( p, q ) 12

Graph construction: cost of n-links Object S c ( p, q ) = B ( p, q ) if ( p, q ) ∈ N B ( p, q ) B ( p, q ) ∝ exp( − ( I p − I q ) 2 1 ) · 2 σ 2 dist ( p, q ) T Background 13

Graph construction: cost of t-link (p, S) Object S If p ∈ O then c ( p, q ) = K K � K = 1 + max B ( p, q ) p ∈ P q :( p,q ) ∈ N T Background 14

Graph construction: cost of t-link (p, S) Object S If p / ∈ O ∪ B then c ( p, q ) = λ · R p ( bkg ) T Background 15

Graph construction: cost of t-link (p, S) Object S 0 If p ∈ B then c ( p, q ) = 0 T Background 16

Claim: min-cut of graph minimizes energy • Min-cut on G is a feasible cut • Each feasible cut has a unique binary segmentation • Segmentation associated with min-cut that satisfies user defined constraints minimizes the energy function 17

Summary of max-flow/min-cut algorithms • Augmenting paths (Ford and Fulkerson) • Push-relabel (Goldberg and Tarjan) • Their implementation (see [2]) 18

From [3] 19

Citations • [1] Kolmogorov, V. and Zabih, R. What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, February 2004. • [2] Boykov, Y. and Kolmogorov, V. An experimental comparison of min- cut/max-flow algoritms for energy minimization in computer vision. IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124-1137, September 2004. • [3] Boykov, Y., Torr, T. and Zabih, R. Tutorial on Discrete Optimization Methods in Computer Vision. ECCV 2004 20