Measuring Affinity Intensity ( ) ( ) = exp − ( ) − I y ( ) 1 2 aff x , y I x 2 σ i 2 Distance ( ) ( ) = exp − 1 2 x − y aff x , y 2 σ d 2 Color ( ) ( ) = exp − ( ) − c y ( ) 1 2 aff x , y c x 2 σ t 2 55
Eigenvectors and affinity clusters • Simplest idea: we want a • This is an eigenvalue vector a giving the problem - choose the association between each eigenvector of A with element and a cluster largest eigenvalue • We want elements within this cluster to, on the whole, have strong affinity with one another • We could maximize a T Aa • But need the constraint a T a = 1 56
Example eigenvector points eigenvector matrix 57
Example eigenvector points eigenvector matrix 58
Scale affects affinity σ =.2 σ =.1 σ =.2 σ =1 59
Scale affects affinity σ =.1 σ =.2 σ =1 60
Some Terminology for Graph Partitioning • How do we bipartition a graph: ∑ ∈ ∑ = = assoc ( A, A' ) W ( u , v ) cut ( A, B) W( u , v ), ∈ ∈ u A, v A' ∈ u A, v B A and A' not necessaril y disjoint ∩ = ∅ with A B 61 [Malik]
Minimum Cut A cut of a graph G is the set of edges S such that removal of S from G disconnects G . Minimum cut is the cut of minimum weight, where weight of cut <A,B> is given as ( ) ( ) ∑ = w A , B w x , y ∈ ∈ x A , y B 62 * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003
Minimum Cut and Clustering 63 * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003
Drawbacks of Minimum Cut • Weight of cut is directly proportional to the number of edges in the cut. Cuts with lesser weight than the ideal cut Ideal Cut 64 * Slide from Khurram Hassan-Shafique CAP5415 Computer Vision 2003
Normalized cuts • Maximize • First eigenvector of affinity matrix captures within cluster cut(A,B) cut(A,B) + similarity, but not across cluster assoc(A,V) assoc(B,V) difference • Min-cut can find degenerate where cut(A,B) is sum of weights clusters that straddle A,B; assoc(A,V) is sum of all edges with one end • Instead, we’d like to maximize in A. the within cluster similarity I.e. construct A, B such that their compared to the across cluster within cluster similarity is high difference compared to their association • Write graph as V, one cluster as with the rest of the graph A and the other as B 65
Solving the Normalized Cut problem • Exact discrete solution to Ncut is NP-complete even on regular grid, – [Papadimitriou’97] • Drawing on spectral graph theory, good approximation can be obtained by solving a generalized eigenvalue problem. 66 [Malik]
Normalized Cut As Generalized Eigenvalue problem cut ( A, B) cut ( A, B ) = + Ncut ( A, B) assoc ( A, V ) assoc ( B, V ) ∑ > D ( i , i ) + T − + − T − − ( 1 x ) ( D W )( 1 x ) ( 1 x ) ( D W )( 1 x ) x 0 = + = ; k i ∑ T − T k 1 D 1 ( 1 k ) 1 D 1 D ( i , i ) i = ... • after simplification, we get T − y ( D W ) y = ∈ − = T Ncut ( A , B ) , with y { 1 , b }, y D 1 0 . i T y Dy 67 [Malik]
Normalized cuts • Instead, solve the generalized eigenvalue problem ( ) subject to y T Dy = 1 ( ) max y y T D − W ( ) y • which gives ( ) y = λ Dy D − W • Now look for a quantization threshold that maximizes the criterion --- i.e all components of y above that threshold go to one, all below go to - b 68
Brightness Image Segmentation 69
Brightness Image Segmentation 70
71
Results on color segmentation 72
Motion Segmentation with Normalized Cuts • Networks of spatial-temporal connections: • Motion “proto-volume” in space-time 73
74
Comparison of Methods Authors Matrix used Procedure/Eigenvectors used = λ 1 st x: Ax x Perona/ Freeman Affinity A Recursive procedure 2 nd smallest generalized Shi/Malik D-A with D a − = λ ( D A x ) Dx eigenvector degree matrix = ∑ Also recursive D i i ( , ) A i j ( , ) j Scott/ Affinity A, Finds k eigenvectors of A, forms V. Normalizes rows of V. Forms Longuet-Higgins User inputs k Q = VV’. Segments by Q. Q(i,j)=1 -> same cluster Ng, Jordan, Weiss Affinity A, Normalizes A. Finds k eigenvectors, forms X. User inputs k Normalizes X, clusters rows 75 Nugent , Stanberry UW STAT 593E
Advantages/Disadvantages • Perona/Freeman – For block diagonal affinity matrices, the first eigenvector finds points in the “dominant”cluster; not very consistent • Shi/Malik – 2 nd generalized eigenvector minimizes affinity between groups by affinity within each group; no guarantee, constraints 76 Nugent , Stanberry UW STAT 593E
Advantages/Disadvantages • Scott/Longuet-Higgins – Depends largely on choice of k – Good results • Ng, Jordan, Weiss – Again depends on choice of k – Claim: effectively handles clusters whose overlap or connectedness varies across clusters 77 Nugent , Stanberry UW STAT 593E
Affinity Matrix Perona/Freeman Shi/Malik Scott/Lon.Higg 1 st eigenv. 2 nd gen. eigenv. Q matrix Affinity Matrix Perona/Freeman Shi/Malik Scott/Lon.Higg 1 st eigenv. 2 nd gen. eigenv. Q matrix Affinity Matrix Perona/Freeman Shi/Malik Scott/Lon.Higg Nugent , Stanberry UW STAT 593E 1 st eigenv. 2 nd gen. eigenv. Q matrix
Segmentation and Line Fitting • Gestalt grouping • Background subtraction • K-Means • Graph cuts • Hough transform • Iterative fitting 79
Fitting • Choose a parametric • Three main questions: object/some objects to – what object represents this represent a set of tokens set of tokens best? – which of several objects • Most interesting case is gets which token? when criterion is not local – how many objects are – can’t tell whether a set of there? points lies on a line by looking only at each point (you could read line for object and the next. here, or circle, or ellipse or...) 80
Fitting and the Hough Transform Different choices of θ , d>0 give • Purports to answer all three • questions different lines – in practice, answer isn’t • For any (x, y) there is a one usually all that much help parameter family of lines • We do for lines only through this point, given by • A line is the set of points (x, y) ( ) x + cos θ ( ) y + d = 0 sin θ such that ( ) x + cos θ ( ) y + d = 0 sin θ • Each point gets to vote for each line in the family; if there is a line that has lots of votes, that should be the line passing through the points 81
tokens votes 82
Mechanics of the Hough transform • Construct an array • How many lines? representing θ , d – count the peaks in the Hough array • For each point, render the • Who belongs to which curve ( θ , d) into this array, line? adding one at each cell – tag the votes • Difficulties – how big should the cells be? • Hardly ever satisfactory in (too big, and we cannot distinguish between quite practice, because different lines; too small, problems with noise and and noise causes lines to be cell size defeat it missed) 83
tokens votes 84
85
86
87
Line fitting What criteria to optimize when fitting a line to a set of points? 88
“Least Squares” Line fitting can be max. likelihood - but choice of model is important “Total Least Squares” 89
Who came from which line? • Assume we know how many lines there are - but which lines are they? – easy, if we know who came from which line • Three strategies – Incremental line fitting – K-means – Probabilistic (later!) 90
91
Incremental line fitting 92
Incremental line fitting 93
Incremental line fitting 94
Incremental line fitting 95
Incremental line fitting 96
97
K-means line fitting 98
K-means line fitting 99
K-means line fitting 100
Recommend
More recommend