Problem Sets E-M Reading: Problem Set 3 Forsyth & Ponce - PDF document

Problem Sets E-M • Reading: • Problem Set 3 – Forsyth & Ponce 16.1, 16.2 – Distributed Tuesday, 3/18. – Forsyth & Ponce 16.3 for challenge – Due Thursday, 4/3 problem. • Problem Set 4 • Yair Weiss: Motion Segmentation using – Distributed Tuesday, 4/1 EM – a short tutorial. – Due Tuesday, 4/15. – www – Especially 1 st 2 pages. • Probably a total of 5 problem sets. Examples of Perceptual Examples of Perceptual Grouping Grouping • Boundaries: • Regions: Find regions with uniform – Find closed contours near edges that are smooth. properties, to help find objects/parts of – Gestalt laws: common form, good continuation, interest. closure. – Color • Smooth Paths: – Intensity – Find open, smooth paths in images. – Texture – Applications: road finding, intelligent scissors (click on points and follow boundary between them). – Gestalt laws: common form, proximity. – Gestalt laws: Good continuation, common form. Examples of Perceptual Parametric Methods Grouping • Useful features: • We discussed Ransac, Hough Transform. – Example, straight lines. Can be used to find vanishing points. • The have some limitations – Gestalt laws: collinearity, proximity. – Object must have few parameters. – Finds an answer, but is it the best answer? – Hard to say because problem definition a bit vague. 1

E-M Definitions Expectation-Maximization (EM) • Models have parameters: u • Can handle more parameters. – Examples: line has slope/intercept; Gaussian has • Uses probabilistic model of all the data. mean and variance. – Good: we are solving a well defined problem. • Data is what we know from image: y – Bad: Often we don’t have a good model of the – Examples: Points we want to fit a line to. data, especially noise. • Assignment of data to models: z – Bad: Computations only feasible with pretty simple – Eg., which points came from line 1. models, which can be unrealistic. – z(i,j) = 1 means data i came from model j. • Finds (locally) optimal solution. • Data and assignments (y & z): x. E-M Definitions E-M Quick Overview • We know data (y). • Missing Data: We know y. Missing • Want to find assignments (z) and values are u and z. parameters (u). • Mixture model: The data is a mixture of • If we know y & u, we can find z more easily. more than one model. • If we know y & z, we can find u more easily. • Algorithm: 1. Guess u. 2. Given current guess of u and y, find z. (E) 3. Given current guess of z and y, find u. (M) 4. Go to 2. Example: Histograms More subtle points • Histogram gives 1D clustering problem. • Guess must be reasonable, or we won’t converge to anything reasonable. • Constant regions + noise = Gaussians. • Guess mean and variance of pixel intensities. – Seems good to start with high variance. • Compute membership for each pixel. • How do we stop. • Compute means as weighted average. – When things don’t change much. • Compute variance as weighted sample – Could look at parameters (u). variance. – Or likelihood of data. • Details: whiteboard; Also, Matlab and Weiss. 2

Overview again Drawbacks • Break unknowns into pieces. If we • Local optimum. know one piece, other is solvable. • Optimization: we take steps that make • Guess piece 1. the solution better and better, and stop when next step doesn’t improve. • Solve for piece 2. Then solve for 1. …. • But, we don’t try all possible steps. • Very useful strategy for solving problems. Local maximum which is an excellent fit to some points A dataset that is well fitted by four lines Drawbacks • How do we get models? – But if we don’t know models, in some sense we just don’t understand the problem. • Starting point? Use non-parametric method. • How many models? 3

Result of EM fitting, with one line (or at least, Result of EM fitting, with two lines (or at least, one available local maximum). one available local maximum). Segmentation with EM Seven lines can produce a rather logical answer Figure from “Color and Texture Based Image Segmentation Using EM and Its Application to Content Based Image Retrieval”,S.J. Belongie et al., Proc. Int. Conf. Computer Vision, 1998, c1998, IEEE Motion segmentation with EM • Model image pair (or • Likelihood video sequence) as – assume consisting of regions of ( ) ( ) = I x + v x , y + v y , t + 1 I x , y , t parametric motion + noise – affine motion is popular Three frames from the MPEG “flower garden” sequence  v x  = a   b     x  t x   +           v y  c d y  t y  • Straightforward • Now we need to missing variable – determine which pixels problem, rest is belong to which region Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE calculation – estimate parameters 4

Grey level shows region no. with highest probability If we use multiple frames to estimate the appearance of a segment, we can fill in occlusions; so we can re-render the sequence with some segments removed. Segments and motion fields associated with them Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Figure from “Representing Images with layers,”, by J. Wang and E.H. Adelson, IEEE Transactions on Image Processing, 1994, c 1994, IEEE Transactions on Image Processing, 1994, c 1994, IEEE Probabilistic Interpretation Generalizations • Multi-dimensional Gaussian. • We want P(u | y) – Color, texture, … – (A probability distribution of models given data). • Or maybe P(u,z | y). Or argmax(u) P(u|y). 1 1 r r r r − − ∑ − exp( ( x u ) ( x u )) • We compute: argmax(u,z) P(y | u,z). − T 1 π ∑ ( 2 ) det( ) 2 d / 2 1 / 2 – Find the model and assignments that make the i data as likely to have occurred as possible. – This is similar to finding most likely model and – Examples: 1D reduces to Gaussian assignments given data, ignoring prior on models. – 2D: nested ellipsoids of equal probability. – Discs if Covariance (Sigma) is diagonal. 5