Announcements Reminders: Assignment 1 due Sept 22 11:59 pm on - PDF document

9/19/2017 Announcements • Reminders: • Assignment 1 due Sept 22 11:59 pm on Canvas Recognizing object categories • No laptops, phones, tablets, etc. in class • Thoughts on review sharing? Kristen Grauman UT-Austin • Questions about presentations, experiments, Wed Sept 13, 2017 discussion proponent/opponent? Last time: Recognizing instances Last time: Recognizing instances • 1. Basics in feature extraction: filtering • 2. Invariant local features • 3. Recognizing object instances Instance recognition: Spatial Verification remaining issues • How to summarize the content of an entire image? And gauge overall similarity? Query Query • How large should the vocabulary be? How to perform quantization efficiently? • Is having the same set of visual words enough to identify the object/scene? How to verify spatial DB image with high BoW agreement? similarity DB image with high BoW similarity Both image pairs have many visual words in common. Kristen Grauman Slide credit: Ondrej Chum 1

9/19/2017 Spatial Verification Spatial Verification: two basic strategies • RANSAC Query Query • Generalized Hough Transform DB image with high BoW similarity DB image with high BoW similarity Only some of the matches are mutually consistent Slide credit: Kristen Grauman Slide credit: Ondrej Chum Outliers affect least squares fit Outliers affect least squares fit RANSAC RANSAC for line fitting Repeat N times: • Draw s points uniformly at random • RANdom Sample Consensus • Fit line to these s points • Find inliers to this line among the remaining • Approach : we want to avoid the impact of outliers, points (i.e., points whose distance from the so let’s look for “inliers”, and use those only. line is less than t ) • If there are d or more inliers, accept the line • Intuition : if an outlier is chosen to compute the and refit using all inliers current fit, then the resulting line won’t have much support from rest of the points. Lana Lazebnik 2

9/19/2017 RANSAC for line fitting example RANSAC for line fitting example Least-squares fit Source: R. Raguram Source: R. Raguram Lana Lazebnik Lana Lazebnik RANSAC for line fitting example RANSAC for line fitting example 1. Randomly select 1. Randomly select minimal subset minimal subset of points of points 2. Hypothesize a model Source: R. Raguram Source: R. Raguram Lana Lazebnik Lana Lazebnik RANSAC for line fitting example RANSAC for line fitting example 1. Randomly select 1. Randomly select minimal subset minimal subset of points of points 2. Hypothesize a 2. Hypothesize a model model 3. Compute error 3. Compute error function function 4. Select points consistent with model Source: R. Raguram Source: R. Raguram Lana Lazebnik Lana Lazebnik 3

9/19/2017 RANSAC for line fitting example RANSAC for line fitting example 1. Randomly select 1. Randomly select minimal subset minimal subset of points of points 2. Hypothesize a 2. Hypothesize a model model 3. Compute error 3. Compute error function function 4. Select points 4. Select points consistent with consistent with model model 5. Repeat 5. Repeat hypothesize-and- hypothesize-and- verify loop verify loop 24 Source: R. Raguram Source: R. Raguram Lana Lazebnik Lana Lazebnik RANSAC for line fitting example RANSAC for line fitting example Uncontaminated sample 1. Randomly select 1. Randomly select minimal subset minimal subset of points of points 2. Hypothesize a 2. Hypothesize a model model 3. Compute error 3. Compute error function function 4. Select points 4. Select points consistent with consistent with model model 5. Repeat 5. Repeat hypothesize-and- hypothesize-and- verify loop verify loop 25 Source: R. Raguram Source: R. Raguram Lana Lazebnik Lana Lazebnik Robust feature-based alignment That is an example fitting a model (line)… What about fitting a transformation (translation, affine…)? Source: L. Lazebnik 4

9/19/2017 Robust feature-based alignment Robust feature-based alignment • Extract features • Extract features • Compute putative matches Source: L. Lazebnik Source: L. Lazebnik Robust feature-based alignment Robust feature-based alignment • Extract features • Extract features • Compute putative matches • Compute putative matches • Loop: • Loop: • Hypothesize transformation T (small group of putative • Hypothesize transformation T (small group of putative matches that are related by T ) matches that are related by T ) • Verify transformation (search for other matches consistent with T ) Source: L. Lazebnik Source: L. Lazebnik Robust feature-based alignment RANSAC: General form • RANSAC loop: 1. Randomly select a seed group of points on which to base transformation estimate 2. Compute model from seed group 3. Find inliers to this transformation 4. If the number of inliers is sufficiently large, re-compute • Extract features estimate of model on all of the inliers • Compute putative matches • Loop: • Keep the model with the largest number of inliers • Hypothesize transformation T (small group of putative matches that are related by T ) • Verify transformation (search for other matches consistent with T ) Source: L. Lazebnik 5

9/19/2017 RANSAC example: Translation RANSAC example: Translation Putative matches Select one match, count inliers Source: Rick Szeliski RANSAC example: Translation RANSAC example: Translation Select one match, count inliers Find “average” translation vector Fitting an affine transformation RANSAC verification ( x i y , ) Approximates viewpoint i   changes for roughly ( x i y , ) i planar objects and roughly orthographic cameras.   m 1     m     For matching specific scenes/objects, common to   2          x   m m   x   t  x y 0 0 1 0 m x use an affine transformation for spatial verification     i i 3  i i  1 2 i  1                 m y m m y t 0 0 x y 0 1 y           i 3 4 i 2 i i 4 i       t       1     t   2 6

9/19/2017 RANSAC verification Spatial Verification: two basic strategies • RANSAC – Typically sort by BoW similarity as initial filter – Verify by checking support (inliers) for possible affine transformations • e.g., “success” if find an affine transformation with > N inlier correspondences • Generalized Hough Transform – Let each matched feature cast a vote on location, scale, orientation of the model object – Verify parameters with enough votes Kristen Grauman Voting Spatial Verification: two basic strategies • It’s not feasible to check all combinations of features by • RANSAC fitting a model to each possible subset. – Typically sort by BoW similarity as initial filter • Voting is a general technique where we let the features – Verify by checking support (inliers) for possible affine vote for all models that are compatible with it . transformations • e.g., “success” if find an affine transformation with > N inlier – Cycle through features, cast votes for model parameters. correspondences – Look for model parameters that receive a lot of votes. • Noise & clutter features will cast votes too, but typically • Generalized Hough Transform their votes should be inconsistent with the majority of – Let each matched feature cast a vote on location, “good” features. scale, orientation of the model object – Verify parameters with enough votes Kristen Grauman Kristen Grauman Difficulty of line fitting Hough Transform for line fitting • Given points that belong to a line, what is the line? • How many lines are there? • Which points belong to which lines? • Hough Transform is a voting technique that can be used to answer all of these questions. Main idea: 1. Record vote for each possible line on which each edge point lies. 2. Look for lines that get many votes . Kristen Grauman Kristen Grauman 7