Camera parameter estimation for image based modeling Jaechul Kim Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Purpose Purpose • Introduce a basic procedure of camera Introduce a basic procedure of camera parameter estimation from multiple images and its application to image ‐ based modeling and its application to image based modeling Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Overview of general procedure Overview of general procedure • Step 1 : Point matches and epipolar geometry Step 1 : Point matches and epipolar geometry estimation (i.e. Fundamental matrix computation) computation) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Overview of general procedure Overview of general procedure • Step2 : Estimation of camera parameters Step2 : Estimation of camera parameters – Focal length, Camera position, Viewing direction etc etc. Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Overview of general procedure Overview of general procedure • Step 3: 3D reconstruction & Texture mapping Step 3: 3D reconstruction & Texture mapping Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature Points Matching & Epipolar Geometry Estimation • General procedure General procedure – Find point correspondences between images – From point correspondences, compute From point correspondences compute fundamental matrices (F ‐ matrices) between images images • Outliers in point correspondences are rejected during F ‐ matrix computation using RANSAC • Output : F ‐ matrix (i.e. projective reconstruction) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step 1 Feature Points Matching • Three methods are tested in this demo Three methods are tested in this demo – Harris corner detector & Window correlation + RANSAC RANSAC – SIFT detector & SIFT descriptor + RANSAC – Manual matching – Manual matching Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Epipolar geometry l • Projective geometry between two views Projective geometry between two views e1,e2 : epipoles X l1,l2 : epipolar lines l2 l1 x1 x2 C1 C2 e1 e2 Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Fundamental matrix d l • Encode epipolar geometry between two views Encode epipolar geometry between two views • Rank ‐ 2 matrix (det(F) = 0) that can be computed from at least 7 point computed from at least 7 ‐ point correspondences = T Fx x x 2 Fx 0 0 1 • Define epiploar line for a given point x 1 or x 2 = l Fx 2 1 = T l l F F x x 1 1 2 2 Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 RANSAC (RANdom SAmple Concensus) ( d l ) • Robust estimation technique under the presence of q p outliers • Algorithm outline – Given putative correspondences, sample 7 or 8 Given putative correspondences sample 7 or 8 correspondences and then compute the Fundamental matrix – Using the computed Fundamental matrix count the – Using the computed Fundamental matrix, count the number of inliers – If the number of inlier is a maximum among iterations, store the Fundamental matrix and inliers store the Fundamental matrix and inliers. – Repeat the sampling. Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step 1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Harris corner detector • Harris corner detector – Parameters to be used • Harris threshold, M c, is 500 • Kappa is set to 0 04 Kappa is set to 0.04 • Gaussian smoothing with sigma 1 is applied to image before corner detection • Window size (u v) is 1 • Window size (u,v) is 1 Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Windows correlation d l – For a detected corner point (x,y) in the image 1, search the corner point (x’,y’) in the image 2 with h h ( ’ ’) h h the minimum SSD error – Parameter to be used P t t b d • Correlation window size 15 • Search area in the image 2 is set to 300 by 300 (1/4 size • Search area in the image 2 is set to 300 by 300 (1/4 size of the image) centered to (x,y) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Harris corner detection d I iti ll d t Initially detected corner points t d i t Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Window correlation + RANSAC • Window correlation + RANSAC Putative matches (626) Inliers after RANSAC (23, 4%) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Examples of false matches E l f f l t h Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Examples of false matches l f f l h Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • More examples (Harris + RANSAC) Initially detected corner points Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • More examples (Harris + RANSAC) l ( ) Putative matches (386) Inliers after RANSAC (141, 37%) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • More examples (Harris + RANSAC) :Good result Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • More examples (Harris + RANSAC) :Good result l Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching Harris corner + Window correlation + RANSAC • Harris + RANSAC Concl sion • Harris + RANSAC ‐ Conclusion – Weak to matching two images with large viewpoint change viewpoint change – Confusion in repetitive textures – Some of image pairs have incorrect F matrices S f i i h i F i – Harris corner detection seems to be more proper to ideo based camera parameter tracking to video based camera parameter tracking where here image change between consecutive frames is small small Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • SIFT + RANSAC – Parameter to be used • Sigma : 0.5 • Number of octaves : 6 • Number of levels per octave: 3 N b f l l t 3 • SIFT descriptor : 128 dimensions – Putative matches are found using nearest Putative matches are found using nearest neighbor between the SIFT descriptors Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • SIFT + RANSAC SIFT + RANSAC 50 50 100 100 150 150 200 200 250 250 300 300 350 350 400 400 450 450 500 500 550 550 100 200 300 400 500 600 700 100 200 300 400 500 600 700 Initially detected SIFT feature points Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • SIFT + RANSAC Putative matches (258) Inliers after RANSAC (133, 52%) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • SIFT + RANSAC Good res lt • SIFT + RANSAC : Good result Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • SIFT + RANSAC : Good result d l Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • Failure examples (SIFT + RANSAC) l l ( ) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Step1 Feature point detection & matching Feature point detection & matching SIFT + RANSAC • Failure examples (SIFT + RANSAC) ( ) Demo presentation ‐ Visual recognition and search, Mar 21, 2008
Recommend
More recommend
