Reconnaissance dobjets Reconnaissance d objets Reconnaissance d - PowerPoint PPT Presentation

Reconnaissance d’objets Reconnaissance d objets Reconnaissance d objets Reconnaissance d’objets et vision et vision artificielle et vision et vision artificielle artificielle artificielle http://www.di.ens.fr/willow/teaching/recvis09 http://www.di.ens.fr/willow/teaching/recvis09 p p g g L Lecture 3 t 3 A refresher on camera geometry A refresher on camera geometry Image alignment and 3D alignment g g g

Check Check it Check Check it it out! it out! out! out! Cours Cours de “Computational photography” Cours Cours de Computational photography de “Computational photography” de Computational photography de Frédo de Frédo Durand Durand L L j Le Le jeudi j jeudi de 9h30 a 12h30 Salle Info 2 di di d 9h30 12h30 S ll I f 2 d 9h30 12h30 S ll I f 2 de 9h30 a 12h30 Salle Info 2 http://people.csail.mit.edu/fredo/Classes/Comp_Photo_ENS/ http://people.csail.mit.edu/fredo/Classes/Comp_Photo_ENS/

N’oubliez pas N oubliez pas N oubliez pas! N’oubliez pas! ! ! Premier Premier exercice Premier Premier exercice exercice de exercice de de programmation de programmation programmation du programmation du du du le 27 le 27 octobre octobre http://www.di.ens.fr/willow/teaching/recvis09/assignment1/ http://www.di.ens.fr/willow/teaching/recvis09/assignment1/

Pinhole perspective equation ⎧ = x x ' f f ' ⎪ ⎪ ⎪ ⎪ z NOTE: z is always negative.. ⎨ y y ⎪ ⎪ = y ' ' f f ' ' ⎪ ⎩ z

Affine models: Weak perspective projection p p p j = − ⎧ x ' mx f ' is the magnification. is the magnification = = − ⎨ ⎨ where where m m = − ⎩ y ' my z 0 When the scene relief is small compared its distance from the Camera, m can be taken constant: weak perspective projection.

Affine models: Orthographic projection ff g p p j When the camera is at a = x ⎧ x ' (roughly constant) distance y ⎨ ⎨ = y ⎩ ⎩ from the scene, take m =1. y ' '

Analytical camera geometry Analytical camera geometry

Coordinate Changes: Pure Translations O B P = O B O A + O A P ⇔ B P = A P + B O A

Coordinate Changes: Pure Rotations ⎡ ⎤ ⎡ ⎤ A T i i j i k i . . . i A B A B A B [ [ ] ] B ⎢ ⎥ ⎢ ⎥ = = = B B A T B B A R i . j j j j . j j k . j j i i j j k k j j ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ A A A B B A A B B A A B B A A B B A A A A ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ A T i k j k k k . . . k ⎣ ⎦ A B A B A B B

Coordinate Changes: Rotations about the z Axis θ θ θ θ ⎡ ⎡ ⎤ ⎤ cos cos sin sin 0 0 ⎢ ⎥ = − θ θ B R sin cos 0 ⎢ ⎥ A ⎢ ⎢ ⎥ ⎥ ⎣ ⎦ 0 0 1

A rotation matrix is characterized by the following properties: properties: • Its inverse is equal to its transpose, and q p • its determinant is equal to 1. Or equivalently: • Its rows (or columns) form a right-handed Its rows (or columns) form a right handed orthonormal coordinate system.

Coordinate changes: g pure rotations ⎡ ⎤ ⎡ ⎤ A B x x ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ [ ] [ ] = = A B i j k i j k OP y y ⎢ ⎥ ⎢ ⎥ A A A B B B ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ A B z z z z ⎣ ⎣ ⎦ ⎦ ⎣ ⎣ ⎦ ⎦ B = ⇒ B A P R P A

Coordinate Changes: Rigid Transformations Coordinate Changes: Rigid Transformations ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ + = + B B B A B A B A P R O P R P O P B B A B = P R = P O = B A A A A ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ T A T ⎣ ⎣ ⎦ ⎦ ⎣ ⎣ ⎦ ⎦ ⎣ ⎣ ⎦ ⎦ ⎣ ⎣ ⎦ ⎦ ⎣ ⎣ ⎦ ⎦ A A A A 0 0 1 1 1 1 1 1 1 1 1 1

Pinhole perspective equation ⎧ = x x ' f f ' ⎪ ⎪ ⎪ ⎪ z NOTE: z is always negative.. ⎨ y y ⎪ ⎪ = y ' ' f f ' ' ⎪ ⎩ z

The intrinsic parameters of a camera Units: k l : pixel/m k,l : pixel/m f : m α β : pixel α,β : pixel Physical image coordinates Normalized image coordinates coordinates

The intrinsic parameters of a camera Calibration matrix Th The perspective ti projection equation

The extrinsic parameters of a camera

Perspective projections induce projective transformations between planes f i b l

Affine cameras Weak-perspective projection Paraperspective projection Paraperspective projection

More affine cameras Orthographic projection Parallel projection

Weak perspective projection model Weak-perspective projection model ( p and P are in homogeneous coordinates) ( p n n m g n u n ) r p = M P p M P ( P is in homogeneous coordinates) p = A P + b p (neither p nor P is in hom. coordinates)

Affine projections induce affine p j transformations from planes onto their images. onto their images.

Image alignment task ? • It helps to be able to compare descriptors of local patches surrounding interest points (cf last lecture). • This is not strictly necessary. We will concentrate here on the geometry of the problem.

Dealing with outliers The set of putative matches still contains a very high percentage of outliers How do we fit a geometric transformation to a small subset of all possible matches? Possible strategies: • RANSAC • Incremental alignment • Hough transform • Hashing

Strategy 1: RANSAC RANSAC loop (Fischler & Bolles, 1981): • Randomly select a seed group of matches • Compute transformation from seed group • Fi d i li Find inliers to this transformation t thi t f ti • If the number of inliers is sufficiently large re compute If the number of inliers is sufficiently large, re-compute least-squares estimate of transformation on all of the inliers • Keep the transformation with the largest number of inliers inliers

RANSAC example: Translation Putative matches

RANSAC example: Translation Select one match, count inliers

RANSAC example: Translation Select one match, count inliers

RANSAC example: Translation Find “average” translation vector

Strategy 2: Incremental alignment Take advantage of strong locality constraints: only pick close-by matches to start with, and gradually add more matches in the same neighborhood Approach introduced in [Ayache & Faugeras, 1982; Hebert & Faugeras, 1983; Gaston & Lozano-Perez, 1984] Illustrated here with the method from S. Lazebnik, C. Schmid and J. Ponce, “Semi-local affine parts for object recognition” BMVC 2004 object recognition , BMVC 2004

Incremental alignment: Details Generating seed groups: • Identify triples of neighboring features ( i , j , k ) in first image y p g g ( , j , ) g • Find all triples ( i' , j' , k' ) in the second image such that i' (resp. j' , k' ) is a putative match of i (resp. j , k ), and j' , k' are neighbors of i' neighbors of i

Incremental alignment: Details A Beginning with each seed triple repeat: Beginning with each seed triple, repeat: • Estimate the aligning transformation between corresponding features in current group of matches • Grow the group by adding other consistent matches in the neighborhood Until the transformation is no longer consistent U til th t f ti i l i t t or no more matches can be found

Incremental alignment: Details A Beginning with each seed triple repeat: Beginning with each seed triple, repeat: • Estimate the aligning transformation between corresponding features in current group of matches • Grow the group by adding other consistent matches in the neighborhood Until the transformation is no longer consistent U til th t f ti i l i t t or no more matches can be found

Incremental alignment: Details A Beginning with each seed triple repeat: Beginning with each seed triple, repeat: • Estimate the aligning transformation between corresponding features in current group of matches • Grow the group by adding other consistent matches in the neighborhood Until the transformation is no longer consistent U til th t f ti i l i t t or no more matches can be found

Incremental alignment: Details A Beginning with each seed triple repeat: Beginning with each seed triple, repeat: • Estimate the aligning transformation between corresponding features in current group of matches • Grow the group by adding other consistent matches in the neighborhood Until the transformation is no longer consistent U til th t f ti i l i t t or no more matches can be found

Strategy 3: Hough transform Suppose our features are scale- and rotation-covariant • Then a single feature match provides an alignment hypothesis (translation scale orientation) (translation, scale, orientation) model David G. Lowe. “Distinctive image features from scale-invariant keypoints”, IJCV 60 (2), pp. 91-110, 2004.