3D Vision Andreas Geiger, Torsten Sattler Spring 2017 Schedule - PowerPoint PPT Presentation

3D Vision Andreas Geiger, Torsten Sattler Spring 2017

Schedule Feb 20 Introduction Feb 27 Geometry, Camera Model, Calibration Mar 6 Features, Tracking / Matching Mar 13 Project Proposals by Students Mar 20 Structure from Motion (SfM) + papers Mar 27 Dense Correspondence (stereo / optical flow) + papers Apr 3 Bundle Adjustment & SLAM + papers Apr 10 Student Midterm Presentations Apr 17 Easter Break Apr 24 Multi-View Stereo & Volumetric Modeling + papers May 1 Labor Day May 8 3D Modeling with Depth Sensors + papers May 15 3D Scene Understanding + papers May 22 4D Video & Dynamic Scenes + papers May 29 Student Project Demo Day = Final Presentations

3D Vision– Class 2 Projective Geometry and Camera model points, lines, planes conics and quadrics transformations camera model Read tutorial chapter 2 and 3.1 http://www.cs.unc.edu/~marc/tutorial/ Chapters 1, 2 and 5 in Hartley and Zisserman 1 st edition Or Chapters 2, 3 and 6 in 2 nd edition

Overview • 2D Projective Geometry • 3D Projective Geometry • Camera Models & Calibration

2D Projective Geometry? Projections of planar surfaces A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images . PhD Thesis 1999.

2D Projective Geometry? Measure distances A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images . PhD Thesis 1999.

2D Projective Geometry? Discovering details Piero della Francesca, La Flagellazione di Cristo (1460) A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images . PhD Thesis 1999.

2D Projective Geometry? Image Stitching

2D Projective Geometry? Image Stitching

2D Euclidean Transformations

Homogeneous Coordinates

Homogeneous Coordinates

Point – Line Duality

Lines to Points, Points to Lines Intersections of lines Line through two points

Ideal Points and the Line at Infinity • Intersections of parallel lines? ( ) T • Parallel lines intersect in Ideal Points x 1 , x 2 ,0

Ideal Points and the Line at Infinity • Ideal points correspond to directions • Unaffected by translation

Ideal Points and the Line at Infinity • Lines through two ideal points? ( ) T l  • Line at infinity intersects all ideal points 0 , 0 , 1  Note that in P 2 there is no distinction   2 2 P R l  between ideal points and others

Conics • Curve described by 2 nd -degree equation in the plane or homogenized or in matrix form { } • 5DOF (degrees of freedom): (defined up to scale) a : b : c : d : e : f

Five Points Define a Conic For each point the conic passes through 2 + bx i y i + cy i 2 + dx i + ey i + f  0 ax i or ( ) ( ) T c  a , b , c , d , e , f  2 2 x , x y , y , x , y , 1 c 0 i i i i i i stacking constraints yields   2 2 x 1 x 1 y 1 y 1 x 1 y 1 1   2 2 x 2 x 2 y 2 y 2 x 2 y 2 1     c  0 2 2 x 3 x 3 y 3 y 3 x 3 y 3 1   2 2 x 4 x 4 y 4 y 4 x 4 y 4 1     2 2  x 5 x 5 y 5 y 5 x 5 y 5 1 

Tangent Lines to Conics The line l tangent to C at point x on C is given by l = Cx l x C

Dual Conics l T C * l  0 • A line tangent to the conic C satisfies C *  C -1 • In general ( C full rank): • Dual conics = line conics = conic envelopes

Degenerate Conics • A conic is degenerate if matrix C is not of full rank m e.g. two lines (rank 2) l C  lm T + ml T e.g. repeated line (rank 1) C  ll T l • Degenerate line conics: 2 points (rank 2), double point (rank1) *  C ( ) C * • Note that for degenerate conics

2D Projective Transformations Definition: A projectivity is an invertible mapping h from P 2 to itself such that three points x 1 , x 2 , x 3 lie on the same line if and only if h (x 1 ), h (x 2 ), h (x 3 ) do. Theorem: A mapping h : P 2  P 2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P 2 represented by a vector x it is true that h (x)= H x Definition: Projective transformation       x ' 1 h 11 h 12 h 13 x 1        x'  H x x ' 2 h 21 h 22 h 23 x 2       or 8DOF        x ' 3   h 31 h 32 h 33   x 3  projectivity = collineation = proj. transformation = homography

Hierarchy of 2D Transformations transformed invariants squares   h h h 11 12 13 Concurrency, collinearity,   Projective order of contact (intersection, h h h   21 22 23 tangency, inflection, etc.), 8dof   h h h cross ratio   31 32 33 Parallellism, ratio of areas,   a a t ratio of lengths on parallel 11 12 x   Affine lines (e.g. midpoints), linear a a t   6dof 21 22 y combinations of vectors   0 0 1 (centroids),   The line at infinity l ∞   sr sr t 11 12 x Ratios of lengths, angles,   Similarity sr sr t The circular points I,J   4dof 21 22 y   0 0 1     r r t 11 12 x   Euclidean Absolute lengths, angles, r r t   3dof 21 22 y areas   0 0 1  

Transformation of 2D Points, Lines and Conics • For a point transformation x'  H x • Transformation for lines l'  H -T l • Transformation for conics C '  H -T CH -1 • Transformation for dual conics C ' *  HC * H T

Application: Removing Perspective Two stages: • From perspective to affine transformation via the line at infinitiy • From affine to similarity transformation via the circular points

The Line at Infinity The line at infinity l  =(0,0,1) T is a fixed line under a projective transformation H if and only if H is an affinity (affine transformation)   0     T A 0      T l H l 0 l       A   T T t A 1    1   Note: not fixed pointwise

Affine Properties from Images projection affine rectification affine transform   T , l 3  0 l   l l 2 l 3 1

Affine Rectification v 1 l ∞ v 2 l   v 1  v 2 l 1 l 3 v 1  l 1  l 2 v 2  l 3  l 4 l 2 l 4   1 0 0   0 1 0      l l 2 l 3  1

The Circular Points The circular points I, J are fixed points under the projective transformation H iff H is a similarity         s cos s sin t 1 1     x             i I H I s sin s cos t i se i I       S y       0 0 1 0 0      

The Circular Points • every circle intersects l ∞ at the “circular points” l ∞ 2 + x 2 2 + dx 1 x 3 + ex 2 x 3 + fx 3 2  0 2 + x 2 2  0 x 1 x 1 x 3  0 ( ) T I  1, i ,0 ( ) T J  1,- i ,0 • Algebraically, encodes orthogonal directions T + i 0,1,0 ( ) ( ) T I  1,0,0

Conic Dual to the Circular Points l ∞   1 0 0     + * * T T C IJ JI C 0 1 0 I   ∞ ∞   0 0 0   J ∞  T * * C H C H S S ∞ * C The dual conic is fixed conic under the ∞ projective transformation H iff H is a similarity * C Note: has 4DOF (det = 0) ∞ l ∞ is the nullvector

Measuring Angles via the Dual Conic ( ) ( ) T T l  l m  m 1 , m 2 , m 3 • Euclidean: 1 , l 2 , l 3 1 m 1 + l 2 m 2 l cos   2 + l 2 2 + m 2 ( ) m 1 ( ) 2 2 l 1 l T C  * m cos   • Projective: ( ) m T C  ( ) l T C  * l * m l T C  * m  0 (orthogonal) • Once we know dual conic on projective plane, we can measure Euclidean angles!

Metric Rectification A -1 • Dual conic under affinity         *   A t I 0 A T  AA T 0 0    C      0 T 0 T t T 0 T  1   0   1   0  • S=AA - T symmetric, estimate from two pairs of l T C  * m  0 orthogonal lines (due to ) Note: Result defined up to similarity

Update to Euclidean Space • Metric space: Measure ratios of distances • Euclidean space: Measure absolute distances • Can we update metric to Euclidean space? • Not without additional information

Overview • 2D Projective Geometry • 3D Projective Geometry • Camera Models & Calibration

3D Points and Planes • 2D: duality point - line, 3D: duality point - plane • Homogeneous representation of 3D points and planes π X + π X + π X + π X = 0 1 1 2 2 3 3 4 4 • The point X lies on the plane π if and only if π T X = 0 • The plane π goes through the point X if and only if π T X = 0