Geometric vision Goal: Recovery - PowerPoint PPT Presentation

•וארטס לט לש םיפקשה לע ססובמרנסה

Geometric vision • Goal: Recovery of 3D structure  What cues in the image allow us to do this? Slide credit: Svetlana Lazebnik

Visual Cues • Shading Merle Norman Cosmetics, Los Angeles 3 B. Leibe Slide credit: Steve Seitz

Visual Cues • Shading • Texture The Visual Cliff , by William Vandivert, 1960 4 B. Leibe Slide credit: Steve Seitz

Visual Cues • Shading • Texture • Focus From The Art of Photography , Canon 5 B. Leibe Slide credit: Steve Seitz

Visual Cues • Shading • Texture • Focus • Perspective 6 B. Leibe Slide credit: Steve Seitz

Visual Cues • Shading • Texture • Focus • Perspective Figures from L. Zhang • Motion 7 Slide credit: Steve Seitz, Kristen Grauman http://www.brainconnection.com/teasers/?main=illusion/motion-shape

Our Goal: Recovery of 3D Structure • We will focus on perspective and motion • We need multi-view geometry because recovery of structure from one image is inherently ambiguous X? X? X? x 8 B. Leibe Slide credit: Svetlana Lazebnik

To Illustrate This Point… • Structure and depth are inherently ambiguous from single views. 9 B. Leibe Slide credit: Svetlana Lazebnik, Kristen Grauman

Stereo Vision Perceptual and Sensory Augmented Computing Computer Vision WS 08/09 http://www.well.com/~jimg/stereo/stereo_list.html Slide credit: Kristen Grauman

What Is Stereo Vision? • Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape 11 B. Leibe Slide credit: Svetlana Lazebnik, Steve Seitz

What Is Stereo Vision? • Generic problem formulation: given several images of the same object or scene, compute a representation of its 3D shape 12 B. Leibe Slide credit: Svetlana Lazebnik, Steve Seitz

What Is Stereo Vision? • Narrower formulation: given a calibrated binocular stereo pair, fuse it to produce a depth image Image 1 Image 2 Dense depth map 13 B. Leibe Slide credit: Svetlana Lazebnik, Steve Seitz

Geometry for a Simple Stereo System • First, assuming parallel optical axes, known camera parameters (i.e., calibrated cameras): 14 B. Leibe Slide credit: Kristen Grauman

World point Depth of p image point image point (right (left) Focal length optical center optical center (right) (left) baseline 15 B. Leibe Slide credit: Kristen Grauman

Geometry for a Simple Stereo System • Assume parallel optical axes, known camera parameters (i.e., calibrated cameras). We can triangulate via: Similar triangles (p l , P , p r ) an (O l , P , O r ):   T x x T  l r  Z f Z T  Z f  x x r l disparity 16 B. Leibe Slide credit: Kristen Grauman

קמועו Disparity תומלצמהמ קחרמ ( depth ) תונומתב הדוקנה ימוקימב לדבה ) ( disparity T  תומלצמ Z f  x x r l disparity

Depth From Disparity Image I(x,y) Disparity map D(x,y) Image I´(x´,y´) (x´,y´)=(x+D(x,y), y) 18 B. Leibe

General Case With Calibrated Cameras • The two cameras need not have parallel optical axes. vs. 19 B. Leibe Slide credit: Kristen Grauman, Steve Seitz

Stereo Correspondence Constraints • Given p in the left image, where can the corresponding point p’ in the right image be? 20 B. Leibe Slide credit: Kristen Grauman

Stereo Correspondence Constraints • Given p in the left image, where can the corresponding point p’ in the right image be? 21 B. Leibe Slide credit: Kristen Grauman

Stereo Correspondence Constraints 22 B. Leibe Slide credit: Kristen Grauman

Stereo Correspondence Constraints • Geometry of two views allows us to constrain where the corresponding pixel for some image point in the first view must occur in the second view. epipolar line epipolar line epipolar plane • Epipolar constraint: Why is this useful?  Reduces correspondence problem to 1D search along conjugate epipolar lines. 23 B. Leibe Slide adapted from Steve Seitz

Epipolar Geometry • Epipolar Plane • Baseline • Epipoles • Epipolar Lines 24 Slide adapted from Marc Pollefeys

Epipolar Geometry: Terms • Baseline : line joining the camera centers • Epipole : point of intersection of baseline with the image plane • Epipolar plane : plane containing baseline and world point • Epipolar line : intersection of epipolar plane with the image plane 25 B. Leibe Slide credit: Marc Pollefeys

Epipolar Constraint • Potential matches for p have to lie on the corresponding epipolar line l’. • Potential matches for p’ have to lie on the corresponding epipolar line l. http://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.html 28 B. Leibe Slide credit: Marc Pollefeys

Example 29 B. Leibe Slide credit: Kristen Grauman

• For a given stereo rig, how do we express the epipolar constraints algebraically? 33 B. Leibe

תיחרכהה הצירטמה תיינב P p p l   r l r O O l r   • רידגנ   T O O r l םוקימ ןיב רשקה , • בוביס תצירטמ רובע P R אוה תינמיל תילאמשה תוטנידרואוקה תכרעמב:     P R P T r l

Rotation Matrix Express 3d rotation as series of rotations around coordinate axes by angles    , , Overall rotation is product of these elementary rotations: R  R R R x y z Slide credit: Kristen Grauman

תיחרכהה הצירטמה תיינב l  • םירוטקוה תשולש ,ו- לע םיאצמנ T ( P T ) P l רושימה ותוא :רושימה ירלופיפאה P p p l   r l r O O l r

Cross Product • Vector cross product takes two vectors and returns a third vector that’s perpendicular to both inputs. • So here, c is perpendicular to both a and b, which means the dot product = 0. Slide credit: Kristen Grauman

תיחרכהה הצירטמה תיינב l  • םירוטקוה תשולש ,ו- לע םיאצמנ T ( P T ) P l רושימה ותוא :רושימה ירלופיפאה   • רושימל בצינה רוטקו אוה  T P l P     •ןאכמ:    T P T T P 0 l l p   •ו תויה- p l     r P R P T l r r l •יזא:   T O O R P P T l r r l •לבקנו האושמב ביצנ:   T     T T R P T P P RT P 0 r l r l

Matrix Form of Cross Product 39 Slide credit: Kristen Grauman

תיחרכהה הצירטמה תיינב   T     T T R P T P P RT P 0 r l r l •תוצירטמ לפכ תועצמאב בתכשנ     – T    T T R P T P P R T P 0 r l r x l P   – רידגנ  E R T x l  p –לבקנו : T P EP 0 p l   r r l r תארקנ • הצירטמה E O O l r תיחרכהה הצירטמה ) ( Essential Matrix

תיחרכהה הצירטמה • תונותנ תונומתה ירושימב תודוקנו תויה דע תוהז ןכש ( ב- תוטנידרואוקבמוה,' ףילחנ p P עובקב לפכ ידכל) l  T p Ep 0 r • רשיה אוה תינמיה הנומתה רושימב רשי  u Ep r l הדוקנה תא ליכמ יכ חטבומ רשא p r תודוקנ תוטנידרואוק ונל תונותנ רשאכ תישומיש • E רושימב הנומתה. •הנומתב םילסקיפ תוטנידרואוק שי ונל...

Essential Matrix Example: Parallel Cameras  R I    T [ d , 0 , 0 ]   0 0 0 E [T ]R x 0 0 d 0 – d 0   Ep  p 0 For the parallel cameras, image of any point must lie on same horizontal line in each image plane. Slide credit: Kristen Grauman

Essential Matrix Example: Parallel Cameras  R I    T [ d , 0 , 0 ]   0 0 0 E [T ]R x 0 0 d 0 – d 0   Ep  p 0 For the parallel cameras, image of any point must lie on same horizontal line in each image plane. Slide credit: Kristen Grauman

More General Case Image I(x,y) Disparity map D(x,y) Image I´(x´,y´) (x´,y´)=(x+D(x,y), y) What about when cameras’ optical axes are not parallel? Slide credit: Kristen Grauman

וארטס תכרעמ לויכ – התידוסיה הצירטמ

תידוסיה הצירטמה l  T p Fp 0 r • תידוסיה הצירטמ Fundamental Matrix F •ו םעפה ךא תידוסיה הצירטמל הייפואב המוד- p p l r תוטנידרואוקבםילסקיפ •    T 1 F M EM r l • ו רובע- יתש לש תוימינפ תוצירטמ M M l l תומלצמה • תועצמאב תידוסיה הצירטמה בושיח" גלא ' הנומש תודוקנה "

Fundamental matrix • Relates pixel coordinates in the two views • More general form than essential matrix: we remove need to know intrinsic parameters • If we estimate fundamental matrix from correspondences in pixel coordinates, can reconstruct epipolar geometry without intrinsic or extrinsic parameters Grauman