Single-View Geometry EECS 442 – Prof. David Fouhey Winter 2019, University of Michigan http://web.eecs.umich.edu/~fouhey/teaching/EECS442_W19/
Application: Single-view modeling A. Criminisi, I. Reid, and A. Zisserman, Single View Metrology, IJCV 2000
Application: Measuring Height
Application: Measuring Height • CSI before CSI • Covered criminal cases talking to random scientists (e.g., footwear experts) • How do you tell how tall someone is if they’re not kind enough to stand next to a ruler?
Application: Camera Calibration • Calibration a HUGE pain …
Application: Camera Calibration • What if 3D coordinates are unknown? • Use scene features such as vanishing points Slide from Efros, Photo from Criminisi
Camera calibration revisited • What if 3D coordinates are unknown? • Use scene features such as vanishing points Vertical vanishing point (at infinity) Vanishing line Vanishing Vanishing point point Slide from Efros, Photo from Criminisi
Recall: Vanishing points image plane vanishing point v camera center line in the scene • All lines having the same direction share the same vanishing point
Calibration from vanishing points Consider a scene with 3 orthogonal directions v 1 , v 2 are finite vps, v 3 infinite vp Want to align world coordinates with directions . . v 1 v 2 v 3
Calibration from vanishing points 𝑸 3𝑦4 ≡ 𝒒 1 𝒒 2 𝒒 𝟒 𝒒 4 It turns out that 𝒒 𝟐 ≡ 𝑸 1,0,0,0 𝑈 VP in X direction 𝒒 𝟑 ≡ 𝑸 0,1,0,0 𝑈 VP in Y direction 𝒒 𝟒 ≡ 𝑸 0,0,1,0 𝑈 VP in Z direction 𝒒 𝟓 ≡ 𝑸 0,0,0,1 𝑈 Projection of origin Note the usual ≡ (i.e., all of this is up to scale) as well as the 0 for the vps
Calibration from vanishing points • Let’s align the world coordinate system with the three orthogonal vanishing directions: 1 0 0 𝒇 𝟐 = 𝒇 𝟑 = 𝒇 𝟒 = 0 1 0 0 0 1 𝜇𝒘 𝒋 = 𝑳[𝑺, 𝒖] 𝒇 𝒋 0 𝜇𝒘 𝒋 = 𝑳𝑺𝒇 𝑗 Drop the t 𝑺 −𝟐 𝑳 −𝟐 𝜇𝒘 𝑗 = 𝒇 𝑗 Inverses
Calibration from vanishing points So 𝒇 𝒋 = 𝑺 −𝟐 𝑳 −𝟐 𝜇𝒘 𝑗 , but who cares? What are some properties of axes? 𝑼 𝒇 𝒌 = 0 for 𝑗 ≠ 𝑘 , so K, R have to satisfy Know 𝒇 𝒋 𝑼 𝑺 −𝟐 𝑳 −𝟐 𝜇 𝑗 𝒘 𝑗 = 𝟏 𝑺 −𝟐 𝑳 −𝟐 𝜇 𝑘 𝒘 𝒌 𝑼 𝑺 𝑼 𝑳 −𝟐 𝜇 𝑗 𝒘 𝑗 = 𝟏 𝑆 −1 = 𝑆 𝑈 𝑺 𝑼 𝑳 −𝟐 𝜇 𝑘 𝒘 𝒌 𝑼 𝑺 𝑼 𝑳 −𝟐 𝒘 𝑗 = 𝟏 𝜇 𝑗 𝜇 𝑘 𝑺 𝑼 𝑳 −𝟐 𝒘 𝒌 Move scalars 𝒘 𝒌 𝑳 −𝑼 𝑺𝑺 𝑼 𝑳 −𝟐 𝒘 𝒋 = 𝟏 Clean up 𝑆𝑆 𝑈 = 𝐽 𝒘 𝒌 𝑳 −𝑼 𝑳 −𝟐 𝒘 𝒋 = 𝟏
Calibration from vanishing points • Intrinsics (focal length f, principal point u 0 ,v 0 ) have to ensure that the rays corresponding to supposedly orthogonal vanishing points are orthogonal 𝒘 𝒌 𝑳 −𝑼 𝑳 −𝟐 𝒘 𝒋 = 𝟏
Calibration from vanishing points Cannot recover focal Can solve for focal length, principal length, principal point is point the third vanishing point
Directions and vanishing points Given vanishing point 𝒘 camera calibration 𝑳 : 𝑳 −𝟐 𝒘 is direction corresponding to that vanishing point. −1 𝑔 0 0 𝒘 𝟒 0 𝑔 0 0 0 1 −1 𝑔 0 0 0 . . v 1 v 2 10 10 0 𝑔 0 1 0 0 1 1/𝑔 0 0 0 0 10 10 /𝑔 10 10 = 0 1/𝑔 0 1 1 v 3 0 0 1 0 0 10 6 /𝑔 →≈ 1 1 0
Directions and vanishing points
Directions and vanishing points v 2 v 1 v 3
Directions and vanishing points If 𝒘 vanishing point, and 𝑳 the camera intrinsics, 𝑳 −𝟐 𝒘 is the corresponding direction. v 2 v 1 [-f,0] [f,0] Set 𝑣 0 , 𝑤 0 = 0,0 𝐿 −1 = −1 𝑔 0 0 1/𝑔 0 0 = 0 𝑔 0 0 1/𝑔 0 [0,∞] 0 0 1 0 0 1 v 3
Directions and vanishing points If 𝒘 vanishing point, and 𝑳 the camera intrinsics, 𝑳 −𝟐 𝒘 is the corresponding direction. v 2 v 1 [-f,0] [f,0] K -1 v 2 = [1,0,1] K -1 v 1 = [-1,0,1] 1/𝑔 0 0 [0,∞] K -1 v 3 = [0,∞,1] 𝐿 −1 = 0 1/𝑔 0 v 3 0 0 1
Rotation from vanishing points Know that 𝜇 𝑗 𝒘 𝒋 = 𝑳𝑺𝒇 𝒋 and have K , but want R So: 𝜇𝑳 −𝟐 𝒘 𝑗 = 𝑺𝒇 𝒋 What does 𝑺𝒇 𝒋 look like? 1 𝑺𝒇 𝟐 = 𝒔 𝟐 𝒔 𝟑 𝒔 𝟒 = 𝒔 𝟐 0 0 The ith column of R is a scaled version of 𝒔 𝒋 = 𝜇𝑳 −𝟐 𝒘 𝒋
Calibration from vanishing points • Solve for K (focal length, principal point) using 3 orthogonal vanishing points • Get rotation directly from vanishing points once calibration matrix known • Pros: • Could be totally automatic! • Cons: • Need 3 vanishing points, estimated accurately, but with at least two finite!
Finding Vanishing Points What might go wrong with the circled points?
Finding Vanishing Points • Find edges 𝐹 = {𝑓 1 , … , 𝑓 𝑜 } • All 𝑜 2 intersections of edges 𝑤 𝑗𝑘 = 𝑓 𝑗 × 𝑓 𝑘 are potential vanishing points • Try all triplets of popular vanishing points, check if the camera’s focal length, principal point “make sense” • What are some options for this?
Finding Vanishing Points
Measuring height Slide by Steve Seitz
Measuring height Slide by Steve Seitz
Measuring height 5.3 5 Camera height 4 3.3 3 2.8 2 1
Measuring height without a ruler Z O ground plane Compute Z from image measurements • Need more than vanishing points to do this
Projective invariant • We need to use a projective invariant : a quantity that does not change under projective transformations (including perspective projection)
Projective invariant • We need to use a projective invariant : a quantity that does not change under projective transformations (including perspective projection) • The cross-ratio of four points: − − P P P P 3 1 4 2 − − P P P P 3 2 4 1 P 4 This is one of the P 3 cross-ratios (can P 2 reorder arbitrarily) P 1
Measuring height − − T B R H = − − R B T R scene cross ratio T (top of object) − − t b v r H t Z = − − r R (reference point) r b v t R Z C H image cross ratio b R v Z B (bottom of object) ground plane
Measuring height without a ruler
v z r vanishing line (horizon) t 0 t v x v y v H R H b 0 b − − t b v r H = Z − − r b v t R Z image cross ratio
Remember This? • Line equation: 𝑏𝑦 + 𝑐𝑧 + 𝑑 = 0 • Vector form: 𝒎 𝑈 𝒒 = 0 , 𝒎 = [𝑏, 𝑐, 𝑑] , 𝐪 = [𝑦, 𝑧, 1] • Line through two points? • 𝒎 = 𝒒 𝟐 × 𝒒 𝟑 • Intersection of two lines? • 𝒒 = 𝒎 𝟐 × 𝒎 𝟑 • Intersection of two parallel lines is at infinity
v z r vanishing line (horizon) t 0 t v x v y v H R H b 0 b − − t b v r H = Z − − r b v t R Z image cross ratio
Examples A. Criminisi, I. Reid, and A. Zisserman, Single View Metrology, IJCV 2000 Figure from UPenn CIS580 slides
Another example • Are the heights of the two groups of people consistent with one another? Piero della Francesca , Flagellation, ca. 1455 A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of paintings, Proc. Computers and the History of Art , 2002
Measurements on planes 4 3 2 1 1 2 3 4
Measurements on planes 4 3 2 p′ 1 p 1 2 3 4
Image rectification: example Piero della Francesca , Flagellation, ca. 1455
Application: 3D modeling from a single image A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of paintings, Proc. Computers and the History of Art , 2002
Application: 3D modeling from a single image J. Vermeer, Music Lesson , 1662 A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the analysis of paintings, Proc. Computers and the History of Art , 2002
Application: Object Detection (1,1) v 0 h v (0,0) 𝑧 𝑝𝑐𝑘𝑓𝑑𝑢 ≈ ℎ𝑧 𝑑𝑏𝑛𝑓𝑠𝑏 “Reasonable” approximation: 𝑤 0 − 𝑤
Application: Object detection
Application: Object detection
Application: Image Editing K. Karsch and V. Hedau and D. Forsyth and D. Hoiem, Rendering Synthetic Objects into Legacy Photographs, SIGGRAPH Asia 2011
Application: Estimating Layout V. Hedau, D. Hoiem, D. Forsyth Recovering the spatial layout of cluttered rooms ICCV 2009
Unsupervised Learning Can we learn 3D simply from regularities? Image Collection … D.F. Fouhey, W. Hussain, A. Gupta, M. Hebert. Single Image 3D without a Single 3D Image. ICCV 2015.
Unsupervised Learning Can we learn 3D simply from regularities? Image Tools From Collection Geometry + … Vanishing Points D.F. Fouhey, W. Hussain, A. Gupta, M. Hebert. Single Image 3D without a Single 3D Image. ICCV 2015.
Recommend
More recommend
