Paul Debevec, 3D Photography Modeling and Rendering Architecture Modeling and Rendering Architecture from Photographs from Photographs Paul Debevec Paul Debevec Computer Science Division Computer Science Division University of California at Berkeley University of California at Berkeley SIGGRAPH 99 Course #28, 3D Photography SIGGRAPH 99 Course #28, 3D Photography Brian Curless and Steve Seitz, organizers Brian Curless and Steve Seitz, organizers August 9, 1999 August 9, 1999 http://www.cs.berkeley.edu/~debevec http://www.cs.berkeley.edu/~debevec SIGGRAPH 99 Course #28 4-1
Paul Debevec, 3D Photography Stereo Stereo Image Image Pair Pair Stereo Image Capture Stereo Image Capture Depth Depth Rig Rig Map Map Immersion ‘94 Immersion ‘94 Michael Naimark Naimark Michael John John Woodfill Woodfill Synthetic Synthetic Paul Debevec Paul Debevec Views Views Leo Leo Villareal Villareal Ramin Zabih Ramin Zabih Interval Research Interval Research Corporation Corporation Ramin Zabih and John Woodfill. Non-parametric local transforms for determining visual correspondence. ECCV, May 1994. Structure from Motion Structure from Motion Tomasi and Kanade 1992 Tomasi and Kanade 1992 Image from sequence Recovered model Scene viewed from Image from sequence Recovered model Scene viewed from same position same position SIGGRAPH 99 Course #28 4-2
Paul Debevec, 3D Photography Structure from Motion Structure from Motion Taylor and Kriegman Taylor and Kriegman 1995 1995 Taylor and Kriegman 1995 Recovered model Recovered model Two of eight original images Two of eight original images Tomasi and Tomasi and Kanade Kanade 1992 1992 Taylor and Kriegman Taylor and Kriegman 1995 1995 Image from sequence Image from sequence One of eight images One of eight images Recovered model Recovered model Recovered model Recovered model SIGGRAPH 99 Course #28 4-3
Paul Debevec, 3D Photography Façade’s Modeling Method: The user represents the scene as a The user represents the scene as a collection of geometric primitives collection of geometric primitives The computer solves for the sizes and The computer solves for the sizes and positions of the blocks according to positions of the blocks according to user-supplied edge correspondences user-supplied edge correspondences Modeling and Rendering Architecure from Photographs Modeling and Rendering Architecure from Photographs (Debevec, Taylor, and Malik 1996) (Debevec, Taylor, and Malik 1996) Block Model Block Model User User- -Marked Edges Marked Edges Recovered Model Recovered Model SIGGRAPH 99 Course #28 4-4
Paul Debevec, 3D Photography Façade Blocks Façade Blocks Parameterized Block Parameterized Block SIGGRAPH 99 Course #28 4-5
Paul Debevec, 3D Photography Parameter Parameter References References Model Model Hierarchy Hierarchy Relation can be: Relation can be: * Arbitrary 6 DOF * Arbitrary 6 DOF * Fixed Rotation * Fixed Rotation * Fixed Translation * Fixed Translation * Geometric Relationship * Geometric Relationship SIGGRAPH 99 Course #28 4-6
Paul Debevec, 3D Photography Reconstruction Algorithm Reconstruction Algorithm An objective function O measures the An objective function O measures the misalignment between the marked edges misalignment between the marked edges and the corresponding projected edges of and the corresponding projected edges of the model the model O is minimized with respect to the model O is minimized with respect to the model parameters and camera positions parameters and camera positions An initial estimate is obtained by a separate An initial estimate is obtained by a separate procedure procedure Completed Completed Reconstruction Reconstruction and Reprojection and Reprojection Marked Edge Marked Edge Projected Model Model Edge Model Edge Error Area Error Area SIGGRAPH 99 Course #28 4-7
Paul Debevec, 3D Photography Algorithm with Initial Estimate Procedure Algorithm with Initial Estimate Procedure 1. Solve for camera rotations, independently, 1. Solve for camera rotations, independently, based on edge orientations based on edge orientations 2. Hold camea rotations fixed; solve for other 2. Hold camea rotations fixed; solve for other parameters (often linear) parameters (often linear) 3. Perform full non-linear optimization, 3. Perform full non-linear optimization, starting from near the solution starting from near the solution Video Video SIGGRAPH 99 Course #28 4-8
Paul Debevec, 3D Photography Photogrammetric Modeling Summary Modeling with blocks works Modeling with blocks works because: because: Convenient for architecture Convenient for architecture Recovers Complete Models Recovers Complete Models Reduces number of model parameters, e.g. Reduces number of model parameters, e.g. Campanile model has: Campanile model has: 2,896 parameters as independent edges 2,896 parameters as independent edges 240 parameters as independent blocks 240 parameters as independent blocks 33 parameters as constrained blocks 33 parameters as constrained blocks • → Few marked features required • → Few marked features required • → Easier to solve • → Easier to solve Surfaces of Revolution Surfaces of Revolution Synthetic View Synthetic View Photograph Photograph Recovered Model Recovered Model SIGGRAPH 99 Course #28 4-9
Paul Debevec, 3D Photography Arches and Arches and Surfaces of Revolution Surfaces of Revolution Taj Mahal Taj Mahal modeled from modeled from one photograph one photograph Image-Based Modeling, Rendering, Image-Based Modeling, Rendering, and Lighting and Lighting SIGGRAPH 99 Course #39 SIGGRAPH 99 Course #39 SIGGRAPH 99 Course #39 Tuesday, August 10, 1999 Tuesday, August 10, 1999 Tuesday, August 10, 1999 Room 152, Los Angeles Convention Center Room 152, Los Angeles Convention Center Room 152, Los Angeles Convention Center 8:30am - - 5:00pm 5:00pm 8:30am 8:30am - 5:00pm Paul Debevec Michael Cohen Paul Debevec Michael Cohen Michael Cohen UC Berkeley Microsoft Research UC Berkeley Microsoft Research Microsoft Research Leonard McMillan Leonard McMillan Chris Bregler Chris Chris Bregler Bregler MIT Stanford University Stanford University MIT Stanford University Richard Richard Szeliski Richard Szeliski Szeliski François Sillion François François Sillion Sillion iMAGIS - GRAVIR/IMAG Microsoft Research Microsoft Research Microsoft Research iMAGIS - GRAVIR/IMAG SIGGRAPH 99 Course #28 4-10
Paul Debevec, 3D Photography Rendering with Projective Texture Mapping Rendering with Projective Texture Mapping View- -Dependent Weighting Dependent Weighting View Function Function SIGGRAPH 99 Course #28 4-11
Paul Debevec, 3D Photography Scene with Geometric Detail Scene with Geometric Detail Model-Based Model-Based Stereo Stereo Approximate Block Model Approximate Block Model Model-Based Stereo Given a key and an offset image, Given a key and an offset image, • Project the offset image onto the model • Project the offset image onto the model • View the model through the key camera • View the model through the key camera → Warped offset image → Warped offset image Stereo becomes feasible between key and Stereo becomes feasible between key and warped offset images because: warped offset images because: • Disparities are small • Disparities are small • Foreshortening is greatly reduced • Foreshortening is greatly reduced SIGGRAPH 99 Course #28 4-12
Paul Debevec, 3D Photography Key Image Key Image Warped Offset Image Warped Offset Image Offset Image Offset Image Disparity Map Disparity Map Synthetic Views Synthetic Views of Refined Model of Refined Model Four images composited with Four images composited with Model-Based Stereo and Model-Based Stereo and VDTM VDTM SIGGRAPH 99 Course #28 4-13
Paul Debevec, 3D Photography Application: Rouen Revisited Application: Rouen Revisited (Golan Levin and Paul Debevec) (Golan Levin and Paul Debevec) SIGGRAPH 96 Art Show SIGGRAPH 96 Art Show Synthetic View: Synthetic View: Synthetic View: Synthetic View: Synthetic View: Synthetic View: 1996 1896 Monet Painting 1996 1896 Monet Painting (Uncalibrated ( Uncalibrated Views) Views) Video Video SIGGRAPH 99 Course #28 4-14
Recommend
More recommend
