Shape from Shading INEL 6088 Computer Vision Radiometry Image - PowerPoint PPT Presentation

Shape from Shading INEL 6088 Computer Vision

Radiometry � Image irradiance E: power of the light/unit area at each point p of the image plane � Scene radiance L: power of the light/unit area ideally emitted at each point P of the surface in 3D space in a given direction d . � Surface reflectance model: model of the way the surface reflects light � Lambertian model: assumes that each surface point appears equally bright from all viewing directions. Approximate the behavior of rough surfaces, matte paint, paper, etc. INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry � The irradiance at a point in the image plane E(x’,y’) is determined by the amount of energy radiated by the corresponding point in the scene L(x,y,z) radiated in the direction of the image point; E(x’,y’) = L(x,y,z, θ ’, φ ’) � To find the image irradiance: � Trace the ray back to the surface patch from which the ray was emitted � Understand how is the light from the scene illumination is reflected by the surface patch INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Illustration of the basic radiometric concepts. INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry Bidirectional reflectance distribution function ( BRDF ): energy radiated from surface patch in some direction BRDF = -------------------------------------------------------------------------- energy arriving at the surface patch from some direction θ , φ = angles in polar coordinate system L( θ e , φ e )=f( θ i , φ i , θ e , φ e ) I( θ i , φ i ) Scene radiance = BRDF × Scene irradiance INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry 
 Reflectance � For most materials, the BRDF depends only on the difference between incident and emitted angles f( θ i , φ i , θ e , φ e ) = f( θ i - θ e , φ i - φ e ) � Total irradiance on a surface patch = sum of contributions arriving at the surface patch from all directions in the hemisphere INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry 
 Image Irradiance Note: book uses Φ instead of ψ Solid angle: projection of surface patch into unit sphere; units are steradians Projected solid angle: INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry 
 Radiance � Unit for measuring the distribution of light in space: � radiance = power (amount of energy per unit time) traveling at some point in a specified direction, per unit area perpendicular to the direction of travel , per unit solid angle. Units are watts per square meter per steredian. INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry � I( θ i , φ i ) = radiance per unit solid angle passing through the hemisphere in the direction ( θ i , φ i ) � Total incident illumination on the surface patch � This is the sum of the radiance coming from all directions in the hemisphere � The additional cosine term is added to account for the fact that an inclined patch looks smaller. This is called foreshortening . INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry 
 Radiance Total radiance reflected by solid patch: The BRDF for a Lamberdian surface is a constant: Where I 0 is the total illumination on the surface patch. INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Radiometry 
 Reflectance � Four types � Lambertian or diffuse � Specular � Combined Lambertian & Specular � Scanning microscope INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Lambertian Reflectance For illumination from a point � source in a direction ( θ s , φ s ) Added so that when I is integrated over Radiance from the surface patch: hemisphere the total illumination is I 0 � The incident angle dependence is caused by foreshortening of the surface patch relative to the direction of illumination, measured with respect to the surface normal. For uniform illumination (instead of point source): L = I 0 INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Specular Reflectance � Reflected rays make the same angle than incident rays with the surface normal, but on the opposite side: Φ e = Φ i + π � BRDF is given by: Like in a mirror… Combined Specular & Lambertian: The factor η controls the mixture. INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Surface Orientation We measure image irradiance – must be related to surface reflectance � and illumination Coordinate system so far was centered on the surface patch � Must be expressed in terms of camera coordinate system � INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Point position is (x,y,z) in camera coordinates. Let depth be a function of image plane coordinates: z = z(x,y) A nearby point appears in the image plane at (x+ δ x, y+ δ y) . The change in depth that correspond to the change in (x,y) position is Where p and q define the gradient of the surface at (x,y,z). The surface normal is = unit surface normal INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

This expression simply says that the amount of displacement in x and y � corresponding to a unit change in depth is p and q, respectively. The orientation of any surface is then specified by the two functions � p=p ( x , y ) and q = q ( x , y ) If a scene patch is illuminated by a point source, it’s radiance is given by � � Using the camera coordinate system, the surface normal is P = (- p ,- q ,1) and the direction of the light source is P S =(- p s ,- q s , 1) � cos θ s = P • P S /| P || P s | INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Reflectance Map The equations from the previous slide can be combined to obtain, given the light source distribution, the reflectance for all surface orientations p and q . These can be computed to yield the reflectance map: INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Assume • a Lambertian surface, • point source illumination, • θ S represent the angle between the surface’s patch normal and the vector direction of the illumination source. Then our previous result L ( θ e , φ e ) = I 0 π cos θ S applies. Observing that the cosine of θ S is the dot product of the surface’s patch normal and the vector direction of the illumination source � � � � 1 1 − p, − q, − p S , − q S , cos θ S = 1 + p 2 + q 2 · p p 1 + p 2 S + q 2 S 1 + p S p + q S q = 1 + p 2 + q 2 p p 1 + p 2 S + q 2 S INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

• Normalising the maximum of the reflectance map to 1 to eliminate its dependence on source intensity, light gathering capability of lens, etc., and • assuming that scene radiance equals image irradiance, we can write that E( x , y ) = R( p , q ) Meaning: image brightness at point (x,y) = reflectance map value for the surface orientation p and q of the corresponding point in the scene. Thus for a Lambertian reflector and a point source, INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Shape from Shading The Reflectance map captures the image intensity at a pixel as a function of � surface orientation For � � fixed illumination and imaging conditions � known surface reflectance properties � changes in surface orientation translate into image intensity changes Shape from shading: inverse problem of recovering surface orientation � from image intensities. Approach: � � image irradiance = E(x,y) = R(p,q) = surface reflectance map calculate p and q for each image point (x,y) � INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Shape from Shading Solve for p and q from E( x , y ) � One equation & two unknowns ( p and q ) – need further constrains � One possibility: Impose smoothness constrain and solve by calculus of � variations (see textbook) to get an iterative solution: where • * denotes the average value computed from 2 × 2 neighborhood (i.e. between neighbors at (i+1,j), (i-1,j), (i,j+1) and (i, j-1) • Subscripts i,j denote the discrete image coordinates INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Photometric Stereo � The idea is to use several sources instead of one to solve shape from shading problem � The relationship between image brightness and the reflectance map is modified to include the albedo factor, ρ E( x , y )= ρ R( p , q ) where 0 < ρ <1 incorporates the fact that not all incident light is radiated from the surface � Let Unit vector indicating the Surface normal direction of the incident light. Assume 3 light sources INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

Photometric Stereo (cont.) Image irradiance Matrix of the direction vectors for the 3 point sources Vector of the three image irradiance measurements Image irradiance equations Solution: INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM

INEL 6088 Computer Vision - Shading Fall 2018 - ECE Dept. UPRM