What is the Range of Surface Reconstructions from a Gradient Field? - PowerPoint PPT Presentation

What is the Range of Surface Reconstructions from a Gradient Field? Writer: Amit Agrawal, Ramesh Raskar, and Rama Chellappa (ECCV 2006) Presenter: Hosna Sattar Uni Saarland Milestones and Advances in Image Analysis, 2013

Motivation Poisson solver M-estimator diffusion motivation Alpha-Surface General framework Milestones and Advances in Image Analysis, 2013 2

Gradient field and its application • Compute gradient of image • Manipulate the gradient field in order to achieve the desired goal Selection Editing Texture Seamless Cloning Flattening Milestones and Advances in Image Analysis, 2013 3 3

Integrating the Modified Gradient Field In order to integrate the gradient field it should be curl-free:      T 0 curl ( f(x)) curl (f (x), f (x)) f (x) - f (x) y yx xy x  f (x) f (x) yx xy f f Second derivatives: and . They are identical! Right: Integration of xy yx f gradient field ( , ) which is identical to original image. f xy yx Milestones and Advances in Image Analysis, 2013 4

Integrating the Modified Gradient Field • In fact, the modified gradient field might even be non-integrable!  x  y   Left: space of all solution right: add add correction  , x  y gradient field to make it integrable. Milestones and Advances in Image Analysis, 2013 5

Problem statement • A common approach to achieve the surface from the non-integrable gradient field is to minimized the last square error function:    2 2 J(Z) ((Z - p) (Z - q) )dxdy x y • The goal is to obtain surface Z. p(x,y) and q(x,y) are given non-integrable gradient field. Milestones and Advances in Image Analysis, 2013 6

Problem statement • Which can also write as:        Z p x         x            Z   q y y • The Euler-Lagrange equation gives the Poisson equation:   p     2 Z div     q Milestones and Advances in Image Analysis, 2013 7

content Poisson solver diffusion M-estimator motivation General framework Alpha-Surface Milestones and Advances in Image Analysis, 2013 8

Problem of Poisson equation • Least square solution doesn't perform well in presence of outliers: Effect of outliers in 2D integration (a) True surface (b) Reconstruction using Poisson solver. (c) If the location of outliers were known, rest of the gradients can be integrated to obtain a much better estimate. Milestones and Advances in Image Analysis, 2013 9

General framework A general solution can be obtained by minimizing the following n-th order error functional:   J E(Z, p, q, Z , p , q , . . .) dxdy a c c d b d x x x y y y     a b k, c d k - 1 for some positive integer k,    k k- 1 k- 1 Z p q    Z , p , q       a b c d c d a b c d c d x y x y x y x y x y x y    1 k n; if k 1   J E(Z, p, q, Z , Z ) dxdy y x Milestones and Advances in Image Analysis, 2013 10

General framework the Euler - Lagrange equation gives :    E E E  div( , ) (1)    Z Z Z x y   E E if we consider following form for , :   Z Z x y  E   f (Z ,Z ) f (p,q),  1 x y 3 Z x (2)  E   f (Z ,Z ) f (p,q)  2 x y 4 Z y Milestones and Advances in Image Analysis, 2013 11

General framework while the modified gradient field is curl free by substituit ing (2) in to(1) :  E   div ( f (Z ,Z ) , f (Z ,Z ) ) div(f (p,q),f (p,q))  2 x y x y 3 4 1 Z In all solutions we assume Neumann boundary conditions given by:   .  Z n 0 Milestones and Advances in Image Analysis, 2013 12

Poisson solver • To achieve Poisson equation from the general solution its just need to assume:  2 Z  div(p, q)  E  f 3 (p,q), f 4 (p,q), f (Z ,Z ) f 2 (Z ,Z ) 0 x y x y  1 Z q p Z Z x y  E   div ( f (Z ,Z ) , f (Z ,Z ) ) div(f (p,q),f (p,q))  2 x y x y 3 4 Z 1 Milestones and Advances in Image Analysis, 2013 13

content Poisson solver M-estimator diffusion motivation General framework Alpha-Surface Milestones and Advances in Image Analysis, 2013 14

Continuum of solution • Techniques for robust estimation: 1. α - Surface: Anisotropic scaling using binary weights 2. Anisotropic scaling using continuous weight 3. Affine transformation of gradient using diffusion tensor Milestones and Advances in Image Analysis, 2013 15

α - Surface • Define initial spanning tree which is all gradient correspond to edge and are inliers       x x Z p x  y      y Z q y If α =0 we get our initial spanning tree and if α= 1 we will get our poisson solver. By changing α one can trace a path in the solution space. Milestones and Advances in Image Analysis, 2013 16

α - Surface formulation The α - Surface is a weighted approach where the weight are 1 for gradients in S and otherwise zero.     b ( x , y ) 1 if Z S, 0 o.w., b (x, y) 1 if Z S, 0 o.w., x x y y    2 2 J(Z) b (Z - p) b (Z - q) dxdy x x y y Corresponding Euler_ Lagrange is:  div ( b Z , b Z ) div ( b p , b q ) x x y y x y Milestones and Advances in Image Analysis, 2013 17

Anisotropic scaling using continuous weight • M- estimator: the effect of outliers is reduced by replacing the squared error residual by another function of residual:   k 1 2 k 1 2      J(Z) w ( )( Z - p) w ( ) (Z - q) dxdy x x y y Milestones and Advances in Image Analysis, 2013 18

Affine transformation of gradient using diffusion tensor • A method for image restoration from noisy image. 2     Z p     x  D (   ) dxdy     E ( Z )   Z   q y • The Euler-Lagrange gives:   p     div ( D . Z ) div ( D )     q Milestones and Advances in Image Analysis, 2013 19

Affine transformation of gradient using diffusion tensor D is 2 × 2 symmetric , positive-definite matrix at each pixel.   d ( x , y ) d ( x , y )    11 12 D     d ( x , y ) d ( x , y ) 21 22 The gradients are scaled and lineary combined as below :       d Z d Z d p d q   11 x 12 y    11 12 div div         d Z d Z d p d q   21 x 22 y 21 22 Milestones and Advances in Image Analysis, 2013 20

results Photometric Stereo on Vase: (Top row) Noisy input images and true surface (Next two rows) Reconstructed surfaces using various algorithms. (Right Column) One-D height plots for a can line across the middle of Vase. Better results are obtained using α -surface, Diffusion and M-estimator as compared to Poisson solver, FC and Regularization Milestones and Advances in Image Analysis, 2013 21

results Photometric Stereo on Mozart: Top row shows noisy input images and the true surface. Next two rows show the reconstructed surfaces using various algorithms. (Right Column) One-D height plots for a scan line across the Mozart face. Notice that all the features of the face are preserved in the solution given by α -surface, Diffusion and M-estimator as compared to other algorithms. Milestones and Advances in Image Analysis, 2013 22

results Photometric Stereo on Flowerpot: Left column shows 4 real images of a flower pot. Right columns show the reconstructed surfaces using various algorithms. The reconstructions using Poisson solver and Frankot-Chellappa algorithm are noisy and all features (such as top of flower pot) are not recovered. Diffusion, α -surface and M-estimator methods discount noise while recovering all the salient features. Milestones and Advances in Image Analysis, 2013 23

results Milestones and Advances in Image Analysis, 2013 24

results Milestones and Advances in Image Analysis, 2013 25

reference • A. Agrawal, R. Raskar: Gradient Domain Manipulation Techniques in Vision and Graphices. ICCV 2007 Course • Advanced Image Analysis, Lecture 9 by Dr. Christian Schmaltz • http://www.cfar.umd.edu/~aagrawal/eccv06/RangeofSurfaceReconstruct ions.html Milestones and Advances in Image Analysis, 2013 26

content Poisson solver M-estimator diffusion motivation General framework Alpha-Surface Milestones and Advances in Image Analysis, 2013 27

Question?