Geometric Registration for Deformable Shapes 2.2 Deformable - PowerPoint PPT Presentation

Geometric Registration for Deformable Shapes 2.2 Deformable Registration Variational Model · Deformable ICP

Variational Model What is deformable shape matching?

Example ? What are the Correspondences? 3 Eurographics 2010 Course – Geometric Registration for Deformable Shapes 3

What are we looking for? Problem Statement: ? f Given: S 2 • Two surfaces S 1 , S 2 ⊆ ℝ 3 S 1 We are looking for: • A reasonable deformation function f : S 1 → ℝ 3 that brings S 1 close to S 2 4 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example ? Correspondences? no shape match too much deformation optimum 5 Eurographics 2010 Course – Geometric Registration for Deformable Shapes 5

This is a Trade-Off Deformable Shape Matching is a Trade-Off: • We can match any two shapes using a weird deformation field • We need to trade-off:  Shape matching (close to data)  Regularity of the deformation field (reasonable match) 6 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Variational Model Components: Matching Distance: Deformation / rigidity: 7 Eurographics 2010 Course – Geometric Registration for Deformable Shapes 7

Variational Model Variational Problem: • Formulate as an energy minimization problem: = + ( ) ( ) match regularize r ( ) ( ) ( ) E f E f E f 8 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Part 1: Shape Matching Assume: • Objective Function: S 2 ( ) = E match ( ) ( f ) dist f ( S ), S 1 , 2 1 2 f ( S 1 ) • Example: least squares distance ∫ = ( ) 2 match ( ) ( , ) E f dist S d x x 1 2 1 ∈ x S 1 1 • Other distance measures: Hausdorf distance, L p -distances, etc. • L 2 measure is frequently used (models Gaussian noise) 9 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Point Cloud Matching Implementation example: Scan matching • Given: S 1 , S 2 as point clouds (1) , …, s n (1) }  S 1 = { s 1 (2) s i (2) , …, s m (2) }  S 2 = { s 1 • Energy function: f i ( S 1 ) ( ) m | | S ∑ 2 = ( ) ( 2 ) match 1 ( ) , E f dist S s 1 i m = 1 i ( ) • How to measure ? , dist S x 1  Estimate distance to a point sampled surface 10 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Surface approximation (2) s i f ( S 1 ) Solution #1: Closest point matching • “Point-to-point” energy ( ) m | | S ∑ 2 = ( ) ( 2 ) ( 2 ) match 1 ( ) , ( ) E f dist s NN s i in S i m 1 = 1 i 11 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Surface approximation (2) s i f ( S 1 ) Solution #2: Linear approximation • “Point-to-plane” energy • Fit plane to k -nearest neighbors • k proportional to noise level, typically k ≈ 6…20 12 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Surface approximation (2) s i f ( S 1 ) Solution #3: Higher order approximation • Higher order fitting (e.g. quadratic)  Moving least squares 13 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Variational Model Variational Problem: • Formulate as an energy minimization problem: = + ( ) ( ) match regularize r ( ) ( ) ( ) E f E f E f 14 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Part II: Deformation Model What is a “nice” deformation field? ( ) regularize r ( ) E f • Isometric “elastic” energies  Extrinsic (“volumetric deformation”)  Intrinsic (“as-isometric-as possible embedding”) • Thin shell model  Preserves shape (metric plus curvature ) • Thin-plate splines  Allowing strong deformations, but keep shape 15 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Elastic Volume Model Extrinsic Volumetric “As-Rigid-As Possible” • Embed source surface S 1 in volume • f should preserve 3 × 3 metric tensor (least squares) [ ] ∫ 2 = ∇ ∇ − ( regularize r ) T E ( f ) f f I dx V 1 first fundamental form I ( ℝ 3 × 3 ) ∇ f f V 1 ambient space f (V 1 ) S 1 S 2 16 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Volume Model Variant: Thin-Plate-Splines • Use regularizer that penalizes curved deformation ∫ = ( regularize r ) 2 E ( f ) H ( x ) dx f V second derivative ( ℝ 3 × 3 ) 1 H f = ∇ ( ∇ f ) f V 1 ambient space f (V 1 ) S 1 S 2 17 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

How does the deformation look like? as-rigid-as possible volume thin plate original splines Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Isometric Regularizer Intrinsic Matching (2-Manifold) • Target shape is given and complete • Isometric embedding [ ] ∫ 2 = ∇ ∇ − ( regularize r ) T E ( f ) f f I dx first fund. form (S 1 , intrinsic) S 1 ∇ f f tangent space S 1 S 2 19 19 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Elastic “Thin Shell” Regularizer “Thin Shell” Energy I II • Differential geometry point of view S 1 f  Preserve first fundamental form I I  Preserve second fundamental form II II  …in a least least squares sense S 2 • Complicated to implement • Usually approximated  Volumetric shells (as shown before)  Other approximation (next slide) 20 20 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example Implementation Example: approximate thin shell model • Keep locally rigid  Will preserve metric & curvature implicitly • Idea  Associate local rigid transformation with surface points  Keep as similar as possible  Optimize simultaneously with deformed surface • Transformation is implicitly defined by deformed surface ( and vice versa ) 21 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Parameterization Parameterization of S 1 • Surfel graph • This could be a mesh, but does not need to edges encode topology surfel graph 22 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformation frame t frame t+1 prediction A i A i Orthonormal Matrix A i per surfel (neighborhood), latent variable 23 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformation frame t frame t+1 prediction A i A i Orthonormal Matrix A i per surfel (neighborhood), latent variable error [ ] ( ) ( ) 2 ∑ ∑ + + = − − − ( regularize r ) t ( t ) ( t ) ( t 1 ) ( t 1 ) E A s s s s i i i i i j j surfels neighbors 24 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Unconstrained Optimization Orthonormal matrices • Local, 1st order, non-degenerate parametrization: α β  0  A = A exp( C )   × i 0 ( t ) i C = − α 0 γ   × i ⋅ ( t )   = I + C A ( ) − β − γ 0 ×   0 i • Optimize parameters α , β , γ , then recompute A 0 • Compute initial estimate using [ Horn 87 ] 25 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Variational Model Variational Problem: • Formulate as an energy minimization problem: = + ( ) ( ) match regularize r ( ) ( ) ( ) E f E f E f 26 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformable ICP

Deformable ICP How to build a deformable ICP algorithm • Pick a surface distance measure • Pick an deformation model / regularizer = + ( ) ( ) match regularize r ( ) ( ) ( ) E f E f E f 28 28 Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformable ICP How to build a deformable ICP algorithm • Pick a surface distance measure • Pick an deformation model / regularizer • Initialize f (S 1 ) with S 1 (i.e., f = id) • Pick a non-linear optimization algorithm  Gradient decent (easy, but bad performance)  Preconditioned conjugate gradients (better)  Newton or Gauss Newton (recommended, but more work)  Always use analytical derivatives! • Run optimization Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example Example • Elastic model • Local rigid coordinate frames • Align A → B, B → A 30 Eurographics 2010 Course – Geometric Registration for Deformable Shapes