Computing and Processing Correspondences with Functional Maps - PowerPoint PPT Presentation

Computing and Processing Correspondences with Functional Maps SIGGRAPH 2017 course Maks Ovsjanikov, Etienne Corman, Michael Bronstein, Emanuele Rodolà, Mirela Ben-Chen, Leonidas Guibas, Frederic Chazal, Alex Bronstein

General Overview Overall Objective : Create tools for computing and analyzing mappings between geometric objects. 2

General Overview Overall Objective : Create tools for computing and analyzing mappings between geometric objects. Rather than comparing points on objects it is often easier to compare real-valued functions defined on them. Such maps can be represented as matrices. 3

Course Overview Course Website: http://www.lix.polytechnique.fr/~maks/fmaps_SIG17_course/ or http://bit.do/fmaps2017 Course Notes: Linked from the website. Or use http://bit.do/fmaps2017_notes Attention: (significantly) more material than in the lectures Sample Code: See Sample Code link on the website. New: demo code for basic operations. 4

Course Schedule 2:00pm – 2:45pm Introduction (Maks) • Introduction to the functional maps framework. 2:50pm – 3:35pm Computing Functional Maps (Etienne) • Optimization methods for functional map estimation. 3:40pm - 4:25pm Functional Vector Fields (Miri) • From functional maps to tangent vector fields and back. 4:30pm - 5:15pm Maps in Shape Collections (Leo) • Networks of maps, shape variability, learning. 5:15pm - Wrapup, Q&A (all) 5

What is a Shape? Continuous: a surface embedded in 3D. Discrete: a graph embedded in 3D (triangle mesh). Common assumptions: • Connected. • Manifold. • Without Boundary. 6

What is a Shape? Continuous: a surface embedded in 3D. Discrete: a graph embedded in 3D (triangle mesh). 5k – 200k triangles 7 Shapes from the FAUST, SCAPE, and TOSCA datasets

Overall Goal Given two shapes, find correspondences between them. Finding the best map between a pair of shapes. 8

Why Shape Matching? Given a correspondence, we can transfer: texture and parametrization segmentation and labels deformation Sumner et al. ‘04. Other applications: shape interpolation, reconstruction ... 9

Rigid Shape Matching Given a pair of shapes, find the optimal Rigid Alignment between them. The unknowns are the rotation/translation parameters of the source onto the target shape. 10

Iterative Closest Point (ICP) Classical approach: iterate between finding correspondences and finding the transformation: example in 2D N M Given a pair of shapes, and , iterate: N M 1. For each find nearest neighbor . y i ∈ N x i ∈ M 2. Find optimal transformation minimizing: R, t X k Rx i + t − y i k 2 arg min 11 2 R,t i

Iterative Closest Point Classical approach: iterate between finding correspondences and finding the transformation: N M Given a pair of shapes, and , iterate: N M 1. For each find nearest neighbor . y i ∈ N x i ∈ M 2. Find optimal transformation minimizing: R, t X k Rx i + t − y i k 2 arg min 12 2 R,t i

Iterative Closest Point Classical approach: iterate between finding correspondences and finding the transformation: N M Given a pair of shapes, and , iterate: N M 1. For each find nearest neighbor . y i ∈ N x i ∈ M 2. Find optimal transformation minimizing: R, t X k Rx i + t − y i k 2 arg min 13 2 R,t i

Iterative Closest Point Classical approach: iterate between finding correspondences and finding the transformation: N M Given a pair of shapes, and , iterate: N M 1. For each find nearest neighbor . y i ∈ N x i ∈ M 2. Find optimal transformation minimizing: R, t X k Rx i + t − y i k 2 arg min 14 2 R,t i

Iterative Closest Point Classical approach: iterate between finding correspondences and finding the transformation: N M Given a pair of shapes, and , iterate: N M 1. For each find nearest neighbor . y i ∈ N x i ∈ M 2. Find optimal transformation minimizing: R, t X k Rx i + t − y i k 2 arg min 15 2 R,t i

Iterative Closest Point Classical approach: iterate between finding correspondences and finding the transformation: N M 1. Finding nearest neighbors: can be done with space- partitioning data structures (e.g., KD-tree). 2. Finding the optimal transformation minimizing: R, t X k Rx i + t − y i k 2 arg min 2 R ∈ SO(3) , t ∈ R 3 i Arun et al., Least- Can be done efficiently via SVD decomposition. Squares Fitting of Two 3-D Point Sets

Non-Rigid Shape Matching Unlike rigid matching with rotation/translation, there is no compact representation to optimize for in non-rigid matching. 17

Non-Rigid Shape Matching Main Questions: What does it mean for a correspondence to be “good”? How to compute it efficiently in practice? 18

Isometric Shape Matching Deformation Model: Good maps must preserve geodesic distances. N d N ( T ( x ) , T ( y )) d M ( x, y ) M Geodesic: length of shortest path lying entirely on the surface. 19

Isometric Shape Matching Approach: N d N ( T ( x ) , T ( y )) d M ( x, y ) M Find the point mapping by minimizing the distance distortion: k d M ( x, y ) − d N ( T ( x ) , T ( y )) k X T opt = arg min T x,y The unknowns are point correspondences. 20

Isometric Shape Matching Approach: N d N ( T ( x ) , T ( y )) d M ( x, y ) M Find the point mapping by minimizing the distance distortion: k d M ( x, y ) − d N ( T ( x ) , T ( y )) k X T opt = arg min T x,y Problem: The space of possible solutions is highly non-linear, non-convex.

Functional Map Representation We would like to define a representation of shape maps that is more amenable to direct optimization. 1. A compact representation for “natural” maps. 2. Inherently global and multi-scale. 3. Handles uncertainty and ambiguity gracefully. 4. Allows efficient manipulations (averaging, composition). 5. Leads to simple (linear) optimization problems. 22

Functional Approach to Mappings Given two shapes and a pointwise map T : N → M N T M The map induces a functional correspondence: T T F ( f ) = g, where g = f ◦ T 23 Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas , SIGGRAPH 2012

Functional Approach to Mappings Given two shapes and a pointwise map T : N → M T F ( f ) = g : N → R T F f : M → R The map induces a functional correspondence: T T F ( f ) = g, where g = f ◦ T 24 Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas , SIGGRAPH 2012

Functional Approach to Mappings Given two shapes and a pointwise map T : N → M T F ( f ) = g : N → R T F f : M → R The map induces a functional correspondence: T T F ( f ) = g, where g = f ◦ T 25 Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, Butscher, Guibas , SIGGRAPH 2012

Functional Approach to Mappings Given two shapes and a pointwise map T : N → M T F ( f ) = g : N → R T F f : M → R The induced functional correspondence is linear: T F ( α 1 f 1 + α 2 f 2 ) = α 1 T F ( f 1 ) + α 2 T F ( f 2 ) 26

Functional Map Representation Given two shapes and a pointwise map T : N → M T F ( f ) = g : N → R T F f : M → R The induced functional correspondence is complete. 27

Observation Assume that both: f ∈ L 2 ( M ) , g ∈ L 2 ( N ) g : N → R T F f : M → R N M Express both and in terms of basis functions: f T F ( f ) X a i φ M X b j φ N f = g = T F ( f ) = i j i j { a i } { b j } Since is linear, there is a linear transformation from to . T F

Functional Map Representation Choice of Basis: Eigenfunctions of the Laplace-Beltrami operator: ∆ φ i = λ i φ i ∆( f ) = − div r ( f ) Generalization of Fourier bases to surfaces. Ordered by eigenvalues and provide a natural notion of scale . 29 λ 0 = 0 λ 1 = 2 . 6 λ 2 = 3 . 4 λ 3 = 5 . 1 λ 4 = 7 . 6

Functional Map Representation Choice of Basis: Eigenfunctions of the Laplace-Beltrami operator: ∆ φ i = λ i φ i Generalization of Fourier bases to surfaces. Ordered by eigenvalues and provide a natural notion of scale . Can be computed efficiently, with a sparse matrix eigensolver. 30

Functional Map Representation Since the functional mapping T F is linear: T F ( α 1 f 1 + α 2 f 2 ) = α 1 T F ( f 1 ) + α 2 T F ( f 2 ) T F = T F can be represented as a matrix C , given a choice of basis for function spaces. Functional maps: a flexible representation of maps between shapes, O., Ben-Chen, Solomon, 31 Butscher, Guibas , SIGGRAPH 2012

Functional Map Definition Functional map: matrix C that translates coefficients from to . Φ N Φ M 32

Example 1 N T M Given two shapes with points and a map: T : N → M n M , n N T : n N × n M matrix encoding the map T , one 1 per column with zeros everywhere else. If functions are represented as vectors (in the hat basis), the functional map is given by matrix-vector product: g = T T f C = T T 33