Spectral Geometry of Shapes JING HUA ( ) GIL Graphics and Imaging - PowerPoint PPT Presentation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Spectral Geometry of Shapes JING HUA ( 华璟 ) GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University 3D Shape Data Polygon mesh Analytical Surface Volume Data Point Cloud

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University 3D Shape Analysis ● Shape Analysis ○ Matching, indexing, searching, retrieval, registration, etc. ● Shape Understanding ○ Pose analysis, 4D time-varying motion, etc.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Regularization and Normalization ● Voxelization ● Geometric Mapping ● Requiring correspondence between shapes

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Challenges ● Irregular sampling ● Different triangulations ● Euclidean transforms ● Non-linear deformation ○ Near-isometric deformation ○ Non-isometric deformation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Contributions ● Feature Computation in spectral domain ○ Invariant to Euclidean transformations, different triangulation, and isometric deformations ● Near-isometric pose analysis ○ Registration free semantic surface segmentation and skeletal analysis ● Non-isometric eigenvalue variation ○ Spectrum alignment for non-isometric deformations

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Outlines ● Feature computation in spectral domain ● Near-isometric pose analysis ● Non-isometric eigenvalue variation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Laplace-Beltrami Operator Laplace-Beltrami Operator is defined with the divergence of the gradient of a function on a manifold as Note the continuous function is arbitrary.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Laplace Eigenvalue Problem The Laplace-Beltrami operator introduces an eigenvalue problem On a closed manifold, the solution is a family of eigenvalues and corresponding eigenfunctions These eigenvalues only rely on the geometry.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Discrete Laplace-Beltrami Operator On triangle meshes, the Laplace-Beltrami operator can be defined on each vertex as

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Laplace Matrix Rewrite Laplace-Beltrami operator in matrix form The Laplace eigen equation becomes a matrix equation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Laplace Matrix cont. The Laplace matrix can be decomposed into two symmetric matrices where

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Laplace Matrix cont. General eigenvalue problem Inner product of vectors Eigenvectors are orthogonal to each other

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Eigenvalue and Eigenfunction of a Manifold Numb Eigen ‐ er Value 0 2.93 e ‐ 17 1 2.06 e ‐ 03 2 8.99 e ‐ 03 10 5.08 e ‐ 02 15 7.86 e ‐ 02

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Shape Spectrum The eigenvalues and the 3rd, 5th, and 10th eigenvectors of discrete Laplace matrices on different poses.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University ● Salient shape feature ● Invariant shape descriptors ● Shape matching ● Shape retrieval ● Partial matching

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Spectrum Analysis Matching signals, coarse to fine.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Shape Spectrum Projection 3D embedding can be projected onto eigenfunctions and reconstructed

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Shape Spectrum Reconstruction Geometric reconstruction with first 5, 20, 100, and 400 eigenfunctions, respectively.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Spectral Salient Features Geometry energy between Salient feature neighboring eigenfunction descriptors reconstructions

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Spectral Salient Features Salient feature points extracted in the spectral domain.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Shape Matching Shape matching between nonlinear deformations, similar shapes, and partial shapes.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Shape Retrieval Shape retrieval with spectral features

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Outlines ● Feature computation in spectral domain ● Near-isometric pose analysis ● Non-isometric eigenvalue variation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University ● Pose analysis ● Motion classification/recognition ● Motion retrieval

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Spectrum Space Global position system Absolute global position system

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Near-Isometric Deformation Transfer spatial points to spectral domain. Non- linear deformations are aligned in the spectral domain.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Spectral Part Clustering Classify rigid parts and articulated parts in the geometry spectral domain

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Pose Analysis

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Outlines ● Feature computation in spectral domain ● Near-isometric pose analysis ● Non-isometric eigenvalue variation

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Motivation ● Extend shape spectrum to non-isometric deformation ● Across object spectrum alignment algorithm ● 4D time varying motion analysis

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Shape Variation with Spectrum Eigen- Eigen- Value Value 5.29 e-17 8.53 e-17 Scale 4.31 e-03 4.23 e-03 Function 1.62 e-02 1.71 e-02 � � 3.03 e-02 3.61 e-02 3.76 e-02 5.52 e-02 3.88 e-2 5.87 e-02 … … 32

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Scale Function • On a compact closed manifold with Riemannian metric , we define a deformation function as a time variant positive � : scale function � � �� �� �� � � ��� � �� and � � N M Determining the scale function Involves derivatives

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Theorem Guarantees The Existence of Scale Function • Theorem 1. λ � is piecewise analytic and, at any regular point, the t-derivative of λ � is: Derivative of scale matrix Eigenvalue index Derivative of � eigenvalues � � � � Voronoi region Eigenfunctions Ω is a nonnegative, continuously differentiable, and diagonal matrix.

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Sketch of the Proof Eigen System � � � t-Derivative � � � � � � � Multiply � v � 1 � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Eigenvalue Variation For mapping N M, we assume that the eigenvalues change linearly � � � � 1 � � � � � � �� � � , �� 0,1 �� � � � � � � � � � � N M We divide the time interval �0,1� into � � steps which we index them as �

� � � � � GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Smoothness Constraints We focus on the global smoothness of scale factors distributed on manifold . � � � � � � � � � � � � � v �

� � GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Algorithm � � at each step can be used to calculate Ω�� � 1� using: The result Ω Theorem � �� � v � �� � � �� � v � Linear 1 � � �� � � � � � � � � � � � ⇒ Ω � � 1 � Ω � � � � � � Ω Eigenvalue Derivative � � W � v � � � 2� � � v � � � � v � Global Smoothness 10 Number of Eigenvalues = 100

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Synthetic Results: Expansion

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Synthetic Results: Shrinkage 1 0.5 0 -0.5 -1

GIL Graphics and Imaging Lab, Dept. of Computer Science, Wayne State University Synthetic Results: Scaling 1 0.5 0 -0.5 -1