Joint diagonalization problem o ff ( P > Λ M ,k 0 P ) + o ff ( Q > Λ N ,k 0 Q ) + µ k P > A � Q > B k 2 , 1 min P , Q P > P = I Q > Q = I s . t . O ff -diagonal elements penalty o ff ( X ) = P i 6 = j x 2 ij Dirichlet energy o ff ( X ) = trace( X ) for k 0 > k If Frobenius norm is used and k 0 = k , due to rotation invariance C = QP > is the functional correspondence matrix Robust norm k X k 2 , 1 = P j k x j k 2 allows coping with outliers Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 7/66
Example of joint diagonalization Isometric Elements of P > Λ M ,k 0 P and Q > Λ N ,k 0 Q Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 8/66
Example of joint diagonalization Isometric Non-isometric Elements of P > Λ M ,k 0 P and Q > Λ N ,k 0 Q Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 8/66
Example of joint diagonalization Mesh with 8.5K vertices Mesh with 850 vertices Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 9/66
Example of joint diagonalization Mesh with 8.5K vertices Point cloud with 850 vertices Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 10/66
Choice of the basis Functional correspondence matrix C expressed in standard Laplacian eigenbases Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 11/66
Choice of the basis Functional correspondence matrix C expressed in coupled approximate eigenbases Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013 11/66
Multiple shapes C ij A i ⇡ A j M j M p M i M 1 M 2 Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013; Kovnatsky, Glasho ff , Bronstein 2016 12/66
Multiple shapes P > i A i ⇡ P > j A j M j M p M i M 1 M 2 Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013; Kovnatsky, Glasho ff , Bronstein 2016 12/66
Multiple shapes P > i A i ⇡ P > j A j P j M j P i P p M p M i P 1 P 2 M 1 M 2 Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013; Kovnatsky, Glasho ff , Bronstein 2016 12/66
Multiple shapes p X X trace( P > k P > i A i � P > min i Λ M i P i ) + µ j A j k P 1 ,..., P p i =1 i 6 = j P > s . t . i P i = I ‘Synchronization problem’ Matrices P 1 , . . . , P p orthogonally align the p eigenbases Kovnatsky, Bronstein 2 , Glasho ff , Kimmel 2013; Kovnatsky, Glasho ff , Bronstein 2016 13/66
Computing Functional Maps with Manifold Optimization 14/66
trace( P > Λ P ) + µ k PA � B k P > P = I min s . t . P 15/66
trace( P > Λ P ) + µ k PA � B k P > P = I min s . t . P Optimization on the Stiefel manifold of orthogonal matrices 15/66
Manifold optimization toy example: eigenvalue problem x 2 R 3 x > Ax x > x = 1 min s . t . Minimization of a quadratic function on the sphere 16/66
Manifold optimization toy example: eigenvalue problem x 2 S (3 , 1) x > Ax min Minimization of a quadratic function on the sphere 16/66
Optimization on the manifold: main idea X ( k ) X ( k +1) S Absil et al. 2009 17/66
Optimization on the manifold: main idea r f ( X ( k ) ) X ( k ) P X ( k ) r S f ( X ( k ) ) T X ( k ) S S Absil et al. 2009 17/66
Optimization on the manifold: main idea r f ( X ( k ) ) X ( k ) P X ( k ) α ( k ) r S f ( X ( k ) ) T X ( k ) S S Absil et al. 2009 17/66
Optimization on the manifold: main idea r f ( X ( k ) ) X ( k ) P X ( k ) α ( k ) r S f ( X ( k ) ) T X ( k ) S R X ( k ) X ( k +1) S Absil et al. 2009 17/66
Optimization on the manifold repeat Compute extrinsic gradient r f ( X ( k ) ) Projection: r S f ( X ( k ) ) = P X ( k ) ( r f ( X ( k ) )) Compute step size α ( k ) along the descent direction �r S f ( X ( k ) ) Retraction: X ( k +1) = R X ( k ) ( � α ( k ) r S f ( X ( k ) )) k k + 1 until convergence ; Absil et al. 2009; Boumal et al. 2014 18/66
Optimization on the manifold repeat Compute extrinsic gradient r f ( X ( k ) ) Projection: r S f ( X ( k ) ) = P X ( k ) ( r f ( X ( k ) )) Compute step size α ( k ) along the descent direction �r S f ( X ( k ) ) Retraction: X ( k +1) = R X ( k ) ( � α ( k ) r S f ( X ( k ) )) k k + 1 until convergence ; Projection P and retraction R operators are manifold-dependent Absil et al. 2009; Boumal et al. 2014 18/66
Optimization on the manifold repeat Compute extrinsic gradient r f ( X ( k ) ) Projection: r S f ( X ( k ) ) = P X ( k ) ( r f ( X ( k ) )) Compute step size α ( k ) along the descent direction �r S f ( X ( k ) ) Retraction: X ( k +1) = R X ( k ) ( � α ( k ) r S f ( X ( k ) )) k k + 1 until convergence ; Projection P and retraction R operators are manifold-dependent Typically expressed in closed form Absil et al. 2009; Boumal et al. 2014 18/66
Optimization on the manifold repeat Compute extrinsic gradient r f ( X ( k ) ) Projection: r S f ( X ( k ) ) = P X ( k ) ( r f ( X ( k ) )) Compute step size α ( k ) along the descent direction �r S f ( X ( k ) ) Retraction: X ( k +1) = R X ( k ) ( � α ( k ) r S f ( X ( k ) )) k k + 1 until convergence ; Projection P and retraction R operators are manifold-dependent Typically expressed in closed form “Black box”: need to provide only f ( X ) and gradient r f ( X ) Absil et al. 2009; Boumal et al. 2014 18/66
trace( P > Λ P ) + µ k PA � B k 2 P > P = I min s . t . 2 P Optimization on the Stiefel manifold 19/66
trace( P > Λ P ) P > P = I min + µ k PA � B k 2 , 1 s . t . P | {z } | {z } smooth non-smooth Non-smooth optimization on the Stiefel manifold 19/66
Manifold ADMM (MADMM) min f ( X ) + g ( X ) X 2 S ( n,k ) | {z } | {z } smooth non-smooth Hestenes 1969; Powell 1969; Kovnatsky, Glasho ff , Bronstein 2016 20/66
Manifold ADMM (MADMM) min f ( X ) + g ( Z ) s . t . Z = X X 2 S ( n,k ) | {z } |{z} smooth non-smooth Z 2 R n ⇥ k Hestenes 1969; Powell 1969; Kovnatsky, Glasho ff , Bronstein 2016 20/66
Manifold ADMM (MADMM) min f ( X ) + g ( Z ) s . t . Z = X X 2 S ( n,k ) | {z } |{z} smooth non-smooth Z 2 R n ⇥ k Apply the method of multipliers only to the constraint Z = X 2 k X � Z + U k 2 min f ( X ) + g ( Z ) + ρ F X 2 S ( n,k ) Z 2 R n ⇥ k Solve alternating w.r.t. X and Z and updating U U + X � Z Hestenes 1969; Powell 1969; Kovnatsky, Glasho ff , Bronstein 2016 20/66
Manifold ADMM (MADMM) min f ( X ) + g ( Z ) s . t . Z = X X 2 S ( n,k ) | {z } |{z} smooth non-smooth Z 2 R n ⇥ k Apply the method of multipliers only to the constraint Z = X 2 k X � Z + U k 2 min f ( X ) + g ( Z ) + ρ F X 2 S ( n,k ) Z 2 R n ⇥ k Solve alternating w.r.t. X and Z and updating U U + X � Z Problem breaks into Smooth manifold optimization sub-problem w.r.t. X , and Non-smooth unconstrained sub-problem w.r.t. Z Hestenes 1969; Powell 1969; Kovnatsky, Glasho ff , Bronstein 2016 20/66
Manifold ADMM (MADMM) Initialize k 1 , Z (1) = X (1) , U (1) = 0 . repeat X -step: X ( k +1) = argmin 2 k X � Z ( k ) + U ( k ) k 2 f ( X ) + ρ F X 2 S Z -step: Z ( k +1) = argmin 2 k X ( k +1) � Z + U ( k ) k 2 g ( Z ) + ρ F Z Update U ( k +1) = U ( k ) + X ( k +1) � Z ( k +1) k k + 1 until convergence ; Kovnatsky, Glasho ff , Bronstein 2016 21/66
Manifold ADMM (MADMM) Initialize k 1 , Z (1) = X (1) , U (1) = 0 . repeat X -step: X ( k +1) = argmin 2 k X � Z ( k ) + U ( k ) k 2 f ( X ) + ρ F X 2 S Z -step: Z ( k +1) = argmin 2 k X ( k +1) � Z + U ( k ) k 2 g ( Z ) + ρ F Z Update U ( k +1) = U ( k ) + X ( k +1) � Z ( k +1) k k + 1 until convergence ; Solver/number of optimization iterations in X - and Z -steps Kovnatsky, Glasho ff , Bronstein 2016 21/66
Manifold ADMM (MADMM) Initialize k 1 , Z (1) = X (1) , U (1) = 0 . repeat X -step: X ( k +1) = argmin 2 k X � Z ( k ) + U ( k ) k 2 f ( X ) + ρ F X 2 S Z -step: Z ( k +1) = argmin 2 k X ( k +1) � Z + U ( k ) k 2 g ( Z ) + ρ F Z Update U ( k +1) = U ( k ) + X ( k +1) � Z ( k +1) k k + 1 until convergence ; Solver/number of optimization iterations in X - and Z -steps X -step and X -step in some problems have a closed form Kovnatsky, Glasho ff , Bronstein 2016 21/66
Manifold ADMM (MADMM) Initialize k 1 , Z (1) = X (1) , U (1) = 0 . repeat X -step: X ( k +1) = argmin 2 k X � Z ( k ) + U ( k ) k 2 f ( X ) + ρ F X 2 S Z -step: Z ( k +1) = argmin 2 k X ( k +1) � Z + U ( k ) k 2 g ( Z ) + ρ F Z Update U ( k +1) = U ( k ) + X ( k +1) � Z ( k +1) k k + 1 until convergence ; Solver/number of optimization iterations in X - and Z -steps X -step and X -step in some problems have a closed form Parameter ρ > 0 can be chosen fixed or adapted Kovnatsky, Glasho ff , Bronstein 2016 21/66
L 2 vs L 2 , 1 data term Least squares Robust (MADMM) Correspondence computed with data containing 10% outliers Kovnatsky, Glasho ff , Bronstein 2016 22/66
Partial Functional Maps 23/66
Partial Laplacian eigenvectors ζ 2 ζ 3 ζ 4 ζ 5 ζ 6 ζ 7 ζ 8 ζ 9 ψ 2 ψ 3 ψ 4 ψ 5 ψ 6 ψ 7 ψ 8 ψ 9 φ 2 φ 3 φ 4 φ 5 φ 6 φ 7 φ 8 φ 9 Laplacian eigenvectors of a shape with missing parts (Neumann boundary conditions) Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 24/66
Partial Laplacian eigenvectors Functional correspondence matrix C Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 25/66
Perturbation analysis: intuition ¯ ¯ ¯ ∆ ¯ φ 1 φ 2 φ 3 M ∆ M φ 1 φ 2 φ 3 ¯ ¯ ¯ λ 1 λ 2 λ 3 M λ 1 λ 2 λ 3 ¯ M φ 1 φ 2 φ 3 ∆ M ¯ ¯ ¯ ∆ ¯ φ 1 φ 2 φ 3 M ¯ ¯ ¯ λ 1 λ 1 λ 2 λ 3 λ 2 λ 3 Ignoring boundary interaction: disjoint parts (block-diagonal matrix) Eigenvectors = Mixture of eigenvectors of the parts Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 26/66
Perturbation analysis: eigenvalues 8 . 00 · 10 � 2 M 6 . 00 4 . 00 r k N 2 . 00 0 . 00 10 20 30 40 50 eigenvalue number k ≈ |M| Slope r |N | (depends on the area of the cut) Consistent with Weyl’s law Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 27/66
Perturbation analysis: details ∆ M M t E ¯ ∆ M + t D M M t E > ∆ ¯ M + t D ¯ M ∆ ¯ M Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 28/66
Perturbation analysis: boundary interaction strength 20 10 Value of f Eigenvector perturbation depends on length and position of the boundary R Perturbation strength ≤ c ∂ M f ( m ) dm , where n ✓ φ i ( m ) φ j ( m ) ◆ 2 X f ( m ) = λ i � λ j i,j =1 j 6 = i Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 29/66
Partial functional maps Model shape M Query shape N T F Part M ✓ M ⇡ isometric to N Part Data f 1 , . . . , f q 2 L 2 ( N ) M Query N g 1 , . . . , g q 2 L 2 ( M ) Partial functional map ( T F f i )( m ) ⇡ g i ( m ) , m 2 M Model M Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 30/66
Partial functional maps Model shape M Query shape N T F Part M ✓ M ⇡ isometric to N Part Data f 1 , . . . , f q 2 L 2 ( N ) v Query N g 1 , . . . , g q 2 L 2 ( M ) Partial functional map T F f i ⇡ g i · v, v : M ! [0 , 1] Model M Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 30/66
Partial functional maps Model shape M Query shape N C Part M ✓ M ⇡ isometric to N Part Data f 1 , . . . , f q 2 L 2 ( N ) v Query N g 1 , . . . , g q 2 L 2 ( M ) Partial functional map ⇡ B ( v ) , v : M ! [0 , 1] CA � � h φ N A = i , f j i L 2 ( N ) � � h φ M Model M B ( v ) = i , g j · v i L 2 ( M ) Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 30/66
Partial functional maps Model shape M Query shape N C Part M ✓ M ⇡ isometric to N Part Data f 1 , . . . , f q 2 L 2 ( N ) v Query N g 1 , . . . , g q 2 L 2 ( M ) Partial functional map ⇡ B ( v ) , v : M ! [0 , 1] CA � � h φ N A = i , f j i L 2 ( N ) � � h φ M Model M B ( v ) = i , g j · v i L 2 ( M ) Optimization problem w.r.t. correspondence C and part v min C ,v k CA � B ( v ) k 2 , 1 + ρ corr ( C ) + ρ part ( v ) Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 30/66
Partial functional maps min C ,v k CA � B ( v ) k 2 , 1 + ρ corr ( C ) + ρ part ( v ) Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 31/66
Partial functional maps min C ,v k CA � B ( v ) k 2 , 1 + ρ corr ( C ) + ρ part ( v ) Part regularization Area preservation R M v ( m ) dx ≈ |N| Spatial regularity = small boundary length (Mumford-Shah) a, Cosmo, Bronstein, Torsello, Cremers 2016; Bronstein 2 2008 Rodol` 31/66
Partial functional maps min C ,v k CA � B ( v ) k 2 , 1 + ρ corr ( C ) + ρ part ( v ) Part regularization Area preservation R M v ( m ) dx ≈ |N| Spatial regularity = small boundary length (Mumford-Shah) Correspondence regularization Slanted diagonal structure Approximate ortho-projection ( C > C ) i 6 = j ≈ 0 rank( C ) ≈ r a, Cosmo, Bronstein, Torsello, Cremers 2016; Bronstein 2 2008 Rodol` 31/66
Structure of partial functional correspondence 4 2 0 0 20 40 60 80 100 C > C C W singular values Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 32/66
Alternating minimization C -step: fix v ⇤ , solve for correspondence C C k CA � B ( v ⇤ ) k 2 , 1 + ρ corr ( C ) min v -step: fix C ⇤ , solve for part v k C ⇤ A � B ( v ) k 2 , 1 + ρ part ( v ) min v Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 33/66
Alternating minimization C -step: fix v ⇤ , solve for correspondence C C k CA � B ( v ⇤ ) k 2 , 1 + ρ corr ( C ) min v -step: fix C ⇤ , solve for part v k C ⇤ A � B ( v ) k 2 , 1 + ρ part ( v ) min v Iteration 1 2 3 4 Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 33/66
Example of convergence Time (sec.) 0 5 10 15 20 25 10 10 C -step 10 9 v -step 10 8 Energy 10 7 10 6 10 5 10 4 0 20 40 60 80 100 Iteration Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 34/66
Examples of partial functional maps Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 35/66
Examples of partial functional maps Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 35/66
Examples of partial functional maps Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 35/66
Examples of partial functional maps Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 35/66
Partial functional maps vs Functional maps 100 150 100 80 50 % Correspondences PFM 60 Func. maps 40 50 100 20 150 0 0 0.05 0.1 0.15 0.2 0.25 Geodesic error Correspondence performance for di ff erent basis size k Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 36/66
Partial correspondence performance Cuts Holes 100 % Correspondences 80 60 40 20 0 0 0.05 0.1 0.15 0.2 0.25 0 0.05 0.1 0.15 0.2 0.25 Geodesic Error Geodesic Error PFM RF IM EN GT SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 ( PFM ); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 37/66
Partial correspondence performance Cuts Holes 1 Mean geodesic error 0 . 8 0 . 6 0 . 4 0 . 2 0 20 40 60 80 20 40 60 80 Partiality (%) Partiality (%) PFM RF IM EN GT SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 ( PFM ); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 38/66
Geometric deep learning + Partial functional maps Correspondence 10% 0 Correspondence error Boscaini, Masci, Rodol` a, Bronstein 2016 39/66
Geometric deep learning + Partial functional maps Correspondence 10% 0 Correspondence error Boscaini, Masci, Rodol` a, Bronstein 2016 40/66
Geometric deep learning + Partial functional maps 7.5% 0 Pointwise geodesic error (in % of geodesic diameter) Monti, Boscaini, Masci, Rodol` a, Svoboda, Bronstein 2016 41/66
Geometric deep learning + Partial functional maps Reference Correspondence visualization (similar colors encode corresponding points) Training: FAUST / Testing: FAUST Monti, Boscaini, Masci, Rodol` a, Svoboda, Bronstein 2016 42/66
Geometric deep learning + Partial functional maps Reference Correspondence visualization (similar colors encode corresponding points) Training: FAUST / Testing: SCAPE+TOSCA Monti, Boscaini, Masci, Rodol` a, Svoboda, Bronstein 2016 42/66
Partial correspondence (part-to-full) Rodol` a, Cosmo, Bronstein, Torsello, Cremers 2016 43/66
Partial correspondence (part-to-part) a, Bronstein 2 , Cremers 2016 Litany, Rodol` 43/66
Key observation M N N M C N N C M M slant / | N | slant / | M | |N| |M| a, Bronstein 2 , Cremers 2016 Litany, Rodol` 44/66
Key observation M N N M C NM = C M M C N M C N N slant / | N | |M| |N| | M | a, Bronstein 2 , Cremers 2016 Litany, Rodol` 44/66
Key observation M N N M C NM = C M M C N M C N N slant / | N | |M| | M | = |M| |N| |N| a, Bronstein 2 , Cremers 2016 Litany, Rodol` 44/66
Partial correspondence (part-to-part) a, Bronstein 2 , Cremers 2016 Litany, Rodol` 45/66
Non-rigid puzzle (multi-part) a, Bronstein 2 , Cremers 2016 Litany, Rodol` 45/66
Recommend
More recommend
