Perturbation analysis: intuition ∆ M φ 1 φ 2 φ 3 M ¯ M ∆ M φ 1 φ 2 φ 3 ¯ ¯ ¯ ∆ ¯ φ 1 φ 2 φ 3 M Ignoring boundary interaction: disjoint parts (block-diagonal matrix) Eigenvectors = Mixture of eigenvectors of the parts Rodol` a, Cosmo, B, Torsello, Cremers 2016 26/59
Perturbation analysis: eigenvalues 8 . 00 · 10 − 2 M 6 . 00 4 . 00 r k N 2 . 00 0 . 00 10 20 30 40 50 eigenvalue number k ≈ |M| Slope r |N | (depends on the area of the cut) Consistent with Weil’s law Rodol` a, Cosmo, B, Torsello, Cremers 2016 27/59
Perturbation analysis: details ∆ M M t E ¯ ∆ M + t D M M t E ⊤ ∆ ¯ M + t D ¯ M ∆ ¯ M Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59
Perturbation analysis: details ∆ M M t E ¯ ∆ M + t D M M t E ⊤ ∆ ¯ M + t D ¯ M ∆ ¯ M “How would the Laplacian eigenvalues and eigenvectors of the red part change if we attached a blue part to it?” Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59
Perturbation analysis: details P M P M n × n n × ¯ n D M E ¯ M “How would the Laplacian eigenvalues and eigenvectors of the red part change if we attached a blue part to it?” Rodol` a, Cosmo, B, Torsello, Cremers 2016 28/59
Perturbation analysis: details M = ¯ Φ ¯ Λ ¯ Denote ∆ M + t P M = Φ ( t ) Λ ( t ) Φ ( t ) ⊤ , ∆ ¯ Φ ⊤ , Φ = Φ (0) , and Λ = Λ (0) . Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues is given by � 0 � d 0 dtλ i = φ ⊤ i P M φ i P M = 0 D M Rodol` a, Cosmo, B, Torsello, Cremers 2016 29/59
Perturbation analysis: details M = ¯ Φ ¯ Λ ¯ Denote ∆ M + t P M = Φ ( t ) Λ ( t ) Φ ( t ) ⊤ , ∆ ¯ Φ ⊤ , Φ = Φ (0) , and Λ = Λ (0) . Theorem 1 (eigenvalues) The derivative of the non-trivial eigenvalues is given by � 0 � d 0 dtλ i = φ ⊤ i P M φ i P M = 0 D M Theorem 2 (eigenvectors) Assuming λ i � = λ j for i � = j and λ i � = ¯ λ j for all i, j , the derivative of the non-trivial eigenvectors is given by � 0 ¯ i P ¯ n φ ⊤ n φ ⊤ � i P M φ j d φ j � � 0 ¯ dt φ i = φ j + φ j P = λ i − ¯ λ i − λ j E 0 λ j j =1 j =1 j � = i Rodol` a, Cosmo, B, Torsello, Cremers 2016 29/59
Perturbation analysis: boundary interaction strength 20 10 Value of f Eigenvector perturbation depends on length and position of the boundary Perturbation strength � d dt Φ � F ≤ c � ∂ M f ( m ) dm , where n � φ i ( m ) φ j ( m ) � 2 � f ( m ) = λ i − λ j i,j =1 j � = i Rodol` a, Cosmo, B, Torsello, Cremers 2016 30/59
Laplacian perturbation: typical picture Plate Punctured plate Figure: Filoche, Mayboroda 2009 31/59
Partial functional maps Model shape M T Query shape N Part M ⊆ M ≈ isometric to N M N Data F , G Partial functional map TG ≈ F ( M ) M Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59
Partial functional maps Model shape M C Query shape N Part M ⊆ M ≈ isometric to N M N Data F , G Partial functional map G ⊤ ΨC ≈ F ( M ) ⊤ Φ M Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59
Partial functional maps Model shape M C Query shape N Part M ⊆ M ≈ isometric to N v N Data F , G Partial functional map G ⊤ ΨC ≈ F ⊤ diag( v ) Φ v ∈ F ( M ) indicator function of M M Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59
Partial functional maps Model shape M C Query shape N Part M ⊆ M ≈ isometric to N v N Data F , G Partial functional map G ⊤ ΨC ≈ F ⊤ diag( η ( v )) Φ v ∈ F ( M ) indicator function of M M η ( t ) = 1 2 (tanh(2 t − 1) + 1) Optimization problem w.r.t. correspondence C and part v C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Rodol` a, Cosmo, B, Torsello, Cremers 2016 32/59
Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Rodol` a, Cosmo, B, Torsello, Cremers 2016 33/59
Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59
Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � �� � � �� � area preservation Mumford − Shah � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59
Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � �� � � �� � area preservation Mumford − Shah � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Correspondence regularization � � ρ corr ( C ) = µ 3 � C ◦ W � 2 ( C ⊤ C ) 2 (( C ⊤ C ) ii − d i ) 2 F + µ 4 ij + µ 5 i � = j i Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59
Partial functional maps C , v � G ⊤ ΨC − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ corr ( C ) + ρ part ( v ) min Part regularization � � 2 � � ρ part ( v ) = µ 1 |N| − η ( v ) dm + µ 2 ξ ( v ) �∇ M v � dm M M � �� � � �� � area preservation Mumford − Shah � � η ( t ) − 1 where ξ ( t ) ≈ δ 2 Correspondence regularization � � ρ corr ( C ) = µ 3 � C ◦ W � 2 ( C ⊤ C ) 2 (( C ⊤ C ) ii − d i ) 2 + µ 4 + µ 5 F ij � �� � i � = j i slant � �� � � �� � rank ≈ r ≈ orthogonality Rodol` a, Cosmo, B, Torsello, Cremers 2016; BB 2008 33/59
Structure of partial functional correspondence 4 2 0 0 20 40 60 80 100 C ⊤ C C W singular values Rodol` a, Cosmo, B, Torsello, Cremers 2016 34/59
Alternating minimization C -step: fix v ∗ , solve for correspondence C C � G ⊤ ΨC − F ⊤ diag( η ( v ∗ )) Φ � 2 , 1 + ρ corr ( C ) min v -step: fix C ∗ , solve for part v v � G ⊤ ΨC ∗ − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ part ( v ) min Rodol` a, Cosmo, B, Torsello, Cremers 2016 35/59
Alternating minimization C -step: fix v ∗ , solve for correspondence C C � G ⊤ ΨC − F ⊤ diag( η ( v ∗ )) Φ � 2 , 1 + ρ corr ( C ) min v -step: fix C ∗ , solve for part v v � G ⊤ ΨC ∗ − F ⊤ diag( η ( v )) Φ � 2 , 1 + ρ part ( v ) min Iteration 1 2 3 4 Rodol` a, Cosmo, B, Torsello, Cremers 2016 35/59
Example of convergence Time (sec.) 0 5 10 15 20 25 10 10 C -step 10 9 v -step 10 8 Energy 10 7 10 6 10 5 10 4 0 20 40 60 80 100 Iteration Rodol` a, Cosmo, B, Torsello, Cremers 2016 36/59
Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59
Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59
Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59
Examples of partial functional maps Rodol` a, Cosmo, B, Torsello, Cremers 2016 37/59
Partial functional maps vs Functional maps 100 150 100 80 50 % Correspondences PFM 60 Func. maps 40 50 100 20 150 0 0 0.05 0.1 0.15 0.2 0.25 Geodesic error Correspondence performance for different rank values k Rodol` a, Cosmo, B, Torsello, Cremers 2016 38/59
Partial correspondence performance Cuts Holes 100 % Correspondences 80 60 40 20 0 0 0.05 0.1 0.15 0.2 0.25 0 0.05 0.1 0.15 0.2 0.25 Geodesic Error Geodesic Error PFM RF IM EN GT SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, B, Torsello, Cremers 2016 ( PFM ); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 39/59
Partial correspondence performance Cuts Holes 1 Mean geodesic error 0 . 8 0 . 6 0 . 4 0 . 2 0 20 40 60 80 20 40 60 80 Partiality (%) Partiality (%) PFM RF IM EN GT SHREC’16 Partial Matching benchmark Rodol` a et al. 2016; Methods: Rodol` a, Cosmo, B, Torsello, Cremers 2016 ( PFM ); Sahillio˘ glu, Yemez 2012 (IM); Rodol` a, Bronstein, Albarelli, Bergamasco, Torsello 2012 (GT); Rodol` a et al. 2013 (EN); Rodol` a et al. 2014 (RF) 40/59
Deep learning + Partial functional maps Correspondence 0 . 1 0 . 0 Correspondence error Boscaini, Masci, Rodol` a, B 2016 41/59
Deep learning + Partial functional maps Correspondence 0 . 1 0 . 0 Correspondence error Boscaini, Masci, Rodol` a, B 2016 42/59
Outline Background: Spectral analysis on manifolds Functional correspondence Partial functional correspondence Non-rigid puzzles 43/59
Litani, BB 2012
Partial correspondence Rodol` a, Cosmo, B, Torsello, Cremers 2016 45/59
Non-rigid puzzle Litani, Rodol` a, BB, Cremers 2016 45/59
Partial Laplacian eigenvectors Functional correspondence matrix C Rodol` a, Cosmo, B, Torsello, Cremers 2016 46/59
Key observation M N N M C N N C M M slant ∝ | N | slant ∝ | M | |N| |M| Litani, Rodol` a, BB, Cremers 2016 47/59
Key observation M N N M C NM = C N N C N M C M M slant ∝ | N | |M| |N| | M | Litani, Rodol` a, BB, Cremers 2016 47/59
Key observation M N N M C NM = C N N C N M C M M slant ∝ | N | |M| | M | = |N| |N| |M| Litani, Rodol` a, BB, Cremers 2016 47/59
Non-rigid puzzles problem formulation Model Parts Input M 1 C 1 Model M N 1 N c Parts N 1 , . . . , N p 2 C 2 M 2 N c 1 N 2 Output N 1 Segmentation M i ⊆ M M 0 Located parts N i ⊆ N i Correspondences C i Clutter N c i M N 2 Missing parts M 0 Litani, Rodol` a, BB, Cremers 2016 48/59
Non-rigid puzzles problem formulation Model Parts Data F i , G i M 1 C 1 Model basis Φ , Φ ( M i ) N 1 N c Part bases Ψ i , Ψ i ( N i ) 2 C 2 M 2 N c 1 N 2 Data term N 1 F ⊤ i Φ ( M i ) ≈ G ⊤ i Ψ i ( N i ) C i M 0 M N 2 Litani, Rodol` a, BB, Cremers 2016 48/59
Non-rigid puzzles problem formulation p � � G ⊤ Ψ i ( N i ) C i − F ⊤ min i Ψ Ψ i Φ Φ Φ( M i ) � 2 , 1 C i i =1 M i ⊆M ,N i ⊆N i p p � � + λ M ρ part ( M i ) + λ N ρ part ( N i ) i =0 i =1 p � + λ corr ρ corr ( C i ) i =1 s.t. M i ∩ M j = ∅ ∀ i � = j M 0 ∪ M 1 ∪ · · · ∪ M p = M | M i | = | N i | ≥ α |N i | , Litani, Rodol` a, BB, Cremers 2016 49/59
Non-rigid puzzles problem formulation p � � G ⊤ Ψ i C i − F ⊤ min i diag( η ( u i ))Ψ Ψ i diag( η ( v i ))Φ Φ Φ � 2 , 1 C i i =1 u i , v i p p � � + λ M ρ part ( η ( v i )) + λ N ρ part ( η ( u i )) i =0 i =1 p � + λ corr ρ corr ( C i ) i =1 p � s.t. η ( u i ) = 1 i =1 a ⊤ M u i = a ⊤ N v i ≥ α a ⊤ N i 1 Litani, Rodol` a, BB, Cremers 2016 49/59
Convergence example Outer iteration 1 Litani, Rodol` a, BB, Cremers 2016 50/59
Convergence example Outer iteration 2 Litani, Rodol` a, BB, Cremers 2016 50/59
Convergence example Outer iteration 3 Litani, Rodol` a, BB, Cremers 2016 50/59
Convergence example Time (sec) 30 32 34 36 38 40 42 44 46 48 80 90 100 110 120 130 140 150 160 Iteration number Litani, Rodol` a, BB, Cremers 2016 51/59
“Perfect puzzle” example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter No Missing part No Data term Dense (SHOT) Litani, Rodol` a, BB, Cremers 2016 52/59
“Perfect puzzle” example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter No Missing part No Data term Dense (SHOT) Segmentation Litani, Rodol` a, BB, Cremers 2016 52/59
“Perfect puzzle” example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter No Missing part No Data term Dense (SHOT) Correspondence Litani, Rodol` a, BB, Cremers 2016 52/59
Overlapping parts example Model/Part Synthetic (FAUST) Transformation Near-isometric Clutter Yes (overlap) Missing part No Data term Dense (SHOT) Segmentation Litani, Rodol` a, BB, Cremers 2016 53/59
Overlapping parts example Model/Part Synthetic (FAUST) Transformation Near-isometric Clutter Yes (overlap) Missing part No Data term Dense (SHOT) Correspondence Litani, Rodol` a, BB, Cremers 2016 53/59
Overlapping parts example Model/Part Synthetic (FAUST) Transformation Near-isometric Clutter Yes (overlap) Missing part No Data term Dense (SHOT) 0.1 0.0 Correspondence error Litani, Rodol` a, BB, Cremers 2016 53/59
Missing parts example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter Yes (extra part) Missing part Yes Data term Dense (SHOT) Litani, Rodol` a, BB, Cremers 2016 54/59
Missing parts example Model/Part Synthetic (TOSCA) Transformation Isometric Clutter Yes (extra part) Missing part Yes Data term Dense (SHOT) Segmentation Litani, Rodol` a, BB, Cremers 2016 54/59
Recommend
More recommend