Point-wise map recovery Task : Recover a point-to-point map from its - PowerPoint PPT Presentation

Point-wise map recovery Task : Recover a point-to-point map from its functional representation n k ⇒ k C n P 1/24

Point-wise map recovery Task : Recover a point-to-point map from its functional representation n k ⇒ k ⇐ rank( C ) ≤ k ≪ n n rank( P ) ≤ n The inverse problem is highly underdetermined 1/24

Point-wise map recovery Task : Recover a point-to-point map from its functional representation n k ⇒ k ⇐ rank( C ) ≤ k ≪ n n rank( P ) ≤ n The inverse problem is highly underdetermined Need to use priors on the expected structure of the underlying map (e.g. bijectivity, smoothness, partiality, etc.) 1/24

Mapping delta functions Algorithm: For each x ∈ M construct the delta function δ x : M → R x ∈ M 2/24

Mapping delta functions Algorithm: For each x ∈ M construct the delta function δ x : M → R δ x : M → R 2/24

Mapping delta functions Algorithm: For each x ∈ M construct the delta function δ x : M → R Compute its image δ x �→ Tδ x δ x : M → R Tδ x : N → R 2/24

Mapping delta functions Algorithm: For each x ∈ M construct the delta function δ x : M → R Compute its image δ x �→ Tδ x Find argmax y ∈N Tδ x (y) δ x : M → R Tδ x : N → R 2/24

Mapping delta functions Algorithm: For each x ∈ M construct the delta function δ x : M → R Compute its image δ x �→ Tδ x Find argmax y ∈N Tδ x (y) Doing this for all points is costly The maximum is delocalized due to the band-limited approximation of T ! δ x : M → R Tδ x : N → R 2/24

Linear assignment problem For orthogonal bases Φ M , Φ N we can write C = Φ ⊤ N PΦ M If the underlying map is known to be bijective, solve the LAP: Π ∈{ 0 , 1 } n × n −� Π , Φ N CΦ ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 min M � Rodol` a et al. 2015 3/24

Linear assignment problem For orthogonal bases Φ M , Φ N we can write C = Φ ⊤ N PΦ M If the underlying map is known to be bijective, solve the LAP: Π ∈{ 0 , 1 } n × n � CΦ ⊤ M − Φ ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 N Π � 2 min F Rodol` a et al. 2015 3/24

Linear assignment problem For orthogonal bases Φ M , Φ N we can write C = Φ ⊤ N PΦ M If the underlying map is known to be bijective, solve the LAP: Π ∈{ 0 , 1 } n × n � CΦ ⊤ M − Φ ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 N Π � 2 min F i ( x ) (a column of Φ ⊤ The delta function δ x has coefficients a i = φ M M ) ⇒ the image of all delta functions on M is simply CΦ ⊤ M Rodol` a et al. 2015; Ovsjanikov et al. 2012 3/24

Linear assignment problem For orthogonal bases Φ M , Φ N we can write C = Φ ⊤ N PΦ M If the underlying map is known to be bijective, solve the LAP: Π ∈{ 0 , 1 } n × n � CΦ ⊤ M − Φ ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 N Π � 2 min F i ( x ) (a column of Φ ⊤ The delta function δ x has coefficients a i = φ M M ) ⇒ the image of all delta functions on M is simply CΦ ⊤ M Interpretation: Seek the permutation aligning the k -dimensional spectral M in the ℓ 2 sense embeddings CΦ ⊤ M and Φ ⊤ Rodol` a et al. 2015; Ovsjanikov et al. 2012 3/24

Linear assignment problem For orthogonal bases Φ M , Φ N we can write C = Φ ⊤ N PΦ M If the underlying map is known to be bijective, solve the LAP: Π ∈{ 0 , 1 } n × n � CΦ ⊤ M − Φ ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 N Π � 2 min F i ( x ) (a column of Φ ⊤ The delta function δ x has coefficients a i = φ M M ) ⇒ the image of all delta functions on M is simply CΦ ⊤ M Interpretation: Seek the permutation aligning the k -dimensional spectral M in the ℓ 2 sense embeddings CΦ ⊤ M and Φ ⊤ Inefficient for large shapes Rodol` a et al. 2015; Ovsjanikov et al. 2012 3/24

Linear assignment problem For orthogonal bases Φ M , Φ N we can write C = Φ ⊤ N PΦ M If the underlying map is known to be bijective, solve the LAP: Π ∈{ 0 , 1 } n × n � CΦ ⊤ M − Φ ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 N Π � 2 min F i ( x ) (a column of Φ ⊤ The delta function δ x has coefficients a i = φ M M ) ⇒ the image of all delta functions on M is simply CΦ ⊤ M Interpretation: Seek the permutation aligning the k -dimensional spectral M in the ℓ 2 sense embeddings CΦ ⊤ M and Φ ⊤ Inefficient for large shapes Lack of desirable properties on the recovered map (e.g. smoothness) Rodol` a et al. 2015; Ovsjanikov et al. 2012 3/24

Nearest neighbors Relaxing bijectivity to stochasticity constraints: P ∈{ 0 , 1 } n × m � CΦ ⊤ M − Φ ⊤ N P � 2 s . t . P ⊤ 1 = 1 min F Rodol` a et al. 2015 4/24

Nearest neighbors Relaxing bijectivity to stochasticity constraints: P ∈{ 0 , 1 } n × m � CΦ ⊤ M − Φ ⊤ N P � 2 s . t . P ⊤ 1 = 1 min F Can be solved efficiently by a nearest-neighbor search in R k Φ ⊤ N CΦ ⊤ M Rodol` a et al. 2015; Ovsjanikov et al. 2012 4/24

Orthogonal refinement (ICP) Orthogonal C ⇔ Area-preserving map Idea: Treat C as a pre-alignment, do orthogonal refinement to improve map quality Ovsjanikov et al. 2012 5/24

Orthogonal refinement (ICP) Orthogonal C ⇔ Area-preserving map Idea: Treat C as a pre-alignment, do orthogonal refinement to improve map quality Algorithm ( ICP ): P -step (nearest neighbors): P ∗ = arg P ∈{ 0 , 1 } n × m � C ∗ Φ ⊤ M − Φ ⊤ N P � 2 s . t . P ⊤ 1 = 1 min F Ovsjanikov et al. 2012 5/24

Orthogonal refinement (ICP) Orthogonal C ⇔ Area-preserving map Idea: Treat C as a pre-alignment, do orthogonal refinement to improve map quality Algorithm ( ICP ): P -step (nearest neighbors): P ∗ = arg P ∈{ 0 , 1 } n × m � C ∗ Φ ⊤ M − Φ ⊤ N P � 2 s . t . P ⊤ 1 = 1 min F C -step (orthogonal Procrustes): C ∗ = arg C ∈ R k × k � CΦ ⊤ M − Φ ⊤ N P ∗ � 2 s . t . C ⊤ C = I min F Ovsjanikov et al. 2012 5/24

Non-orthogonal refinement (CPD) For more general deformations (e.g. non-area preserving, non-isometric), do non-rigid refinement: D KL ( CΦ ⊤ M , Φ ⊤ N P ) + λ � CΦ ⊤ M − Φ ⊤ N P � 2 min Ω P ∈{ 0 , 1 } n × m P ⊤ 1 = 1 s . t . Rodol` a et al. 2015 6/24

Non-orthogonal refinement (CPD) For more general deformations (e.g. non-area preserving, non-isometric), do non-rigid refinement: D KL ( CΦ ⊤ M , Φ ⊤ + λ � CΦ ⊤ M − Φ ⊤ N P � 2 min N P ) Ω P ∈{ 0 , 1 } n × m � �� � � �� � Kullback − Leibler coherence P ⊤ 1 = 1 s . t . Rodol` a et al. 2015 6/24

Non-orthogonal refinement (CPD) For more general deformations (e.g. non-area preserving, non-isometric), do non-rigid refinement: D KL ( CΦ ⊤ M , Φ ⊤ + λ � CΦ ⊤ M − Φ ⊤ N P � 2 min N P ) Ω P ∈{ 0 , 1 } n × m � �� � � �� � Kullback − Leibler coherence P ⊤ 1 = 1 s . t . � · � 2 Ω promotes smooth displacements Rodol` a et al. 2015 6/24

Non-orthogonal refinement (CPD) For more general deformations (e.g. non-area preserving, non-isometric), do non-rigid refinement: D KL ( CΦ ⊤ M , Φ ⊤ + λ � CΦ ⊤ M − Φ ⊤ N P � 2 min N P ) Ω P ∈{ 0 , 1 } n × m � �� � � �� � Kullback − Leibler coherence P ⊤ 1 = 1 s . t . � · � 2 Ω promotes smooth displacements λ controls the regularity (rigid for λ → ∞ ) Rodol` a et al. 2015 6/24

Non-orthogonal refinement (CPD) For more general deformations (e.g. non-area preserving, non-isometric), do non-rigid refinement: D KL ( CΦ ⊤ M , Φ ⊤ + λ � CΦ ⊤ M − Φ ⊤ N P � 2 min N P ) Ω P ∈{ 0 , 1 } n × m � �� � � �� � Kullback − Leibler coherence P ⊤ 1 = 1 s . t . � · � 2 Ω promotes smooth displacements λ controls the regularity (rigid for λ → ∞ ) Solved by coherent point drift Rodol` a et al. 2015; Myronenko and Song 2010 6/24

Non-orthogonal refinement (CPD) For more general deformations (e.g. non-area preserving, non-isometric), do non-rigid refinement: D KL ( CΦ ⊤ M , Φ ⊤ + λ � CΦ ⊤ M − Φ ⊤ N P � 2 min N P ) Ω P ∈{ 0 , 1 } n × m � �� � � �� � Kullback − Leibler coherence P ⊤ 1 = 1 s . t . � · � 2 Ω promotes smooth displacements λ controls the regularity (rigid for λ → ∞ ) Solved by coherent point drift Does not scale well Rodol` a et al. 2015; Myronenko and Song 2010 6/24

Comparison Reference Nearest neighbors Ovsjanikov et al. 2012; Rodol` a et al. 2015 7/24

Comparison Reference Nearest neighbors ICP Ovsjanikov et al. 2012; Rodol` a et al. 2015 7/24

Comparison Reference Nearest neighbors ICP CPD Ovsjanikov et al. 2012; Rodol` a et al. 2015 7/24

Product manifold filter (PMF) Given P 0 point-to-point (e.g. from nearest-neighbors), consider the LAP: Π ∈{ 0 , 1 } n × n trace( Π ⊤ K M P 0 K ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 max N ) with K M = exp( − D 2 M /σ 2 ) and K N = exp( − D 2 N /σ 2 ) Vestner, Rodol` a, Litman, Bronstein, Cremers 2016 8/24

Product manifold filter (PMF) Given P 0 point-to-point (e.g. from nearest-neighbors), consider the LAP: Π ∈{ 0 , 1 } n × n trace( Π ⊤ K M P 0 K ⊤ s . t . Π ⊤ 1 = 1 , Π1 = 1 max N ) with K M = exp( − D 2 M /σ 2 ) and K N = exp( − D 2 N /σ 2 ) Interpretation as an inference problem from stochastic data Vestner, Rodol` a, Litman, Bronstein, Cremers 2016 8/24