Shape representation as distribution of geometric features: from - PowerPoint PPT Presentation

Workshop on Statistical modeling for shapes and imaging Shape representation as distribution of geometric features: from currents to oriented varifolds Irène Kaltenmark Inria - Institut de Mathématiques de Bordeaux Université de Bordeaux joint work with Nicolas Charon and Benjamin Charlier March 2019 1/46

Outline Shape representation as distribution of geometric features Context of computational anatomy Distributions of points Current and varifold representations Functional shapes Conclusion Growth model for computational anatomy 2/46

Shape Spaces [Grenander, Miller, Trouvé, Younes, Beg, ..., Arguillère] Define a metric on a space of shapes . = { ψ v ∈ L 2 ([0 , 1] , V ) } Consider G 1 | v where V is a space of vector fields on a fixed ambient space , e.g. R n , and � t ψ v v s ◦ ψ v t = Id + s ds . 0 G is equipped with a distance: � 1 d G ( Id , ψ ) . | v t | 2 V dt | ψ v = inf { 1 = ψ } . 0 G acts on S ( R n ) : ψ · S = ψ ( S ) and induces an infinitesimal action of V on d ( S 0 , S 1 ) S ( R n ) : . = inf { d G ( Id , g ) | g · S 0 = S 1 } . ξ ( S ) : V → T S S ( R n ) , v �→ v · S = v ( S ) . Strength: this approach handles a large type of shape data and combines definition of distances with shape registration 3/46

From Shape Spaces to inexact matching How to model the shapes ? Approximate Inexact matching problem: � 1 E ( v ) = 1 V dt + λ | v t | 2 2 D ( ψ ( S 0 ) , S 1 ) 2 2 0 Data attachment term: currents and varifolds model shapes as distributions of their local tangent or normal vectors [Glaunès 05, Charon 13]. Reproducing kernels of Hilbert spaces lead to various types of data fidelity metrics between shapes that do not depend on the parametrization . Framework presented here: the choice of the metric reduces to one or two scalar functions. Related work: normal cycles [Roussillon 16], optimal transport as data term [Feydy 17], square-root velocity (SRV) representation [Srivastava 09, 12] 4/46

Back to simple functions How to evaluate f ? Properties of the test functions : ◮ Scale (e.g. for Gaussian distributions) ◮ Shape of the tail 5/46

Shapes as distributions � ϕ ( || x − x 0 || ) d vol X ( x ) ≈ mass of X around x 0 , X � ϕ ( || x − y || ) d vol X ( x ) = 0 . X 6/46

Shapes duality Functions Shapes RKHS approach µ X ∈ H ′ µ X ∈ D ′ I f ∈ D ′ ϕ X ∈ H ϕ ∈ D ω ∈ D X (submanifold) f : R → R X (submanifold) I f ( ϕ ) = � µ X ( ω ) = � X ω ( x ) d vol X ( x ) R f ϕ General kernel: � X ← → ϕ X = ϕ x X Degenerate kernel (Dirac) : � X ← → 1 1 X = δ x X 7/46

Shapes duality RKHS approach µ X ∈ H ′ I f ∈ D ′ µ X ∈ D ′ ϕ X ∈ H ϕ ∈ D ω ∈ D X (submanifold) f : R → R X (submanifold) I f ( ϕ ) = � µ X ( ω ) = � X ω ( x ) d vol X ( x ) R f ϕ For two shapes X , Y , we have � µ X , µ Y � H ′ = � ϕ X , ϕ Y � H . For two points x 0 , y 0 ∈ R n � δ x 0 , δ y 0 � H ′ = ϕ ( || y 0 − x 0 || R n ) . Note that here ϕ : R → R (equivariance to rigid motion), e.g. ϕ ( u ) = exp( − u 2 /σ 2 ). 8/46

Scalar product and norm � � ϕ ( || y − x || ) d vol X ( x ) d vol Y ( y ) = µ Y ( ϕ X ) . � µ X , µ Y � H ′ = Y X µ Y acts linearly on ϕ x 0 : it integrates the mass of Y in a neigh- borhood of x 0 � ϕ ( || y − x 0 || ) d vol Y ( y ) . µ Y ( ϕ x 0 ) = Y X and Y are similar around x 0 if µ Y ( ϕ x 0 ) ≈ µ X ( ϕ x 0 ) . . Finally, d ( X , Y ) 2 . H ′ = || µ X || 2 − 2 � µ X , µ Y � + || µ Y || 2 . = || µ X − µ Y || 2 9/46

Discretization � � ϕ ( || y − x || ) d vol X ( x ) d vol Y ( y ) � µ X i , µ Y j � = Y j X i � i || ) d vol Y ( y ) ≈ m X ϕ ( || y − c X i Y j ≈ m X i m Y j ϕ ( || c Y − c X i || ) j � m X i m Y j ϕ ( || c Y − c X � µ X , µ Y � ≈ i || ) j i , j � µ X 0 , µ X � ≈ 1 × ϕ (0) � 1 2 + 1 � � µ X 0 , µ X ′ � ≈ × ϕ (0) 2 If the diameters of the faces are small wrt to the scale of ϕ (how flat is ϕ around 0), the associ- ated metric is robust to parametrization . 10/46

Discretization ◮ This metric induces registration algorithms that do not require pointwise correspondences between shapes. For example, one can register a curve with 100 points on a curve with 150 points. ◮ Shapes union: if X = X 1 ∪ X 2 , then µ X = µ X 1 + µ X 2 − µ X 1 ∩ X 2 . ◮ This metric is quite robust to discontinuities. Conversely, normal cycles allow to discriminate topological properties [Roussillon 16]. 11/46

Mass sensitivity X µ Y ≈ 2 µ X Y so that d ( X , Y ) 2 = || µ X − µ Y || 2 ≈ || µ X || 2 Application to brain registration X and Y have the same length but ” || µ X || > || µ Y || ” Inclusion detection ? This is not without consequences ” d ( X , Y ) 2 ≈ || µ Y 2 || 2 ” 12/46

Data compression [Gori 17] Consider a fiber bundle B = � n i X i such that for any i , X i is close to an average fiber ˜ X , i.e. µ X i ≈ µ ˜ X , then n � µ B = µ X i ≈ n µ ˜ X . Cross section of 3 fiber bundles i 13/46

Outline Shape representation as distribution of geometric features Context of computational anatomy Distributions of points Current and varifold representations Functional shapes Conclusion Growth model for computational anatomy 14/46

Limitations The tangent space distributions of shapes highlight natural correspondences... as well as the shapes’ orientation: 15/46

Tangent bundle For curves and surfaces in R 2 or R 3 , oriented tangent spaces can be represented by a unique unit vector (tangent or normal). µ X is now a distribution on a space of test functions ω : R n × S n − 1 → R , ( n = 2 , 3) : � ω ( x , − → t x ) d vol X ( x ) µ X ( ω ) = X − → − → y � H ′ = ϕ ( || y − x || R n ) γ ( �− → t x , − → t y t x � δ x , δ t y � R n ) . Note that ϕ : R → R , γ : R → R (equivariance to rigid motion). 16/46

Representation choices X , Y two shapes, µ X , µ Y the respective distributions of their local tangent or normal vectors �� ϕ ( || y − x || R n ) γ ( �− → t x , − → t y � R n ) d vol X ( x ) d vol Y ( y ) � µ X , µ Y � H ′ = X × Y − → − → � m X i m Y j ϕ ( || c Y − c X t X t Y ≈ i || ) γ ( � i , j � ) j i , j γ ( �− → t x , − → Data type t y � ) γ ( u ) �− → t x , − → Currents [Glaunès 05] t y � u �− → t x , − → t y � 2 u 2 (unoriented) Varifolds [Charon 13] 2 �− → t x , − → exp � t y � /σ 2 � exp � 2 u /σ 2 � Oriented Varifolds* T T −|− → t x − − → ∝ exp � � t y | 2 /σ 2 T Remark: σ T does not depend on the scale of the shapes. *[Kaltenmark,Charlier,Charon 17] 17/46

Representation choices X , Y two shapes, µ X , µ Y the respective distributions of their local tangent or normal vectors �� ϕ ( || y − x || R n ) γ ( �− → t x , − → t y � R n ) d vol X ( x ) d vol Y ( y ) � µ X , µ Y � H ′ = X × Y − → − → � m X i m Y j ϕ ( || c Y − c X t X t Y ≈ i || ) γ ( � i , j � ) j i , j γ ( �− → t x , −− → Data type t x � ) − γ ( �− → t x , − → Currents [Glaunès 05] t x � ) = − 1 γ ( �− → t x , − → (unoriented) Varifolds [Charon 13] t x � ) = 1 γ ( �− → t x , −− → Oriented Varifolds* t x � ) = 0 *[Kaltenmark,Charlier,Charon 17] 18/46

Current regularization 2 − → − → i ϕ ( || y i − x || ) �− → � t x m Y t Y � δ x , µ Y � H ′ = t x , i � i =1 − → − → ≈ ϕ (0) �− → t x , m Y t Y 1 + m Y t Y 2 � 1 2 y − x || ) �− → t x , − → ≈ ˜ m ϕ ( || ˜ y � t ˜ − → − → t ˜ t x y ≈ � δ x , ˜ m δ y � H ′ ˜ − → − → Mass cancellation effect: if m Y t Y 1 + m Y t Y 2 ≈ 0, then || µ Y || H ′ ≈ 0 . 1 2 Counterpart: thin and sharp structures are not seen by the metric or at best their mass is underestimated, e.g. tails, tip of horns, leaf stem, etc. 19/46

Mass conservation with varifolds Theorem (Charon 13) If ϕ and γ are two continuous positive functions, ϕ (0) > 0 , and γ (1) > 0 , then there exists c > 0 , such that for any k-dimensional compact submanifold X cvol k ( X ) ≤ || µ X || H ′ . 20/46

Applications in the diffeomorphometric framework Inexact matching problem: deform a shape S 0 towards a target shape S 1 � 1 E ( v ) = 1 V dt + λ | v t | 2 2 | µ ψ 1 ( S 0 ) − µ S 1 | 2 H ′ 2 0 This energy is minimized by a gradient descent. Each step consists in: ◮ Computing the gradient of the data attachment term ◮ Pulling it backward over time ◮ Updating t → v t ◮ Updating ψ 1 ( S 0 ) (end point of the geodesic generated by v ) 21/46

Denoising with currents Source: blue. Target: red. 22/46

Denoising with currents Source: green. Target: red. 23/46

Comparisons Source: blue. Target: red. Currents Unoriented varifolds Oriented varifolds 24/46

Unoriented varifolds and multi-attachment terms Source: blue. Target: red. 25/46

Restoration with oriented varifolds Source: blue. Target: red. Initial shape is an ellipsoid 26/46