Discrete Thin-Plate Splines for Large Data Sets Linda Stals Steve - PowerPoint PPT Presentation

Outline tps dtps Convergence Results Future Work Discrete Thin-Plate Splines for Large Data Sets Linda Stals Steve Roberts Department of Mathematics Australian National University July 13, 2006 Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Thin Plate Splines Smoothing Splines Radial Basis Functions Discrete Thin Plate Splines Finite Element Approximation System of Equations Convergence Analysis Dirichlet Boundary Conditions Finite Element Convergence Interpolation Error Results Holes 3D Examples Future Work Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Thin-Plate Splines • 3D Image Recovery • Finger Print Analysis • Image Warping • Medical Image Analysis • Data Mining Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Thin-Plate Splines • 3D Image Recovery • Finger Print Analysis • Image Warping • Medical Image Analysis • Data Mining Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Smoothing Splines Given a set of attributes vectors x = ( x 1 , x 2 , · · · , x d ) T , build a predictive model y = f ( x ) . y ≈ f ( x ) . To estimate f by a 2nd-order smoothing spline minimise: � 2 � � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x , n ν Ω i =1 | ν | =2 The first term penalises lack of fit, the second penalises roughness. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Smoothing Splines Given a set of attributes vectors x = ( x 1 , x 2 , · · · , x d ) T , build a predictive model y ≈ f ( x ) . To estimate f by a 2nd-order smoothing spline minimise: � � 2 � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x , ν n Ω i =1 | ν | =2 The first term penalises lack of fit, the second penalises roughness. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Smoothing Splines Given a set of attributes vectors x = ( x 1 , x 2 , · · · , x d ) T , build a predictive model y ≈ f ( x ) . To estimate f by a 2nd-order smoothing spline minimise: � � 2 � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x , ν n Ω i =1 | ν | =2 The first term penalises lack of fit, the second penalises roughness. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Smoothing Splines Given a set of attributes vectors x = ( x 1 , x 2 , · · · , x d ) T , build a predictive model y ≈ f ( x ) . To estimate f by a 2nd-order smoothing spline minimise: � � 2 � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x , ν n Ω i =1 | ν | =2 The first term penalises lack of fit, the second penalises roughness. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Radial Basis Functions The standard approach is to represent f as a linear combination of radial basis functions M n � � w i U ( x , x ( i ) ) , f ( x ) = a φ k ( x ) + α k =1 i =1 where φ k are monomials of order up to 1 and U are suitable radial basis functions. Favoured method as it gives an analytical solution. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Radial Basis Functions The standard approach is to represent f as a linear combination of radial basis functions M n � � w i U ( x , x ( i ) ) , f ( x ) = a φ k ( x ) + α k =1 i =1 where φ k are monomials of order up to 1 and U are suitable radial basis functions. Favoured method as it gives an analytical solution. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Radial Basis Functions 2D Eg: U ( x , x ( i ) ) = − 1 16 π r 2 ln ( r ) . Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Thin Plate Splines • Requires a solution of a dense system of matrices. • System may be ill-conditioned. • Size increases with the number of data points. Not practical for large data sets. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Thin Plate Splines • Requires a solution of a dense system of matrices. • System may be ill-conditioned. • Size increases with the number of data points. Not practical for large data sets. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Finite Element Approximation Represent f as a linear combination of linear finite elements. In vector notation f will be of the form f ( x ) = b ( x ) T c . Minimise J α over all f of this form � � 2 � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x . n ν Ω i =1 | ν | =2 Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Finite Element Approximation Represent f as a linear combination of linear finite elements. In vector notation f will be of the form f ( x ) = b ( x ) T c . Minimise J α over all f of this form � � 2 � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x . n ν Ω i =1 | ν | =2 Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Finite Element Approximation Represent f as a linear combination of linear finite elements. In vector notation f will be of the form f ( x ) = b ( x ) T c . Minimise J α over all f of this form � � 2 � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x . n ν Ω i =1 | ν | =2 Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Non-Conforming Finite elements The smoothing term (derivatives) is not defined for piecewise multi-linear functions. Use non-conforming finite elements. Represent the gradient of f by u = ( b T g 1 , ..., b T g d ) where � � ∇ f ( x ) · ∇ v ( x ) d x = u ( x ) · ∇ v ( x ) d x , Ω Ω for all piecewise multi-linear function v . Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Non-Conforming Finite elements � � ∇ f ( x ) · ∇ v ( x ) d x = u ( x ) · ∇ v ( x ) d x , Ω Ω is equivalent to d � L c = G s g s , s =1 where L is a discrete approximation to the negative Laplace operator and ( G 1 , ..., G d ) is a discrete approximation to the transpose of the gradient operator. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Non-Conforming Finite elements � � ∇ f ( x ) · ∇ v ( x ) d x = u ( x ) · ∇ v ( x ) d x , Ω Ω is equivalent to d � L c = G s g s , s =1 where L is a discrete approximation to the negative Laplace operator and ( G 1 , ..., G d ) is a discrete approximation to the transpose of the gradient operator. Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Finite Element Approximation 2nd-order smoothing spline: minimise � 2 � � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x . n ν Ω i =1 | ν | =2 Finite element approximation: minimise n d � � J α ( c , g 1 , g 2 , · · · , g d ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α g T s L g s , n i =1 s =1 subject to d � L c = G s g s . s =1 Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work Finite Element Approximation 2nd-order smoothing spline: minimise � 2 � � n � � J α ( f ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α ( D ν f ( x )) 2 d x . n ν Ω i =1 | ν | =2 Finite element approximation: minimise n d � � J α ( c , g 1 , g 2 , · · · , g d ) = 1 ( f ( x ( i ) ) − y ( i ) ) 2 + α g T s L g s , n i =1 s =1 subject to d � L c = G s g s . s =1 Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets

Outline tps dtps Convergence Results Future Work 2D Formulation 2nd-order smoothing spline: minimise n � 1 ( f ( x ( i ) ) − y ( i ) ) 2 J α ( f ) = n i =1 � �� � 2 � � 2 + 2 ( ∂ 1 ∂ 2 f ( x )) 2 + � ∂ 2 ∂ 2 + α 1 f ( x ) 2 f ( x ) d x , Ω n � 1 ( b ( x ( i ) ) T c − y ( i ) ) 2 J α ( c , g 1 , g 2 ) = n i =1 � ∇ b T ( x ) g 1 . ∇ b T ( x ) g 1 + ∇ b T ( x ) g 2 . ∇ b T ( x ) g 2 d x + α Ω Linda Stals, Steve Roberts Discrete Thin-Plate Splines for Large Data Sets