Multimodal 3D Registration of Anatomic (MRI) and Functional (fMRI - - PowerPoint PPT Presentation

Multimodal 3D Registration

• f Anatomic (MRI) and Functional (fMRI and PET)

Intra-Patient Images of the Brain

A.G. Legaz-Aparicio1, R. Verdú-Monedero1, J. Larrey-Ruiz1, F . López-Mir2, V. Naranjo2, A. Bernabeu3

1Universidad Politécnica de Cartagena, Cartagena 30202, Spain 2Universidad Politécnica de Valencia, I3BH LabHuman, 46022 Valencia, Spain 3Inscanner S.L, Unidad de Resonancia Magnética, 03016 Alicante, Spain

6th International Work-conference on the Interplay between Natural and Artificial Computation - IWINAC 2015

Outline

1

Introduction

2

Methodology

3

Results

4

Conclusions

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 2 / 29

Outline

1

Introduction

2

Methodology

3

Results

4

Conclusions

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 3 / 29

• Introduction. Image registration

Image registration: process of finding the optimum geometrical transformation which relates corresponding points of two dataset. Geometrically, the image registration consist of aligning one of the datasets, known as the template set (T) with the other set, known as the reference set (R). The datasets can be taken at different times, from different viewpoints, and/or by different sensors. Applied to medical imaging, the registration process helps to improve the diagnosis and tracking of a wide group of pathologies, as well as assist to plan the most appropriate treatment.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 4 / 29

• Introduction. Image registration

Image registration: process of finding the optimum geometrical transformation which relates corresponding points of two dataset. Geometrically, the image registration consist of aligning one of the datasets, known as the template set (T) with the other set, known as the reference set (R). The datasets can be taken at different times, from different viewpoints, and/or by different sensors. Applied to medical imaging, the registration process helps to improve the diagnosis and tracking of a wide group of pathologies, as well as assist to plan the most appropriate treatment.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 4 / 29

• Introduction. Image registration

Image registration: process of finding the optimum geometrical transformation which relates corresponding points of two dataset. Geometrically, the image registration consist of aligning one of the datasets, known as the template set (T) with the other set, known as the reference set (R). The datasets can be taken at different times, from different viewpoints, and/or by different sensors. Applied to medical imaging, the registration process helps to improve the diagnosis and tracking of a wide group of pathologies, as well as assist to plan the most appropriate treatment.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 4 / 29

• Introduction. Image registration

Image registration: process of finding the optimum geometrical transformation which relates corresponding points of two dataset. Geometrically, the image registration consist of aligning one of the datasets, known as the template set (T) with the other set, known as the reference set (R). The datasets can be taken at different times, from different viewpoints, and/or by different sensors. Applied to medical imaging, the registration process helps to improve the diagnosis and tracking of a wide group of pathologies, as well as assist to plan the most appropriate treatment.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 4 / 29

• Introduction. Anatomic-functional images of the brain

The functional images of the brain, i.e. fMRI or PET,

Do not have detailed structural information. Do not provide a specific anatomical location of the functional information. have a spatial resolution, signal to noise ratio and contrast lower than anatomical images

It is necessary to register the functional image with anatomical image to provide a geometrical localization, i.e. MRI. This type of registration is known as multimodal registration due to the different contrast and intensity of the datasets.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 5 / 29

• Introduction. Anatomic-functional images of the brain

The functional images of the brain, i.e. fMRI or PET,

Do not have detailed structural information. Do not provide a specific anatomical location of the functional information. have a spatial resolution, signal to noise ratio and contrast lower than anatomical images

It is necessary to register the functional image with anatomical image to provide a geometrical localization, i.e. MRI. This type of registration is known as multimodal registration due to the different contrast and intensity of the datasets.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 5 / 29

• Introduction. Anatomic-functional images of the brain

The functional images of the brain, i.e. fMRI or PET,

Do not have detailed structural information. Do not provide a specific anatomical location of the functional information. have a spatial resolution, signal to noise ratio and contrast lower than anatomical images

It is necessary to register the functional image with anatomical image to provide a geometrical localization, i.e. MRI. This type of registration is known as multimodal registration due to the different contrast and intensity of the datasets.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 5 / 29

• Introduction. Anatomic-functional images of the brain

The functional images of the brain, i.e. fMRI or PET,

Do not have detailed structural information. Do not provide a specific anatomical location of the functional information. have a spatial resolution, signal to noise ratio and contrast lower than anatomical images

It is necessary to register the functional image with anatomical image to provide a geometrical localization, i.e. MRI. This type of registration is known as multimodal registration due to the different contrast and intensity of the datasets.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 5 / 29

• Introduction. Anatomic-functional images of the brain

The functional images of the brain, i.e. fMRI or PET,

Do not have detailed structural information. Do not provide a specific anatomical location of the functional information. have a spatial resolution, signal to noise ratio and contrast lower than anatomical images

It is necessary to register the functional image with anatomical image to provide a geometrical localization, i.e. MRI. This type of registration is known as multimodal registration due to the different contrast and intensity of the datasets.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 5 / 29

• Introduction. Anatomic-functional images of the brain

The functional images of the brain, i.e. fMRI or PET,

Do not have detailed structural information. Do not provide a specific anatomical location of the functional information. have a spatial resolution, signal to noise ratio and contrast lower than anatomical images

It is necessary to register the functional image with anatomical image to provide a geometrical localization, i.e. MRI. This type of registration is known as multimodal registration due to the different contrast and intensity of the datasets.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 5 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

• Introduction. Software for image registration

ITK (2005)

Open source library implemented in C++ Only incorporates parametric methods

• L. Ibáñez et al., The ITK Software Guide, Kitware, Clifton Park, NY, 2005.

Elastix (2010)

Library in C++ based on ITK Non-rigid transformations use parametric models (B-Splines).

• S. Klein et al., “Elastix: A toolbox for intensity-based medical image registration”, IEEE Trans. Medical Imaging,
• vol. 29, no. 1, pp. 196-205, 2010.

FLIRT - Fast and Flexible Image Registration Toolbox (2007)

Library implemented in C++. Elastic registration (2D y 3D), diffusion (2D) and curvature (2D)

Nils Papenberg et al., “A Fast and Flexible Image Registration Toolbox”, Proc. of the BVM 2007. Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 6 / 29

Outline

1

Introduction

2

Methodology

3

Results

4

Conclusions

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 7 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

• Methodology. Registration framework

Non-rigid displacement field . Variational approach. Diffusion registration. Registration framework formulated in the frequency domain.

• J. Larrey-Ruiz et al., “A Fourier domain framework for variational image registration”, J. Math. Imaging Vis., vol.

32, no. 1, pp. 57-72, 2008.

Registration framework implemented in the frequency domain.

In terms of efficiency, two times faster than the fastest implementation available in the spatial domain .

• R. Verdú-Monedero et al., “Frequency implementation of the Euler-Lagrange equations for variational image

registration”, IEEE Signal Processing Letters, vol. 15, pp. 321-324, 2008 Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 8 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] R is the (Reference) dataset. T es the (Template) dataset. u(x) = (u1(x), u2(x), u3(x)) is the displacement field. x = (x1, x2, x3) is the spatial position. D is the energy term which measures the distance between reference set and template set. S is the penalty term (which acts as a regularizer). Adds a priori knowledge.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 9 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] Similarity terms used D: Correlation ratio DCR[R, T; u] := −CR[R, Tu] = −1 + E{Var{Tu|R}} Var{Tu} Regularizer term: diffusion energy Sdiff[u] := 1 2

3

• l=1
• Ω

∇ul2 dx

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 10 / 29

Methodology

Variational formulation. Joint energy functional

J [u] = D[R, T; u] + αS[u] Similarity terms used D: Correlation ratio DCR[R, T; u] := −CR[R, Tu] = −1 + E{Var{Tu|R}} Var{Tu} Regularizer term: diffusion energy Sdiff[u] := 1 2

3

• l=1
• Ω

∇ul2 dx

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 10 / 29

Methodology

Variational formulation. Joint energy functional

Solution u = min

u {J [u]}

Minimizer of the joint energy functional is equivalent to cancel the Gâteaux derivative of the join energy functional. This leads to the Euler-Lagrange equation in the spatial domain f(x; u) + αA[u](x) = 0

f(x; u) are the external forces, which are obtained from energy term. A[u](x) are the internal forces, which are obtained of regularizer term (restriction to the solution).

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 11 / 29

Methodology

Variational formulation. Joint energy functional

Solution u = min

u {J [u]}

Minimizer of the joint energy functional is equivalent to cancel the Gâteaux derivative of the join energy functional. This leads to the Euler-Lagrange equation in the spatial domain f(x; u) + αA[u](x) = 0

f(x; u) are the external forces, which are obtained from energy term. A[u](x) are the internal forces, which are obtained of regularizer term (restriction to the solution).

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 11 / 29

Methodology

Variational formulation. Joint energy functional

Solution u = min

u {J [u]}

Minimizer of the joint energy functional is equivalent to cancel the Gâteaux derivative of the join energy functional. This leads to the Euler-Lagrange equation in the spatial domain f(x; u) + αA[u](x) = 0

f(x; u) are the external forces, which are obtained from energy term. A[u](x) are the internal forces, which are obtained of regularizer term (restriction to the solution).

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 11 / 29

Methodology

Variational formulation. Joint energy functional

Solution u = min

u {J [u]}

Minimizer of the joint energy functional is equivalent to cancel the Gâteaux derivative of the join energy functional. This leads to the Euler-Lagrange equation in the spatial domain f(x; u) + αA[u](x) = 0

f(x; u) are the external forces, which are obtained from energy term. A[u](x) are the internal forces, which are obtained of regularizer term (restriction to the solution).

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 11 / 29

Methodology

Variational formulation. Joint energy functional

Solution of the E-L equation in the frequency domain ˜ f(ω) + α˜ A(ω)˜ u(ω) = 0

˜ f(ω) if the 3D FT of the external forces. ˜ A(ω) is a diagonal 3 × 3 matrix ˜ Aii(ω) = 2

3

• k=1

(1 − cos ωk) To solve the E-L equation we use the following semi-implicit iterative scheme ˜ u(k)

l

(ω) = 1 1 + τα ˜ All(ω)

• ˜

u(k−1)

l

(ω) − τ ˜ f (k−1)

l

(ω)

• Legaz et al. (UPCT-LABHUMAN)

IWINAC 2015 June, 3 12 / 29

Methodology

Variational formulation. Joint energy functional

Solution of the E-L equation in the frequency domain ˜ f(ω) + α˜ A(ω)˜ u(ω) = 0

˜ f(ω) if the 3D FT of the external forces. ˜ A(ω) is a diagonal 3 × 3 matrix ˜ Aii(ω) = 2

3

• k=1

(1 − cos ωk) To solve the E-L equation we use the following semi-implicit iterative scheme ˜ u(k)

l

(ω) = 1 1 + τα ˜ All(ω)

• ˜

u(k−1)

l

(ω) − τ ˜ f (k−1)

l

(ω)

• Legaz et al. (UPCT-LABHUMAN)

IWINAC 2015 June, 3 12 / 29

Methodology

Variational formulation. Joint energy functional

Solution of the E-L equation in the frequency domain ˜ f(ω) + α˜ A(ω)˜ u(ω) = 0

˜ f(ω) if the 3D FT of the external forces. ˜ A(ω) is a diagonal 3 × 3 matrix ˜ Aii(ω) = 2

3

• k=1

(1 − cos ωk) To solve the E-L equation we use the following semi-implicit iterative scheme ˜ u(k)

l

(ω) = 1 1 + τα ˜ All(ω)

• ˜

u(k−1)

l

(ω) − τ ˜ f (k−1)

l

(ω)

• Legaz et al. (UPCT-LABHUMAN)

IWINAC 2015 June, 3 12 / 29

Outline

1

Introduction

2

Methodology

3

Results

4

Conclusions

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 13 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Results

Registration of images of the brain of the same patient

the registration of functional and anatomic images (intra-patient)

• f the brain of the same patient is an essential part of the

functional localization process. Experiment 1: Registration of MRI (anatomical) with fMRI (functional). Experiment 2: Registration of MRI (anatomical) with PET (functional). Sizes of the datasets:

MRI: 336 × 336 × 200 voxels. fMRI: 64 × 64 × 30 voxels. PET: 128 × 128 × 46 voxels.

Re-sampled 128 × 128 × 64 voxels

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 14 / 29

Experiment 1: Registration MRI (anatomical) - fMRI (functional)

R(x) T(x − u(x)) T(x) Slice 20 Slice 30 Slice 40

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 15 / 29

Experiment 1: Registration MRI (anatomical) - fMRI (functional)

R(x) - T(x − u(x)) R(x) - T(x) Slice 20 Slice 30 Slice 40

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 16 / 29

Experiment 1: Registration MRI (anatomical) - fMRI (functional)

After registration: R(x) - T(x − u(x)) Before registration: R(x) - T(x) Slice 20

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 17 / 29

Experiment 1: Registration MRI (anatomical) - fMRI (functional)

After registration: R(x) - T(x − u(x)) Before registration: R(x) - T(x) Slice 30

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 18 / 29

Experiment 1: Registration MRI (anatomical) - fMRI (functional)

After registration: R(x) - T(x − u(x)) Before registration: R(x) - T(x) Slice 40

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 19 / 29

Results

Experiment 1: Registration MRI (anatomical) - fMRI (functional)

Registration parameters:

α = 20 ξ = 100 iterations Similarity measure: CR (multimodal registration)

PSNR MI CR Tiempo Before registration 21.64 dB 1.01 bits 64.54% – Proposed method 29.33 dB 1.24 bits 90.55% 71 s Elastix 28.70 dB 1.17 bits 88.53% 126 s

PC: 16GB RAM, Intel Core i5-2500K@3.3GHz, Windows 7 - 64 bits Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 20 / 29

Experiment 2: Registration MRI (anatomical) - PET (functional)

R(x) T(x − u(x)) T(x) Slice 20 Slice 30 Slice 40

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 21 / 29

Experiment 2: Registration MRI (anatomical) - PET (functional)

R(x) - T(x − u(x)) R(x) - T(x) Slice 20 Slice 30 Slice 40

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 22 / 29

Experiment 2: Registration MRI (anatomical) - PET (functional)

After registration: R(x) - T(x − u(x)) Before registration: R(x) - T(x) Slice 20

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 23 / 29

Experiment 2: Registration MRI (anatomical) - PET (functional)

After registration: R(x) - T(x − u(x)) Before registration: R(x) - T(x) Slice 30

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 24 / 29

Experiment 2: Registration MRI (anatomical) - PET (functional)

After registration: R(x) - T(x − u(x)) Before registration: R(x) - T(x) Slice 40

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 25 / 29

Results

Experiment 2: Registration MRI (anatomical) - PET (functional)

Registration parameters:

α = 50 ξ = 150 iterations Similarity measurement: CR (multimodal registration)

PSNR MI CR Tiempo Before registration 17.29 dB 0.82 bits 62.85% – Proposed method 27.56 dB 1.17 bits 88.27% 108 s Elastix 26.40 dB 1.12 bits 85.53% 192 s

PC: 16GB RAM, Intel Core i5-2500K@3.3GHz, Windows 7 - 64 bits Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 26 / 29

Outline

1

Introduction

2

Methodology

3

Results

4

Conclusions

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 27 / 29

Conclusions

Registration of anatomic and functional images of the same patient to help in the functional localization process. Application of medical image registration based on a efficient implementation of registration by diffusion. Formulated and implemented in the frequency domain. Non-rigid registration of images from different modalities. Results on different experiments show the ability, efficiency and high accuracy of the proposed method . Proposed method improves the results provides by Elastix, and reduces the computational time.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 28 / 29

Conclusions

Registration of anatomic and functional images of the same patient to help in the functional localization process. Application of medical image registration based on a efficient implementation of registration by diffusion. Formulated and implemented in the frequency domain. Non-rigid registration of images from different modalities. Results on different experiments show the ability, efficiency and high accuracy of the proposed method . Proposed method improves the results provides by Elastix, and reduces the computational time.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 28 / 29

Conclusions

Registration of anatomic and functional images of the same patient to help in the functional localization process. Application of medical image registration based on a efficient implementation of registration by diffusion. Formulated and implemented in the frequency domain. Non-rigid registration of images from different modalities. Results on different experiments show the ability, efficiency and high accuracy of the proposed method . Proposed method improves the results provides by Elastix, and reduces the computational time.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 28 / 29

Conclusions

Registration of anatomic and functional images of the same patient to help in the functional localization process. Application of medical image registration based on a efficient implementation of registration by diffusion. Formulated and implemented in the frequency domain. Non-rigid registration of images from different modalities. Results on different experiments show the ability, efficiency and high accuracy of the proposed method . Proposed method improves the results provides by Elastix, and reduces the computational time.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 28 / 29

Conclusions

Registration of anatomic and functional images of the same patient to help in the functional localization process. Application of medical image registration based on a efficient implementation of registration by diffusion. Formulated and implemented in the frequency domain. Non-rigid registration of images from different modalities. Results on different experiments show the ability, efficiency and high accuracy of the proposed method . Proposed method improves the results provides by Elastix, and reduces the computational time.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 28 / 29

Conclusions

Registration of anatomic and functional images of the same patient to help in the functional localization process. Application of medical image registration based on a efficient implementation of registration by diffusion. Formulated and implemented in the frequency domain. Non-rigid registration of images from different modalities. Results on different experiments show the ability, efficiency and high accuracy of the proposed method . Proposed method improves the results provides by Elastix, and reduces the computational time.

Legaz et al. (UPCT-LABHUMAN) IWINAC 2015 June, 3 28 / 29

Multimodal 3D Registration

• f Anatomic (MRI) and Functional (fMRI and PET)

Intra-Patient Images of the Brain

A.G. Legaz-Aparicio1, R. Verdú-Monedero1, J. Larrey-Ruiz1, F . López-Mir2, V. Naranjo2, A. Bernabeu3

1Universidad Politécnica de Cartagena, Cartagena 30202, Spain 2Universidad Politécnica de Valencia, I3BH LabHuman, 46022 Valencia, Spain 3Inscanner S.L, Unidad de Resonancia Magnética, 03016 Alicante, Spain

6th International Work-conference on the Interplay between Natural and Artificial Computation - IWINAC 2015