cuDIMOT: A CUDA toolbox for modelling the brain tissue - PowerPoint PPT Presentation

cuDIMOT: A CUDA toolbox for modelling the brain tissue microstructure from diffusion-MRI Mois´ es Hern´ andez Fern´ andez Istvan Reguly, Mike Giles, Stephen Smith and Stamatios N. Sotiropoulos GPU Technology Conference Europe 2017 Talk ID: 23165

Brain tissue microstructure We want to gain information about tissue microstructure from diffusion MRI (dMRI) data: Understand the brain mechanisms Develop new biomarkers Fibres dispersion Fibres Orientation Superior - Inferior Medial - Lateral Anterior - Posterior 0 1 Mois´ es Hern´ andez Fern´ andez, FMRIB cuDIMOT 2 / 23

Outline 1. Diffusion MRI 2. cuDIMOT: CUDA Diffusion Modelling Toolbox Parallel design Functionality and features 3. Results: Validation & Performance gains 4. Conclusions Mois´ es Hern´ andez Fern´ andez, FMRIB Outline 3 / 23

diffusion MRI (dMRI) Molecules are in constant motion. We want to quantify water diffusion within a tissue. Different tissues: Grey Matter: Diffusion without preferred direction. White Matter: Diffusion along preferred direction. Information about tissue microstructure features can be gained getting several diffusion-weighted measurements and modelling the diffusion process using biophysical parameters. Mois´ es Hern´ andez Fern´ andez, FMRIB 1. diffusion MRI 4 / 23

diffusion MRI (dMRI) Molecules are in constant motion. We want to quantify water diffusion within a tissue. Different tissues: Grey Matter: Diffusion without CSF Grey matter White matter preferred direction. White Matter: Diffusion along preferred direction. Information about tissue Particle 1 Particle 2 Particle 3 microstructure features can be gained getting several diffusion-weighted measurements and modelling the diffusion process using biophysical parameters. Mois´ es Hern´ andez Fern´ andez, FMRIB 1. diffusion MRI 4 / 23

diffusion MRI (dMRI) Molecules are in constant motion. We want to quantify water diffusion within a tissue. Different tissues: Grey Matter: Diffusion without preferred direction. White Matter: Diffusion along preferred direction. Information about tissue microstructure features can be gained getting several diffusion-weighted measurements and modelling the diffusion process using biophysical parameters. Mois´ es Hern´ andez Fern´ andez, FMRIB 1. diffusion MRI 4 / 23

cuDIMOT: Motivation Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Parellel design 5 / 23

cuDIMOT: Design Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Parellel design 6 / 23

cuDIMOT: Parallel design v1 Thread Block 0 Vx VOXELS 0 1 2 ................................... K-1 Vy VOXELS . . . Thread Block (Vx*Vy/K) -1 0 ................................... K-1 1 2 Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Parellel design 7 / 23

cuDIMOT: Parallel design v2 ... m 31 m 0 m 1 m 2 ... m 32 m 33 m 34 m 63 m 64 m 65 ... M measurements ... Thread Thread Thread Thread 0 (Leader) 1 2 31 Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Parellel design 8 / 23

cuDIMOT: Parallel design v2 Thread Block 0 Vx VOXELS Group 0 Thread Block (Vx*Vy/B) -1 0 1 2 .................................. 31 . Group 0 . Vy VOXELS . Group B-1 0 1 2 .................................. 31 . . . 0 1 2 .................................. 31 Group B-1 0 1 2 .................................. 31 Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Parellel design 9 / 23

cuDIMOT: Levenberg Implementation Cost function: sum of squared Levenberg differences between measurements Active Threads in a Block and model predictions I iterations 0 1 2 3 4 ...... 30 31 1. Step 1 Gradient descent method: needs 2. synchronise() ...... partial derivatives for Gradient and 3. Step 2 ...... 4. Step 3 Jacobian ( NParameters × 5. synchronise() ...... NMeasurements ) 6. Step 4 7. synchronise() ...... 8. Step 5 Threads collaborate for computing the partial derivatives STEPS: 1. Calculate partial derivatives for all the parameters. 2. Calculate Gradient & Jacobian & Hessian Hessian = Jacobian ∗ Jacobian T 3. LU solver 4. Compute model predicted signal & squared residuals Shuffle instructions 5. Calculate Cost function Accept/Reject step & Adapt λ Thread working Idle thread 2 warps (and voxels) per block Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Parellel design 10 / 23

A Generic toolbox Options at compilation time: Bounds - Lower and/or Upper limits (any routine). Levenberg kernel implements reparameterisations internally Priors (MCMC): Gaussian probability distribution Gamma probability distribution Automatic Relevance Determination (ARD) Angle uniformly distributed on a sphere Constraints: relation between parameters Different noise models: Gaussian & Rician Numerical differentiation in Levenberg kernel Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Functionality 11 / 23

A Generic toolbox Options at compilation time: Model Predicted Signal f( θ )= θ 1 * exp(- θ 2 * x) and Partial Derivatives MACRO T Predicted_Signal (int npar, T* P, T* CFP, T* FixP){ return P[0]*exp(-P[1]*CFP[0]); } MACRO void Partial_Derivatives (int npar, T* P, T* CFP, T* FixP, T* derivatives){ derivatives[0]=exp(-P[1]*CFP[0]); derivatives[1]=-P[0]*CFP[0]*exp(-P[1]*CFP[0]); } Parameters Bounds and Priors bounds[0] = (80,120) bounds[1] = (,1.5) prior[0] = Gaussian(100,10) prior[1] = ARD(1) Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Functionality 12 / 23

A Flexible toolbox Functionality at execution time: Choosing fitting routines: Grid search, Levenberg-Marquardt, MCMC Selecting number of iterations in Levenberg-Marquardt Selecting number of iterations in MCMC (burn-in, total, sample thinning interval) Cascaded model fitting (Initialising parameters) Choose parameters of the model to be kept fixed during the fitting process Bayesian & Akaike Inference Criterion The toolbox can be easily extended Mois´ es Hern´ andez Fern´ andez, FMRIB 2. cuDIMOT: Functionality 13 / 23

Validation: Fibre Orientation We have implemented several diffusion models Fibre Orientation estimation: Coronal Sagittal Axial CPU Superior - Inferior Medial - Lateral Anterior - Posterior GPU cuDIMOT Mois´ es Hern´ andez Fern´ andez, FMRIB 3. Results: Validation 14 / 23

Validation: Fibre Orientation Mean estimates: 1000 repeats ph 1 S 0 d f 1 f 2 th 1 3.09 0.70 1800 0.40 1.4 0.0028 0.65 0.35 3.08 1795 1.3 0.60 0.0026 0.30 1790 3.07 0.55 1.2 Corpus 0.25 Callosum 1785 0.0024 0.50 3.06 0.20 1.1 1780 0.45 0.15 0.0022 3.05 0.40 1.0 0.10 1775 3.04 0.35 0.0020 0.05 0 1 0 1 0 1 0 1 0 1 0 1 0.00125 1.10 1970 0.260 2.115 0.225 1.09 0.00120 0.255 1960 2.110 1.08 0.220 0.00115 2.105 0.250 1950 1.07 0.00110 0.215 2.100 0.245 1940 Centrum 1.06 2.095 Semiovale 0.00105 0.240 0.210 1930 1.05 2.090 0.00100 0.235 0.205 1.04 1920 2.085 0.230 0.00095 0.200 1.03 1910 2.080 0.00090 0.225 1.02 0.195 2.075 0 1 0 1 0 1 0 1 0 1 0 1 0.00120 1350 5.0 0.12 0.20 1.8 0.00115 4.5 0.10 1340 1.7 0.18 4.0 0.08 0.00110 1330 1.6 Grey 3.5 0.16 0.06 Matter 0.00105 3.0 1.5 0.04 1320 0.14 2.5 0.00100 0.02 1.4 2.0 1310 0.12 0.00 1.3 0.00095 1.5 0 1 0 1 0 1 0 1 0 1 0 1 CPU GPU cuDIMOT Mois´ es Hern´ andez Fern´ andez, FMRIB 3. Results: Validation 15 / 23

Validation: Fibre Orientation Standard deviation estimates: 1000 repeats Orientation Uncertainty 1 S 0 d f 1 f 2 0.006 28 0.12 0.12 0.00030 0.005 26 0.10 0.10 24 0.00025 0.004 0.08 0.08 22 0.00020 Corpus 0.003 20 Callosum 0.06 0.06 0.00015 18 0.002 0.04 0.04 16 0.00010 0.001 0.02 14 0.02 0.00005 0.000 12 0 1 0 1 0 1 0 1 0 1 30.0 0.016 0.006 0.00012 0.016 27.5 0.014 0.005 0.00010 0.014 25.0 Centrum 22.5 0.00008 0.012 0.012 0.004 Semiovale 20.0 0.00006 0.010 0.010 0.003 17.5 0.00004 0.008 15.0 0.008 0.002 0.00002 12.5 0 1 0 1 0 1 0 1 0 1 26 0.06 24 0.06 0.00010 0.4 22 0.05 0.05 0.3 0.00008 0.04 20 0.04 Grey 0.03 18 0.2 Matter 0.00006 0.03 0.02 16 0.1 0.02 0.00004 14 0.01 0.01 12 0.00 0.0 0.00002 0 1 0 1 0 1 0 1 0 1 CPU GPU cuDIMOT Mois´ es Hern´ andez Fern´ andez, FMRIB 3. Results: Validation 16 / 23

Validation: Fibre Dispersion Resources Fiso Fintra OD / Time 72 CPU NODDI cores Watson 40 Difference Difference Difference MATLAB hours % Fiso % Fintra % OD 1 GPU NODDI Watson 6.8 cuDIMOT minutes 0 0.2 0 1 0% 20% Mois´ es Hern´ andez Fern´ andez, FMRIB 3. Results: Validation 17 / 23

Validation: Fibre Dispersion Mois´ es Hern´ andez Fern´ andez, FMRIB 3. Results: Validation 18 / 23

Performance Time fitting different dMRI models Speedup 6.98x Times in seconds (logarithm scale) 10 5 CPU tool - 72 cores cuDIMOT - single K80 GPU Speedup 3.7x Speedup 3.26x Speedup 352x 10 4 Speedup 3.88x Speedup 3.85x 10 3 10 2 10 1 NODDI NODDI Ball & Ball & Ball & Ball & Watson Bingham 1 Stick 1 Stick 2 Sticks 2 Sticks Matlab Matlab C++ Gamma C++ Gamma C++ C++ Mois´ es Hern´ andez Fern´ andez, FMRIB 3. Results: Performance gains 19 / 23