Towards Automated Bioimage Analysis: Algorithms & Applications - PowerPoint PPT Presentation

Towards Automated Bioimage Analysis: Algorithms & Applications João Manuel R. S. Tavares tavares@fe.up.pt www.fe.up.pt/~tavares March 2, 2016

Outline • Introduction • Segmentation • Analysis of Objects – Tracking – Matching – Registration • 3D Reconstruction • Conclusions • Research Team • Publications & Events @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 2

Introduction

Presentation • Associate Professor at FEUP (DEMec) • Senior Research and Projects Coordinator of the Institute of Science and Innovation in Mechanical and Industrial Engineering (INEGI) • Habilitation in Mechanical Engineering from UP • PhD and MSc degrees in Electrical and Computer Engineering from FEUP • BSc degree in Mechanical Engineering from FEUP • Research Areas: Image Processing and Analysis, Biomedical Imaging, Biomechanics, Human Posture and Control, Product Development @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 4

Introduction • The researchers of Image Processing and Analysis aim the development of algorithms to perform fully or semi-automatically tasks performed by the (quite complex) human vision system Original images Computational 3D voxelized and poligonized models built Azevedo et al. (2010) Computer Methods in Biomechanics and Biomedical Engineering 13(3):359-369 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 5

Introduction • Image processing and analysis are topics of the most importance for our Society • Algorithms of image processing and analysis are frequently used, for example, in: – Natural Sciences – Sports – Biology – Industry – Engineering – Medicine @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 6

Introduction • Examples of common tasks involving algorithms of image processing and analysis: – Noise removal – Geometric correction – Segmentation , recognition (2D-4D) – Motion and/or deformation tracking and analysis, including matching and registration – 3D reconstruction – Assisted medical diagnosis and intervention @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 7

Introduction • Image: a matrix with n rows and m columns (and l in 3D) , being each basic element known as pixel (or voxel in 3D) @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 8

Introduction • Image Processing: by applying mathematical operations/rules using the values of the image pixels (or voxels in 3D) in the Cartesian or in another domain Sobel operator ⎡ ⎤ − 1 + 1 0 ⎢ ⎥ ∗ − 2 = G x 0 2 ⎢ ⎥ − 1 + 1 ⎢ ⎥ 0 ⎣ ⎦ 2 + G y 2 = G G x ⎡ ⎤ − 1 − 2 − 1 ⎢ ⎥ ∗ = G y 0 0 0 ⎢ ⎥ + 1 + 2 + 1 ⎢ ⎥ ⎣ ⎦ ( denotes convolution) ∗ @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 9

Introduction • Image Acquisition: a sensor captures the energy reflected or emitted by the imaged object http://what-when-how.com/introduction-to-video-and-image-processing/image-acquisition-introduction-to- video-and-image-processing-part-1 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 10

Introduction • Difficulties : noise, artifacts, occlusion, poor illumination, reflections, complex objects and backgrounds https://rahaddadi.files.wordpress.com/2011/05/face_black_and_white_optical_illusion_cool-s453x562-92306-5803.jpg http://s1.cdn.autoevolution.com/images/news/the-longest-traffic-jam-in-history-12-days-62-mile-long-47237_1.jpg @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 11

Introduction Usual Pipeline Image(s) Image(s) segmentation / enhancement / features extraction Image(s) correction image (pre)processing tracking 3D vision computer vision matching motion analysis registration image analysis / computational vision @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 12

Segmentation

Segmentation • It is intended to identify in a fully or semi- automatically manner objects (2D/3D) presented in static images or in image sequences • The most usual methodologies are based on thresholding, region growing, template matching, statistical, geometric or physical modeling, or artificial classifiers • It is one of the most usual operations involved in the computational analysis of objects in images • Frequent problems: noise, artifacts, low resolution, reduced contrast, shapes not previously known, occlusion, multiple objects, etc. @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 14

Segmentation • Image segmentation by threshold (binarization) Ma et al. (2010) Computer Methods in Biomechanics and Biomedical Engineering 13(2):235-246 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 15

Segmentation • Example: segmentation of contours in dynamic pedobarography (Otsu’s method, morphologic dilation, xor operation) pressure opaque layer lamp transparent layer lamp glass reflected light contact layer + glass mirror camera Segmented images Original images Bastos & Tavares (2004) Lecture Notes in Computer Science 3179:39-50 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 16

Segmentation • Image segmentation by region growing Ma et al. (2010) Computer Methods in Biomechanics and Biomedical Engineering 13(2):235-246 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 17

Segmentation • Example: segmentation of ear structures (region growing) Region Growing, x=215; y=254 X: 254 Y: 214 Index: 116.7 RGB: 0.459, 0.459, 0.459 Segmentation obtained Original Image (bony labyrinth) Barroso et al. (2011) CNME 2011 Ferreira et al. (2014) Computer Methods in Biomechanics and Biomedical Engineering 17(8):888-904 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 18

Segmentation • Segmentation of objects based on deformable templates Example of a deformable template (for the eye) Carvalho & Tavares (2006) CompIMAGE 2006, 129-134 Carvalho & Tavares (2007) VipIMAGE 2007, 209-215 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 19

Segmentation • Example: segmentation of eye features (deformable geometric template) Segmentation of the iris using a Original image and associated force (or energy) fields deformable template (a circle) Segmentation of an eye using an Carvalho & Tavares (2006) CompIMAGE 2006, 129-134 deformable template Carvalho & Tavares (2007) VipIMAGE 2007, 209-215 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 20

Segmentation • Segmentation of based on active shape models (point distribution models, optimization) Vasconcelos & Tavares (2008) Computer Modeling in Engineering & Sciences 36(3):213-241 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 21

Segmentation • Example: analysis of the vocal tract during speech production from MR images (active shape model) Original image Final segmentation Vasconcelos et al. (2011) Journal of Voice 25(6):732-742 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 22

Segmentation • Segmentation of objects based on active contours (i.e. snakes – parametric models ) 1 ∫ E snake = ( v ( s )) + E ext ( v ( s )) ds E int s = 0 2 2 + β ( s ) d 2 v ( s ) E int = α ( s ) dv ( s ) ds 2 ds Tavares et al. (2009) International Journal for Computational Vision and Biomechanics 2(2):209-220 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 23

Segmentation • Example: segmentation of medical images (active contours - snakes) Initial contour Final contour Tavares et al. (2009) International Journal for Computational Vision and Biomechanics 2(2):209-220 Gonçalves et al. (2008) Computer Modeling in Engineering & Sciences 32(1):45-55 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 24

Segmentation • Segmentation of objects based on the level-set method ( geometrical models ) Typical form of the motion equation: ∂ φ ∂ t + F Δ φ = 0 Ma et al. (2010) Medical Engineering & Physics 32(7):766-774 Ma et al. (2010) Computer Methods in Biomechanics and Biomedical Engineering 13(2):235-246 @2016 João Manuel R.S. Tavares Towards Automated Bioimage Analysis: Algorithms & Applications 25