COMP 364: Computer Tools for Life Sciences Introduction to image - PowerPoint PPT Presentation

COMP 364: Computer Tools for Life Sciences Introduction to image analysis with scikit-image (part one) Christopher J.F. Cameron and Carlos G. Oliver 1 / 27

Key course information Quiz #9 ◮ the penultimate quiz ◮ available Monday, November 27th (closes 11:59:59 pm) ◮ covers topics from the last two weeks HW5 ◮ available early next week ◮ due Thursday, December 7th, 2017 at 11:59:59 pm ◮ shorter than previous assignments Course evaluations ◮ available now at the following link: ◮ https://horizon.mcgill.ca/pban1/twbkwbis.P_ WWWLogin?ret_code=f 2 / 27

Why perform digital image analysis? Digital image analysis (DIA) The extraction of useful information from images ◮ important for good feature desgin ◮ emphasizes important traits while diluting noisy ones For example ◮ in machine vision, image preprocessing plays a huge role ◮ before extracting features from an digital image ◮ it’s extremely useful to be able to augment it ◮ to highlight aspects that are important for the machine learning task to stand out 3 / 27

DIA in Python scikit-image module or (skimage) ◮ image processing module in Python ◮ holds a wide library of image processing algorithms: filters, transforms, point detection ◮ API ◮ http://scikit-image.org/docs/dev/api/api.html We’ll start with an example image using the io module ◮ basic I/O submodule of scikit-image ◮ API ◮ http: //scikit-image.org/docs/dev/api/skimage.io.html 4 / 27

5 / 27

Reading an image into memory import skimage.io as io 1 2 # read image into memory 3 image = io.imread("./../images/monkey.jpg") 4 # print top-left pixel RGB values 5 print(image[0,0]) 6 # prints: [255 255 255] 7 # write image to disk 8 io.imsave("./../images/monkey_copy.jpg",image) 9 What are RGB values? 6 / 27

RGB colors Red green blue (RGB) An RGB color value is specified with: rgb(red, green, blue) Each parameter (red, green, and blue) defines the intensity of the color as an integer between 0 and 255 For example, rgb(0, 0, 255) ◮ is rendered as blue ◮ because the blue parameter is set to its highest value (255) ◮ the others are set to 0 RGB color picker/codes chart: http://www.rapidtables.com/web/color/RGB_Color.htm 7 / 27

Handling colors Let’s make copies of our image and ◮ increase intensity for each color intensity ◮ red, green, blue ◮ note: the format of our image object is ◮ image[ #ycoordinate , #xcoordinate , [red green blue]] ◮ top-left pixel is [0, 0, [red green blue]] red, green, blue = image.copy(), 1 image.copy(), image.copy() 2 red[:,:,(1,2)] = 0 # NumPy indexing 3 green[:,:,(0,2)] = 0 4 blue[:,:,(0,1)] = 0 5 io.imsave("./../images/monkey_red.jpg",red) 6 io.imsave("./../images/monkey_green.jpg",green) 7 io.imsave("./../images/monkey_blue.jpg",blue) 8 8 / 27

red intensity green intensity blue intensity 9 / 27

Grayscaling Most image processing algorithms assume a 2D matrix ◮ not an image with a third dimension of color To bring the image into two dimensions ◮ we need to summarize the three colors into a single value ◮ this process is more commonly know as grayscaling ◮ where the resulting image only holds intensities of gray ◮ with values between 0 and 1 skimage submodule color has useful functions for this task ◮ API http://scikit-image.org/docs/dev/api/skimage. color.html 10 / 27

from skimage.color import rgb2gray 1 2 # read image into memory 3 image = io.imread("./../images/monkey.jpg") 4 # convert to grayscale 5 gray_image = rgb2gray(image) 6 io.imsave("./../images/monkey_grayscale.jpg",gray_image) 7 print(image[0,0]) 8 # prints: [255 255 255] 9 print(gray_image[0,0]) 10 # prints: 1.0 11 After we view the grayscale image ◮ let’s find a better way to view the transformation using histograms 11 / 27

12 / 27

Histogram of RGB intensities import matplotlib.pyplot as plt 1 2 for index,label in zip([0,1,2],["red","green","blue"]): 3 # .flatten() converts 2D list to 1D 4 plt.hist(image[:,:,(index)].flatten(),50 5 ,label=label,edgecolor="k",linewidth=1, 6 facecolor=label[0],alpha=0.75) 7 plt.xlabel("pixel intensity",size=16) 8 plt.xlim([0,255]) 9 plt.ylabel("frequency",size=16) 10 plt.tight_layout() 11 plt.savefig("./../images/histogram_"+label+".png") 12 plt.close() 13 13 / 27

Red Green Blue 14 / 27

15 / 27

Image enhancement - histogram equalization Histogram equalization (HE) Take a grayscale image ◮ attempt to distribute intensities more evenly along the range of possible values (0 to 1) ◮ pixels still rank the same ◮ a pixel with a higher value than another will still have a higher value after the transform ◮ ...but the image as a whole becomes far more contrasted and normalized We’ll use the submodule exposure to perform HE ◮ API http://scikit-image.org/docs/dev/api/skimage. exposure.html 16 / 27

HE with Python’s skimage module from skimage.exposure import equalize_hist 1 2 gray_image = rgb2gray(image) 3 print(gray_image[0,0]) 4 # prints: 1.0 5 equalized_image = equalize_hist(gray_image) 6 print(equalized_image[0,0]) 7 # prints: 1.0 8 io.imsave("./../images/monkey_HE.jpg",equalized_image) 9 Based on what you have learned about HE ◮ why does the top-left most pixel’s value not change? 17 / 27

Grayscale Histogram equalization Why does the image become more contrasted ? ◮ pixels that started with similar intensity values ◮ which were relatively hard to differentiate ◮ are now more distinctly separated Let’s look at the histograms for both images 18 / 27

# plot hist of HE pixel intensities 1 plt.hist(equalized_image[:,:].flatten(),50,label="HE", 2 edgecolor="k",linewidth=1,facecolor="blue", 3 alpha=0.75) 4 # plot hist of grayscale pixel intensities 5 plt.hist(gray_image[:,:].flatten(),50, 6 label="grayscale",edgecolor="k",linewidth=1, 7 facecolor="red",alpha=0.75) 8 plt.xlim([0,1]) 9 plt.xlabel("pixel intensity",size=16) 10 plt.ylabel("frequency",size=16) 11 plt.legend(loc="best") 12 plt.tight_layout() 13 plt.savefig("./../images/histogram_HE.png") 14 plt.close() 15 19 / 27

20 / 27

Image enhancement - binarizing and blurring Sometimes, it helps to simplify an image even further ◮ instead of grayscale, binarize the image ◮ results in each pixel hold only one of two values ◮ more commonly recognized as a pure black and white image The objective is to separate the foreground from the background ◮ to make feature generation even easier A simple way of doing this is to just choose a threshold ◮ every pixel above that threshold is set to 1 ◮ every pixel below it to 0 21 / 27

Binarizing and blurring In our case, ◮ we’ll select the mean of our grayscale image as the threshold ◮ every pixel above the mean is set to white (1.0) ◮ those below are set to black (0.0) import numpy as np 1 2 gray_image = rgb2gray(image) 3 #print(gray_image[0,0]) 4 # prints: 1.0 5 binary_image = np.where(gray_image > np.mean(gray_image) 6 ,1.0,0.0) 7 io.imsave("./../images/monkey_binary.jpg",binary_image) 8 print(binary_image[0,0]) 9 # prints: 1.0 10 22 / 27

23 / 27

Image enhancement - blurring/smoothing Binary images may capture more detail than is helpful ◮ for example, the objective is to identify prominent features of the image ◮ monkey’s hands and fur, foliage etc. ◮ the position for every piece of fur (or leaf) isn’t necessary ◮ blurring/smoothing the image is a reasonable alternative Scikit-image’s Gaussian filter ( filter submodule) ◮ takes a weighted average of surrounding pixels ◮ so individual pixels incorporate local intensities into their own ◮ this produces a pretty recognizable blur/smoothing effect 24 / 27

from skimage.filters import gaussian 1 2 equalized_image = equalize_hist(gray_image) 3 for sigma,name in zip([3,6],["blurred","really_blurred"]): 4 blurred_image = gaussian(equalized_image,sigma=sigma) 5 fig,ax = plt.subplots() 6 ax.imshow(blurred_image,vmin=0, vmax=1) 7 # remove ticks 8 ax.set_xticks([]) 9 ax.set_yticks([]) 10 # remove spines 11 for spine in ["top","bottom","right","left"]: 12 ax.spines[spine].set_visible(False) 13 plt.savefig("./../images/monkey_"+name+".jpg") 14 plt.close() 15 25 / 27

Sigma = 3 Sigma = 6 The equalized image (rather than grayscale) has been used ◮ maintains a high level of contrast through the filtering ◮ as sigma increases, so does the blurring 26 / 27

Next time in COMP 364 More digital image analyses! ◮ edge and corner detection ◮ maybe more? 27 / 27