Einfhrung in Visual Computing U it 15 I Unit 15: Image Segmentation - PowerPoint PPT Presentation

Einführung in Visual Computing U it 15 I Unit 15: Image Segmentation S t ti http:// www.caa.tuwien.ac.at/cvl/teaching/sommersemester/evc  Content:  Introduction  Greylevel Thresholding  Greylevel Thresholding  Clustering  Relaxation Labelling  Region Growing  Split and Merge 1 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g Image Segmentation Sky Tree Tree ? ? Grass 2 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g The shape shape of an object can be described in terms of:  Its boundary boundary – requires image edge detection edge detection  The region region it occupies – requires image segmentation image segmentation in homogeneous regions, Image regions generally have g g g g g y homogeneous characteristics (e.g. intensity, texture)  The purpose p p purpose of image segmentation is to partition an image into p p g g p g meaningful regions meaningful regions with respect to a particular application  The segmentation is based on measurements The segmentation is based on measurements measurements taken from the measurements taken from the image like  Greylevel Color Texture  Greylevel, Color, Texture  Depth  Motion 3 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Finding Objects in Images g j g  To do this we need to divide the image into two parts g p  the object of interest (the foreground foreground ) foreground foreground )  everything else (the Sky background background ) background ) background Tree Tree Tree  The definition of foreground Th d fi i i f f d ? ? and background depends on the task at hand th t k t h d Grass 4 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g  Usually image segmentation is an initial step initial step in a series of processes aimed at image understanding of processes aimed at image understanding image understanding image understanding  Applications of image segmentation include A li ti f i t ti i l d  Identifying Identifying objects in a scene for object ‐ based measurements measurements such as size and shape h i d h  Identifying objects in a moving scene for object ‐ based video compression compression (MPEG4) compression compression (MPEG4)  Identifying objects which are at different distances from a sensor using depth measurements sensor using depth measurements depth measurements from a laser range finder depth measurements from a laser range finder enabling path planning path planning for a mobile robot 5 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g  Example 1  Segmentation based on greyscale  Segmentation based on greyscale greyscale greyscale  Very simple ‘model’ of greyscale leads to inaccuracies in object labelling labelling 6 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g  Example 2  Segmentation based  Segmentation based on texture texture  Enables object  Enables object surfaces with varying patterns of grey to patterns of grey to be segmented 7 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g  Example 3  Segmentation based on motion  Segmentation based on motion motion motion  The main difficulty of motion segmentation is that an segmentation is that an intermediate step is required to (either implicitly or explicitly) (either implicitly or explicitly) estimate an optical flow field  The segmentation must be based  The segmentation must be based on this estimate and not, in general the true flow general, the true flow 8 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Introduction to Image Segmentation g g  Example 4  Segmentation based on depth  Segmentation based on depth depth depth Range image  This example shows a range image, obtained with a laser image obtained with a laser range finder  A segmentation based on the A segmentation based on the range (the object distance from Segmented image the sensor) is useful in guiding the sensor) is useful in guiding mobile robots 9 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Image Segmentation g g  In analysis of objects in images it is essential to distinguish between objects of interest objects of interest and "the rest" = background background .  Techniques that are used to find objects of interest are referred to as segmentation segmentation techniques ‐ segmenting the foreground from background.  Image segmentation describes the division of the image in homogenous segments homogenous segments (no abrupt intensity changes within segments)  Edge detection Edge detection may (not necessarily) produce a segmentation 10 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

What do we mean by “Labeling” an Image? y g g  When we say we ”extract “ an object in an image , we mean that we identify the pixels identify the pixels that make it up. y y p p p  To express this information , we create an array of the same size as the original image and we give to each pixel as the original image and we give to each pixel each pixel a label each pixel a label label. label.  All pixels that make up the object object are given the same label same label . The label is usually a number but it could be anything: a letter or label is usually a number, but it could be anything: a letter, or color.  Often label images  Often label images label images are also referred to as classified images as label images are also referred to as classified images as they indicate the class class to which each pixel each pixel belongs. 11 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

How can we divide an Image into Uniform Regions ? Regions ?  Segmentation techniques can be classified as either contextual or non ‐ contextual .  Non ‐ contextual techniques ignore ignore the relationships that exist between features between features features in an image; pixels are simply grouped features in an image pixels are simply grouped grouped together grouped together on the basis of some global attribute global attribute , such as grey level.  Contextual techniques , additionally exploit exploit the relationships between image features features . Thus, a contextual technique might group together pixels that have similar grey levels and are close to one another . 12 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Greylevel Threshholding 13

Greylevel Histogram ‐ based Segmentation y g g  First, we will look at two very simple non non ‐ contextual contextual image segmentation techniques that are based on the greylevel g q g greylevel g y y histogram histogram of an image:  Thresholding Thresholding  Clustering  We will use a very simple object ‐ background test image  We will consider a zero, low and high noise image 14 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Greylevel Histogram ‐ based Segmentation y g g Noise free Low noise High noise  How do we characterise low noise and high noise? 15 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Greylevel Histogram ‐ based Segmentation y g g Noise free Low noise High noise  We can consider the histograms of our images  For the noise free image, its simply two spikes For the noise free image its simply two spikes two spikes at i=100, i=150 two spikes at i 100 i 150  For the low noise image, there are two clear peaks two clear peaks centred on i 100 i 150 i=100, i=150  For the high noise image, there is a single peak single peak – two greylevel populations corresponding to object and background have merged 16 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Greylevel Histogram ‐ based Segmentation y g g Low noise Noise free Hi h High noise i 17 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation

Greylevel Histogram ‐ based Segmentation y g g  We can define the input image Signal Signal ‐ to to ‐ Noise Noise ratio ratio in terms of the mean mean greylevel greylevel value of the object g g y y object pixels and background j p background g pixels and the additive noise standard deviation noise standard deviation     b o S / N   For our test images :  S/N (noise free) =   S/N (low noise) = 5  S/N (high noise) = 2 18 Robert Sablatnig, Computer Vision Lab, EVC ‐ 15: Image Segmentation