6. Image databases Image representations: Digitized (sampled) - PowerPoint PPT Presentation

6. Image databases Image representations: � Digitized (sampled) representation of field-based spatial data � ‘ Raw ’ images ≈ digital images ≈ bitmapped images ≈ raster images � m × n matrix of pixels, resolution = sampling rate, pixels per inch � Each pixel represented by k bits (= accuracy = color depth ); 2 k possible values. Image types: � Binary (bi-level) images (black = 0, white = 1; e.g. telefax) � Grey-scale images (usually k = 8; enables 256 grey-levels) � Color images (various representations) Sources: � Devices: scanner, digital camera, electron microscope, medical imaging devices (PET, MRI) � Wavelengths: visible light, infrared, X-rays MMDB-6 J. Teuhola 2012 141

Color images ‘True’ color schemes: � Three components per pixel (possibly 4 th for α -channel = transparency) � RGB = Red - Green - Blue (typically 3 x 8 = 24 bits per pixel) � CMY = Cyan - Magenta - Yellow ( CMYK used in printing, K = black) � HSI = Hue - Saturation - Intensity (used in image processing) � YUV ≈ YCbCr = Luminance (brightness) + 2 x chrominance (color) � Used in image compression (JPEG) � Correlations between color components reduced. � Most information is collected in the Y -component. Indexed color schemes: � Palette of e.g. 256 colors � Mapping table from color indices to RGB-values � Saves space, sufficient for many applications MMDB-6 J. Teuhola 2012 142

Image formats Tens of formats exist for different environments and applications, e.g. � BMP = Bitmap image file (MS Windows) � GIF = Graphics Interchange Format (indexed colors; sincludes compression, supported by web browsers) � JBIG = Joint Bi-level Image experts Group file interchange format � JPEG = Joint Photographic Experts Group (JFIF = JPEG File Interchange Format) � JP2 = JPEG 2000 � PBM = Portable Bitmap Format (black-and-white) � PGM = Portable Greymap Format (grey-scale) � PPM = Portable Pixmap Format (color) � PNG = Portable Network Graphics � TIFF = Tagged Image File Format (large number of options) MMDB-6 J. Teuhola 2012 143

Image compression � Necessary for large image archives: � saves space, � reduces transmission time. � Possible due to redundancy in images � Several methods specialized for different types of images � Image formats with compression: � JPEG, based on cosine transform � JPEG 2000, based on wavelet transform � GIF, based on LZW string compression � PNG, based on LZ77 string compression � JBIG (bi-level images), based on prediction by context MMDB-6 J. Teuhola 2012 144

Compression method characteristics � Lossless / lossy methods: Can the original image be recovered precisely or only approximately? E.g. JPEG is typically lossy. � Compression efficiency (bit rate), measured in bits/pixel � Speed (separately for compression and decompression) � Distortion (for lossy methods): � MAE = Mean Absolute Error � MSE = Mean Square of Errors � RMS = Root Mean Square error � SNR = Signal to Noise Ratio � PSNR = Peak Signal to Noise Ratio � Robustness against transmission errors � Blockiness , blurring , ... (for lossy methods) MMDB-6 J. Teuhola 2012 145

Searching from an image database 1. Using a hierarchical classification of images: The user follows paths in the hierarchy, e.g. Art works Paintings France 18 th century 2. Search using keywords in metadata Images can be considered similar to documents with index terms 3. Search by content features Pattern matching based on similarity with a query image, shape, color distribution, etc. MMDB-6 J. Teuhola 2012 146

Feature extraction and indexing of images � Extraction of descriptive attributes from images � Manually, automatically, or using a hybrid scheme (automatic segmentation & manual assignment of properties). Manual indexing: � Performed by a ‘knowledge worker’, trained on patterns and vocabulary of the image database application � Multiple indexers: Strict consistency rules, common glossary. � Automatic tools may help in pattern recognition. � Each interesting object (spatial structure) is presented manually to the system for indexing, equipped with descriptive attributes. � Assistance in selecting index terms: Hierarchical dictionaries, cross- referencing systems, domain thesaurus. � Time-consuming and costly; possibility to community-indexing, cf. http://gimp-savvy.com/PHOTO-ARCHIVE/ MMDB-6 J. Teuhola 2012 147

Automatic indexing � Specialized for various application domains (document recognition, optical character recognition (OCR), engineering drawings, x-rays, ...) � The system must first ‘learn’ and categorize domain element objects. � A certain amount of uncertainty (fuzziness) must be tolerated. � Important area of automatic image analysis and object recognition: Transformation of paper documents into digital form, and indexing those documents appropriately (so called document imaging → digital libraries ). MMDB-6 J. Teuhola 2012 148

Color feature extraction � Usually based on color histograms , i.e. number of pixels of each color (or color component): #pixels #pixels #pixels 0 0 0 255 255 255 GREEN RED BLUE � Separate histograms can be built for various subregions of the image (e.g. top-left, top-right, middle, ...) � The quantification can be made coarser than 0..255 by grouping adjacent histogram values, in order to reduce the dimensionality of the resulting feature vectors. MMDB-6 J. Teuhola 2012 149

Image segmentation � Detection of interesting regions within images. � A segment is a connected region that satisfies a homogeneity predicate. � Basis for subsequent search. � One of the most difficult tasks in image processing. � Several possible (heuristic) methods. Connected region: � For each pair ( x 1 , y 1 ), ( x n , y n ) of pixels, there exists a chain of pixels {( x 1 , y 1 ), ..., ( x n , y n )} in the region such that {( x i , y i ), ( x i +1 , y i +1 )} are adjacent for all i . MMDB-6 J. Teuhola 2012 150

Examples of homogeneity predicates � Binary images: p % of the pixels of the connected region have the same color (black or white) � Classified grey-scales, e.g. 0...9, 10...19, etc. A connected region is homogeneous, if at least p % of its pixels belong to the same class. � Dynamic grey-scale classification: Class boundaries are not predefined, but the interval size is: p % of the cells should have a grey-level within δ units. � Grey-scale images with a reference function f for homogeneity: The number of pixels in { ( x , y ) | ⏐ grey-level ( x , y ) - f ( x , y ) ⏐ < δ } should be at least p % of the pixels in the region. MMDB-6 J. Teuhola 2012 151

Miscellaneous segmentation techniques (a) Regular block segmentation : � Example: Quadtree or binary tree decomposition until homogeneous regions are obtained. � Does not usually satisfy the maximality condition for segmentation: Neighboring blocks may constitute a homogeneous region. � Generalization of binary tree segmentation: blocks can be split in any direction: polygon segmentation. Compromise solution: splitlines only in 0 ° , 45 ° , 90 ° , and 135 ° directions. (b) Splitting and merging : � Augments category (a) methods to satisfy the maximality condition. � Merging tests the obtained regions pairwise for homogeneity. � Does not usually produce a unique segmentation for an arbitrary homogeneity predicate. MMDB-6 J. Teuhola 2012 152

Miscellaneous segmentation techniques (cont.) (c) Thresholding : � Applicable, if objects of interest and the background have sufficiently distinct grey-level values. � The grey-level histogram of the image has two or more peaks, between which we can choose the threshold grey-level values. � Must usually be augmented with more sophisticated techniques. (d) Region growing : � Start from a set of seed points. � Include neighboring pixels as long as homogeneity holds. � Difficulty: How to choose the seeds? (e) Edge-following algorithms : � Follow a (hopefully circular) path of largest gradients (steepest slope) around the object to be detected. MMDB-6 J. Teuhola 2012 153

154 MMDB-6 J. Teuhola 2012 Threshold = 128 Example: thresholding

Examples: Region growing Edge detection 0 -1 0 Tolerance = 80 Convolution kernel: -1 4 -1 0 -1 0 MMDB-6 J. Teuhola 2012 155

Segment feature extraction from images � Various approaches, e.g. � area of the segment � eccentricity/circularity � shape approximation � curvature � Desirable properties of segment features: � Invariance to translation � Invariance to rotation � Invariance to scaling MMDB-6 J. Teuhola 2012 156

Representing the shape of segments � See: Sven Loncaric: ” A Survey of Shape Analysis Techniques”, Pattern Recognition, 31 (8), pp. 983-1001,1998 � Example: boundary scalar transform: � Another possibility: tangent angles at regular intervals � Both can be made rotation and scaling invariant. � The resulting 1D-function is usually Fourier-transformed : amplitude ( magnitude ) values are rotation invariant; phase determines orientation and starting point). � The lower-frequency Fourier coefficients can be used as the feature vector representing the shape. About 20 is often enough. � Technical problems: Non-convex shapes; shapes with holes . MMDB-6 J. Teuhola 2012 157