Content Based Image Retrieval Techniques Ambrose Tuscano - PowerPoint PPT Presentation

Content Based Image Retrieval Techniques Ambrose Tuscano (atuscan1@umbc.edu) University of Maryland Baltimore County, CMSC 676 Information Retrieval

Introduction Image retrieval systems aim to find similar images to a query image among an image dataset.

Represented as • Pixels (Also called Rasters) • Vectors

● By annotation (manual) • Text retrieval • Semantic level (good for picture with people, architectures) ● By the content (automatic) • Color, texture, shape • Vague description of picture (good for pictures of scenery and with pattern and texture)

Features in an Image ● Color : Low level, Can't specify context. ● Texture : Produce a mathematical characterisation of a repeating pattern in the image. ● Shape: Region based and Contour(outline) based. ● Local Image Features : small parts of a big image. ○ extracted from the images at salient points and dimensionality reduced using Principal Component Analysis (PCA) transformation ○ SIFT using Harris interest points

Structure

IMAGE RETRIEVAL METHODS

Text based Image Retrieval First annotated the images by text and then used text-based database management systems to perform image retrieval.

Text based Image Retrieval Three Ways to go ● Manually Assign Keywords to each image ● Use text associated with the images (captions, web pages) ● Analyse the image content to automatically assign keywords(Computer Vision?)

Content Based Image Recognition ● A technique which uses visual contents to search images from large scale image databases according to users' interests. ● CBIR research is mainly contributed by the computer vision community

Content based Image Recognition To use local features for image retrieval, three different methods are available: ● Direct transfer: nearest neighbors for each of the local features of the query searched and the database images containing most of these neighbors returned. ● Local feature image distortion model (LFIDM): Compares the distances between local features from the query image to the local features of each image of the database . The images with the lowest total distances are returned. ● Histograms of local features: A reasonably large amount of local features from the database is clustered and then each database image represented by a histogram of indices of these clusters.

● Color Histogram ● Color Correlogram ● Color AutoCorrelogram ● Color Coherence vector ● Dominant Color Descriptors

● A shape is the form of an object or its external boundary, outline,or external surface, as opposed to other properties like color, texture or composition. ● Fourier Descriptors ● Canny Algorithm ● SIFT Descriptors ● Moment Invariants ● Eccentric and Axis Oriented

● ● ● ● ●

● ● ● ○ ● ○ ○ ○

● Smoothing: Blur image to remove Noise ● FInd Gradients : Edges are marked where gradients of image have large magnitudes. ● Non-Max Suppression: Only local Maxima is marked for edges. ● Double Thresholding: Potential Edges are determined ● Hysteresis : Finally Edges which are not connected/near to many other potential edges are removed.

Texture Extraction- Motif Co-Occurrence Matric ● MCM is used to represent transveral of adjacent pixel color difference in an image. ● Each Pixel corresponds to four adjacent pixel colors ● Each image can be presented by four images of motifs of scan pattern, which can be further constructed into four two dimensional matrices of the image size. ● The attribute of the image will be computed with motifs of scan pattern and a color motif cooccurence matrix(CMCM) will be obtained

● Euclidean DIstance ● Mahalanobis Distance ● MInkowski Distance ● Histogram Intersection Distance ● Quadratic Form Distance

Techniques used by CBIR ● K-Means K-means clustering algorithm is proposed as it improves the scalability. ● Wavelet Transform Feature vectors of images are be constructed from wavelet transformations, which can also be utilized to distinguish images through measuring distances between feature vectors. ● Support Vector Machine: SVM classifier can be trained using training data of images marked by users . ● Neural Network: A CNN doesn’t need complex work like feature extraction to work. Having a proper labelled data, we can train the system to learn the data features using complex layer structure.

CBIR + TBIR ◦ CBIR can be costly in the fact that it needs a lot of complex computations. ◦ TBIR can be comparatively fast but has low precision. ◦ A hybrid model is currently being implemented. ◦ a text-based image meta-search engine retrieves images from the Web using the text information from the Query. ◦ Techniques like matching term frequency-inverse document frequency (tf-idf) weightings and cosine similarity are used. ◦ use the CBIR approach to re-filter the search results.

● X.Y. Wang,Y.J. Hong and H.Y.Yang,”An effective image retrieval scheme using color, texture and shape features” ● Nidhi Singh ,Kanchan Singh and Ashok Sinha “A Novel Approach for Content Based Image Retrieval” ● Yogita Mistry,and D. T. Ingole “Survey on Content Based Image Retrieval Systems” ● John Canny, “A Computational Approach to Edge Detection” ● Mussarat Yasmin, Muhammad Sharif and Sajjad Mohsin, “Use of Low Level Features for Content Based Image Retrieval: Survey” ● N. Jhanwar, Subhasis Chaudhuri, et.al. Content based image retrieval using motif cooccurrence matrix