Method Results Conclusions Content-based Image Retrieval by Indexing Random Subwindows with Randomized Trees Rapha¨ el Mar´ ee, Pierre Geurts, Louis Wehenkel GIGA Bioinformatics Platform Dept. EE & CS (Montefiore Institute) University of Li` ege Belgium ACCV, 22 th November 2007, Tokyo Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 1
Method Results Conclusions Content-Based Image Retrieval (CBIR) Goal Given a reference database of unlabeled images, retrieve images similar to a new query image based only on visual content. Challenges To be robust to uncontrolled conditions To be fast (efficient indexing structures) and accurate (rich image descriptions) To avoid tedious manual adaptation specific to a task Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 2
Method Results Conclusions Content-Based Image Retrieval (CBIR) Goal Given a reference database of unlabeled images, retrieve images similar to a new query image based only on visual content. Challenges To be robust to uncontrolled conditions To be fast (efficient indexing structures) and accurate (rich image descriptions) To avoid tedious manual adaptation specific to a task Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 3
Method Results Conclusions Starting point: our method at CVPR05 Image classification with labeled training images and single class prediction Fast method Random subwindow extraction Extremely randomized decision trees [Geurts et al. 2006] Good accuracy results on various tasks Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 4
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees This work: extension for CBIR Overview Detector : random subwindows Descriptor : subwindow raw pixel values Indexing subwindows : totally randomized trees Image similarity measure : derived from similarity measure between subwindows defined by trees Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 5
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Extraction of Random Subwindows Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 6
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Indexing subwindows with one Totally Randomized Tree Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 7
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Indexing subwindows with an Ensemble of T Trees Parameters T : the number of totally randomized trees n min : the minimum node size, stop-spliting of a node if # node < n min Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 8
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Similarity between two subwindows (one tree) A tree T defines a similarity between two subwindows s and s ′ : � 1 if s and s ′ reach the same leaf L containing N L subwindows, k T ( s , s ′ ) = N L 0 otherwise Two subwindows are very similar if they fall in a same leaf that has a very small subset of training subwindows Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 9
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Similarity between two subwindows (ensemble of T trees) The similarity induced by an ensemble of T trees is defined by: T k ens ( s , s ′ ) = 1 � k T t ( s , s ′ ) (1) T t =1 Two subwindows are similar if they are considered similar by a large proportion of the trees Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 10
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Similarity between two images We derive a similarity between two images I and I ′ by: 1 � k ( I , I ′ ) = k ens ( s , s ′ ) (2) | S ( I ) || S ( I ′ ) | s ∈ S ( I ) , s ′ ∈ S ( I ′ ) The similarity between two images is thus the average similarity between all pairs of their subwindows (2) is estimated by extracting at random from each image an a priori fixed number of subwindows Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 11
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Similarities between I Q and all reference images... ... are obtained by propagating subwindows from I Q , and by incrementing, for each subwindow s of I Q , each tree T , and each reference image (I R ), the similarity k ( I Q , I R ) by the proportion of subwindows of I R in the leaf reached by s in the tree T , and by normalizing the resulting score. Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 12
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Propagation of one subwindow into trees Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 13
Method Random Subwindows Results Totally Randomized Trees Conclusions Similarity measure defined by trees Extensions Model recycling : Given a large set of unlabeled images we can build an ensemble of trees on these images, and then use this model to compare new images from another set. Incremental mode : It is possible to incorporate the subwindows of a new image into an existing indexing structure by propagating and recording their leaf counts. If a leaf happens to contain more than n min subwindows, split the node. Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 14
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters ZuBuD (1/3) : images of 201 buildings Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 15
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters ZuBuD (2/3) : results Protocol 1005 unlabeled reference images (640 × 480) 115 labeled test images (320 × 240) Recognition rate of the first ranked image Results Dataset ls/ts us OM05 OM02 ZuBuD 1005/115 96.52 % 93 % to 98.2 % 100 % (with 10 trees, 1000 subwindows per image, nmin = 2 ie. fully developed trees) Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 16
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters ZuBuD (3/3) : query − → top 10 retrieved images − → − → Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 17
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters IRMA (1/3) : X-Ray images (from http://irma-project.org/ ) Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 18
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters IRMA (2/3) : Results Protocol 9000 unlabeled reference images (approx. 512 × 512) 1000 labeled test images (57 classes) Recognition rate of the first ranked image Results Dataset ls/ts us na¨ ıve NN KDGN07 IRMA 9000/1000 85.4 % 29.7 % 63.2 % 87.4 % (with 10 trees, 1000 subwindows per image, nmin = 2 ie. fully developed trees) Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 19
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters IRMA (3/3) : query − → top 5 retrieved images − → − → − → Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 20
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters UkBench (1/2) : images of 2550 “objects” Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 21
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters UkBench (2/2) : results Protocol 10200 unlabeled reference images (640 × 480) Same images for test ( labeled) Recognition rate of the top-4 ranked images (Number of correct images in first 4 retrieved images / 40800) ∗ 100% Results Dataset ls=ts us NS06 PCISZ07 UkBench 10200 75.25 % 76.75 % to 82.35 % 86.25 % (with 10 trees, 1000 subwindows per image, nmin = 4) Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 22
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters META (1/2) : images from various sources Sources: LabelMe Set1-16, Caltech-256, Aardvark to Zorro, CEA CLIC, Pascal Visual Object Challenge 2007, Natural Scenes A. Oliva, Flowers, WANG, Xerox6, Butterflies, Birds. Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 23
Method ZuBuD, IRMA, UKBench Results META: model recycling Conclusions Influence of parameters META (2/2) : results Protocol 205763 unlabeled reference images 10200 UkBench labeled test images Recognition rate of the top-4 ranked images (Number of correct images in first 4 retrieved images / 40800) ∗ 100% Results Dataset ls/ts us NS06 META/UkBench 205763/10200 66.74 % 54 % to 79 % (with 10 trees, 50 subwindows per META image, 1000 subwindows per UkBench image, nmin = 2 ie. fully developed trees) Mar´ ee et al. Indexing Random Subwindows with Randomized Trees 24
Recommend
More recommend
