Tour the World: building a webscale landmark recognition engine 1 Y a n - T a o Z h e n g , M i n g Z h a o , Y a n g S o n g , H a r t w i g A d a m , U l r i c h B u d d e m e i e r , A l e s s a n d r o B i s s a c c o , F e r n a n d o B r u c h e r , T a t - S e n g C h u a , a n d H a r t m u t N e v e n C V P R 2 0 0 9 P r e s e n t e r : C a n s ı n Y ı l d ı z 03.12.2009 Cansın Yıldız
Introduction 2 03.12.2009 Cansın Yıldız
Problems 3 Discovering Landmarks in the World Mining True Landmark Images Efficiency 03.12.2009 Cansın Yıldız
Problems 4 Discovering Landmarks in the World Two complementary sources: GEO-tagged photos from picasa.google.com Travel guide articles from wikitravel.com Mining True Landmark Images Efficiency 03.12.2009 Cansın Yıldız
Problems 5 Discovering Landmarks in the World Mining True Landmark Images Visual clustering on the noisy image set Further cleaning of clusters Efficiency 03.12.2009 Cansın Yıldız
Problems 6 Discovering Landmarks in the World Mining True Landmark Images Efficiency Parallel computing of landmark models Efficient clustering algorithm Efficient image matching 03.12.2009 Cansın Yıldız
Overview 7 03.12.2009 Cansın Yıldız
GPS Tagged Photos 8 1. 1. Learning landmarks from GPS-tagged photos 03.12.2009 Cansın Yıldız
GPS Tagged Photos 9 1.Geo-cluster Create clusters ( I 1 ) based on GPS coordinates Delete clusters with not enough unique authors ~140k geo-clusters, ~14k visual clusters, 2240 landmarks 03.12.2009 Cansın Yıldız
Travel Guide Articles 10 2. 2. Learning landmarks from travel guide articles 03.12.2009 Cansın Yıldız
Travel Guide Articles 11 1.Download wikitravel articles of every city on earth 2.Mine landmark names from articles if, Text is within Section “See” or “To See” Text is within a bullet list. Text is written in bold. 3.Retrieve landmark images ( I 2 ) from google image search 7315 landmark candidates, 3246 landmarks 03.12.2009 Cansın Yıldız
Noisy Landmark Image Set 12 Noisy Image Set I 2 I 1 geo geo landmark landmark … … cluster 1 cluster m images 1 images n … … … visual clustering … Visual Model visual visual visual visual … … cluster 1 cluster k cluster 1 cluster z 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 13 3. 3. Visual Clustering and Cleaning 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 14 Perform visual clustering for each set in set I= I 1 + I 2 1. Object matching based on local features 2.Constructing match region graph 3.Graph clustering on match regions 4.Cleaning visual model 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 15 1.Object matching based on local features Use LOG filters to detect interest points Use SIFT for local descriptors Get match score and match region 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 16 2.Constructing match region graph 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 17 3.Graph clustering on match regions Lack of priori knowledge, can’t use k-means Use hierarchical agglomerative clustering instead. 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 18 4.Cleaning visual model Clean clusters having map images using photographic vs. non- photographic image classifier. Clean clusters having not enough number of authors. Clean clusters having images dominated by people using multi-view face detector . 03.12.2009 Cansın Yıldız
Visual Clustering and Cleaning 19 Sample Visual Cluster 03.12.2009 Cansın Yıldız
Results - Distribution 20 Distribution of Recognized Landmarks 03.12.2009 Cansın Yıldız
Results – True Positives 21 Examples of True Positives 03.12.2009 Cansın Yıldız
Results – False Positives 22 Landmarks can be locally visually similar Regions in landmark model can be non-representative Negative images and landmark model images can be similar 03.12.2009 Cansın Yıldız
Related Work 23 I know what you did last summer: object-level auto- annotation of holiday snaps, T. Quack et al., ICCV 2009 03.12.2009 Cansın Yıldız
Thank You 24 Q U E S T I O N S ? 03.12.2009 Cansın Yıldız
Recommend
More recommend
