Fixing WTFs - Detecting Image Matches caused by Watermarks, - PowerPoint PPT Presentation

Fixing WTFs - Detecting Image Matches caused by Watermarks, Timestamps, and Frames in Internet Photos Tobias Weyand, Chih-Yun Tsai, Bastian Leibe

Fixing WTFs Poster No. 26 Overview • Many Computer Vision Applications use Internet photos, e.g. • Image Retrieval , Image Clustering and Structure from Motion • Internet photos increasingly contain Watermarks, Timestamps, or Frames (WTFs) that harm these applications. • We propose a simple , effective and fast method to detect WTFs during matching. • Code and dataset are publicly available at: tiny.cc/wtf

Fixing WTFs Poster No. 26 WTFs in Image Matching

Fixing WTFs Poster No. 26 WTFs in Image Matching Invalid matches

Fixing WTFs Poster No. 26 WTFs in Image Matching Invalid matches

Fixing WTFs Poster No. 26 WTFs in Image Matching Invalid matches √

Fixing WTFs Poster No. 26 WTFs in Image Matching

Fixing WTFs Poster No. 26 WTFs in Image Matching Invalid matches

Fixing WTFs Poster No. 26 WTFs in Image Matching Invalid matches

Fixing WTFs Poster No. 26 WTFs in Image Matching Invalid matches X

Fixing WTFs Poster No. 26 WTFs in Image Retrieval Query Image

Fixing WTFs Poster No. 26 WTFs in Image Retrieval Query Image

Fixing WTFs Poster No. 26 WTFs in Image Retrieval Query Image

Fixing WTFs Poster No. 26 WTFs in Image Retrieval Query Image Results

Fixing WTFs Poster No. 26 WTFs in Image Retrieval Query Image Results X X X X

Fixing WTFs Poster No. 26 WTFs in Image Clustering

Fixing WTFs Poster No. 26 WTFs in Image Clustering

Fixing WTFs Poster No. 26 WTFs in Image Clustering

Fixing WTFs Poster No. 26 WTFs in Image Clustering

Fixing WTFs Poster No. 26 WTFs in Image Clustering Pseudo-Clusters

Fixing WTFs Poster No. 26 Method

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions .

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map Spatial Histogram

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map Classifier Decision Spatial Histogram “WTF”

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map Classifier Decision Spatial Histogram “WTF”

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map Classifier Decision Spatial Histogram “WTF”

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map Spatial Histogram Classifier Decision “WTF”

Fixing WTFs Poster No. 26 Method Key assumptions: 
 WTFs have similar appearance and occur in certain image positions . Input Image Pair with feature matches Similarity Map Spatial Histogram Classifier Decision “WTF” “non- WTF”

Fixing WTFs Poster No. 26 Dataset • 36,240 image pairs from Flickr and Panoramio • 10% WTFs, 90% non-WTFs • Publicly available at: tiny.cc/wtf WTFs Non-WTFs … …

Fixing WTFs Poster No. 26 Results 3% False-positive rate @ 99% True positive rate 1 0.8 True Positive Rate 0.6 0.4 0.2 Our Method (0.03 f99, 0.998 AUC) GPS (0.96 f99, 0.499 AUC) GPS+Heuristic (0.96 f99, 0.865 AUC) 0 0 0.2 0.4 0.6 0.8 1 False Positive Rate

Fixing WTFs Poster No. 26 Clustering results Clusters with multiple objects were split. X X X X X X X X X X

Fixing WTFs Poster No. 26 Clustering results Pseudo-clusters were removed. X X X

Fixing WTFs Poster No. 26 Clustering results Polluted clusters were cleaned. X X X X X

Fixing WTFs Poster No. 26 Conclusion • WTF matches harm many vision applications. • We propose a simple , fast and effective detector for them. • Our code is open source and easy to integrate : tiny.cc/wtf

Come visit us at poster 26! Get the code and dataset at tiny.cc/wtf