TRECVID 2008 CBCD TRECVID 2008. CBCD MCG-ICT-CAS MCG-ICT-CAS Sheng Tang Yongdong Zhang Ke Gao Xiao Wu Sheng Tang, Yongdong Zhang, Ke Gao, Xiao Wu, Xiaoyuan Cao, Huamin Ren,Yufen Wu, Jian Yang Institute of Computing Technology, Chinese Academy of Sciences
N New Challenges Ch ll Various Transformations Insertions of patterns, Picture in picture, Cam-coding, ……. A large amount of A l t f Varied Contents Dataset Videos: TVs, Moives, Sports,… 200hours, 438 videos 438 videos. Query Videos: 2010 clips, 4000 minutes 4000 minutes. 2
O Our Contributions C ib i Placeho Multi-module processing lder text Result fusion Novel feature method for each module ceholder text Time sequency consistence q y For copy location 3
O li Outline Multi-module system Multi module system Novel feature for each module Time sequence consistence Result fusion Result and discussion 4
Multi-module System i S Module 1 Module 3 Global PIP Global quality Picture in picture type 1 decrease such decrease such (The original video is as blur, System adding noise, inserted in front of a …… background video) four modules Module 4 Module 2 Flip p Local Local Partial content alteration Video horizontal mirroring such as occlusion, shift, and crop, …… (including the Picture in Picture (including the Picture in Picture type 2, the original video is the 5 background)
S System Overview O i Feature Copy py R R Query Query Query Query Module Fusion Module Fusion Extraction Detection features E Global Commit Global Commit Global Frame matching S Local Local Local Depend Local Depend p Time matching U Feature set PIP PIP PIP Depend L Video Index Confidence Flip Flip Flip Flip Flip Commit Flip Commit Dataset Dataset calculating l l ti Time Code Ti C d T T Flowchart of ICT MCG CBCD System _ _ y 6
O li Outline Multi-module system Multi module system Novel feature for each module Time sequence consistence Result fusion Result and discussion 7
Feature for Global Module f G Our Contributions for Global Module System y � DC coefficients based Block Gradient Histogram Feature � 【 Advantage 】 fast, low-dimension, robust to global transformations global transformations 8
Block Gradient Histogram Feature Image Block 1- Gradient of each pixel Image Block 1- Gradient of each pixel 1 1 1 1 2 3 2 3 2 2 3 3 Image X Image X 4 4 5 6 5 6 X axis X axis 7 8 9 7 8 9 Block Gradient Histogram Block Gradient Histogram Image Y axis Image Y axis (3, 1, 1, 1.5, 0.5, 3, 1, 0.8) (3, 1, 1, 1.5, 0.5, 3, 1, 0.8) 1- 8dims 2- 8dims 1- 8dims 2- 8dims 1- 8dims 2- 8dims 1- 8dims 2- 8dims ………… ………… ………… ………… Illustration of Block Gradient Histogram for Global Module 9
Block Gradient Histogram Feature � The Global feature is robust to many transformations: � Change of image quality: 1 0.9 0.8 0.7 0.6 系列1 0.5 系列2 0.4 0.3 0.2 0.1 0 1 2 3 4 5 6 7 8 � C � Change of image content: f i 10c vs 10s 0.6 0.5 0.4 系列1 0.3 系列2 0.2 0.1 0 1 2 3 4 5 6 7 8 10
Feature for Local Module f Our Contributions for Local Module System y � KLT based Local Patch Feature with Spatial Information � 【 Advantage 】 robust to partial occlusion, crop, and shift and shift 11
Local Feature with Spatial Local Feature with Spatial Information � To increase discriminability of local features, we present a method to add spatial information: Local Patch - Gradient Histogram Local Patch - Gradient Histogram Block Gradient Histogra Block Gradient Histogra Block Gradient Histogra Block Gradient Histogra Block Gradient Histogra Block Gradient Histogra Block Gradient Histogra Block Gradient Histogra (3, 1, 1, 1.5, 0.5, 3, 1, 0.8 (3, 1, 1, 1.5, 0.5, 3, 1, 0.8 (3, 1, 1, 1.5, 0.5, 3, 1, 0.8 (3, 1, 1, 1.5, 0.5, 3, 1, 0.8 Local Local patch patch t h t h Spatial Neighborhood - block gray rank Spatial Neighborhood - block gray rank a 3 3 3 4 4 4 (3, 4, 1, 2) (3, 4, 1, 2) (3 4 1 2) (3 4 1 2) 1 1 1 2 2 2 Ill Illustration of local feature with spatial information t ti f l l f t ith ti l i f ti 12
Local Feature with Spatial Local Feature with Spatial Information � The introduction of spatial information could effectively increase discriminability of local features, thus improve the matching precision: precision: Comparison of retrieval precision Comparison of matching effect Comparison of matching effect before and after using spatial information b f d f i i l i f i before and after using spatial information 13
Feature for PIP Module f Our Contributions for PIP Module System y � Edge detection based PIP boundary determination � Block Gradient Histogram features extraction for PIP region(the same as global module ) PIP i (th l b l d l ) � 【 Advantage 】 robust to change of scale and 【 Advantage 】 robust to change of scale and position, simple and fast than scale-invariant local feature based method 14
15 PIP Boundary Detection Illustration of PIP boundary location and some instances ti D t d PIP B
Feature for Flip Module f i Our Contributions for Flip Module System y � Vertical Mirror feature � Global features and local features extraction for flip module( the same as extraction for flip module( the same as previous steps ) � 【 Advantage 】 robust to vertical mirror, simple and fast 16
17 Rotation-Invariant Feature Find Mirror Dimension i (3,4,1,5,3,2,3,3) Obtain Mirror Feature i
O li Outline Multi-module system Multi module system Novel feature for each module Time sequence consistence Result fusion Result and discussion 18
Time Sequence Consistency method (2) ng frame lt of matchin Resul o Graph for copy location Graph for copy location tching video j ∑ (1) = max ( ) M weight node l i j X , , , , j Result of mat = l i l i → β ⎧ frame frame of Video , if M > (2) = ⎨ i j x location Video similarity based on matching frame-pairs ⎩ none, else using time sequence consistency method R 19
O li Outline Multi-module system Multi module system Novel feature for each module Time sequence consistence Result fusion Result and discussion 20
Result Fusion Method � We tried number of fusion methods including non-hierarchical method and hierarchical method. � Non-hierarchical method means we use 4 modules to calculate each query separately at the same time, and only the one with high score will be submitted be submitted. Score: Global Query 0.95 video Score: Local 0.87 PIP Score: 0 32 0.32 Flip Score: 0.10 【 character 】 simple but slow ， high recall but low precision, 【 character 】 simple but slow ， high recall but low precision, 21 hard to determine the score threshold
Result Fusion Method � We tried number of fusion methods including non- hierarchical method and hierarchical method. � Hierarchical method submits the result of each module in some sequence. For each query, if any previous module has found its corresponding video, we submit the result, and then turn to process the next query. p q y Query Not Glob video copy Loc PI Fli IP ip bal cal p Is copy N N N N Submit? Submit? Submit? Submit? Submit? Submit? Submit? Submit? Y Y Y Y 【 character 】 fast and high precision, but depend heavily on process sequence 22
O li Outline Multi-module system Multi module system Novel feature for each module Time sequence consistence Result fusion Result and discussion 23
Result and Discussion l d i i � Tested in the CIVR07_CBCD dataset, the hierarchical method performs best for most queries, and processing time is reduced greatly. � The results of our system in TRECVID2008_CBCD in accordance h l f i i d with the phenomena. 24
Result and Discussion l d i i � Tested in the CIVR07_CBCD dataset, the hierarchical method performs best for most queries, and processing time is reduced greatly. � The results of our system in TRECVID2008_CBCD in accordance h l f i i d with the phenomena. 160 ) ing time(s) 140 120 100 80 80 an processi 60 40 20 Mea 0 1 2 3 4 5 6 7 8 9 10 Transformation number 25
26 C Combine based Block Gradient Histogram based Block Gradient Histogram based Block Gradient Histogram based Block Gradient Histogram Feature could avoid the influence E Everything g and can be calculated very fast. of gamma change 、 recoding, Change C Result and Discussion The use of DC coefficients Content 5 C C Change i Content 3 C D Decrease Q Quality 5 D Decrease Q Quality 3 i Change C gamma g d Strong S recoding r In nsertions l of pattern o PIP type1 P Cam Codin C ng
27 C Combine Everything E g Change C Result and Discussion Content 5 C Change C i Content 3 C effectively, such as PIP type1. process could deal with some process could deal with some Decrease D Quality 5 Q The use of multi-module specific transformations D Decrease Q Quality 3 i C Change g gamma d S Strong r recoding In nsertions l of pattern o PIP type1 P C Cam Codin ng
Recommend
More recommend
