Merging Results from Multiple Sources in Video Retrieval Wei-Hao - PowerPoint PPT Presentation

Merging Results from Multiple Sources in Video Retrieval Wei-Hao Lin Language Technologies Institute School of Computer Science Carnegie Mellon University whlin@cs.cmu.edu

Multiple-Source Video Retrieval CNN Query: Find video segments containing ABC “Madeleine Albright” C-SPAN 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Motivation We want to exploit source-dependent characteristics to � improve video retrieval performance. However, Source Merging isn’t a trivial problem. � Task Data MAP Random Find video segments ABC 0.053 0.006 that are Weather News CNN 0.780 0.014 ABC + CNN 0.625 0.007 Merging ABC ?? + CNN 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Problem: Merging Results Merging Method 1 Merging Method 2 Merging Method 3 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Issues Degree of Document Overlapping � Degree of Cooperation � Local Relevance Function � Local-to-Global Transformation Function � 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Degree of Document Overlapping When overlapping is � S 1 S 2 S 3 high, we can exploit the cross-resource information . 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Degree of Cooperation When resources are � Corpus Statistics cooperative, we can exploit information Cooperative DID DID DID in addition to Source Pr(R|D) Pr(R|D) Pr(R|D) ranks . Text Text Text Uncooperative DID DID DID Source 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Local Relevance Function Express the extent to which the � document from a source is relevant to the query • Rank • Score, Normalized Score • Probability that the document is relevant (conditioned on the rank) 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Local-to-Global Transformation Function Map the relevance Merged � S 1 S 2 List score of a document in one source to a G(L 1 ( di )) score that reflects its L 1 ( d i ) relevance across sources more L 2 ( d i ) G(L 2 ( di )) accurately 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Related Works Sampled Weighted Linear Data + With Borda Fuse, Combination Regression Learning Bayes Fuse Exp + Nor Local Search CombANZ, Cross- CombSUM, Borda Local Local Without Round Source CORI CombMNZ Fuse Search Search Learning Robin Similarity Rank Score Summary Rank Score Summary Full Text Distributed IR Meta-Search Low High Degree of Document Overlapping 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Multi-Source Video Retrieval Low or not-so-obvious document overlapping � • Approaches developed for meta-search may not be suitable here. The data are fully accessible. � • Score, rank, full “text”, source statistics are available. • No need to sample and build source description Video isn’t just Text � • Judgments from multimodalities must be combined. 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Planned Evaluation TREC 2003 Video track � • ~120 hours of ABC World News Tonight, CNN Headlines News, ~13 hours of C-SPAN • Half of them are already labeled and will be the test bed. Tasks � • Feature Extraction (17 features) Evaluation Metric � • Mean Average Precision (MAP) 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

Planned Experiments Establish random baseline and upper bound � Full “text” vs. multiple-source retrieval + � merging Source merging in individual modality, � including text (closed caption, speech transcript) and image, face Local relevance function � Local-to-global transformation function � 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)

References Distributed Information Retrieval � • sampled data + regression: Luo Si and Jamie Callan. Using sampled data and regression to merge search engine results. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, Toronto, Canada, July 28 - August 01 2003. • Use summary (headline): Xiao Mang Shou and Mark Sanderson. Experiments on data fusion using headline information. In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval, University of Tampere, Finland, August 11 - 15 2002. • Cross-rank similarity: Jamie Callan, Fabio Crestani, Henrik Nottelmann, Pietro Pala, and Xiao Mang Shou. Resource selection and data fusion in multimedia distributed digital libraries. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, Toronto, Canada, July 28 - August 01 2003. • CORI: J. Callan. Advances in Information Retrieval, chapter Distributed Information Retrieval, pages 127--150. Kluwer Academic Publishers, 2000. Meta-Search � • CombMNZ: Joon Ho Lee. Analyses of multiple evidence combination. In Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, Philadelphia, Pennsylvania, United States, July 27 - 31 1997. • Use full text for local search: Steve Lawrence and C. Lee Giles. Context and page analysis for improved web search. Internet Computing, 2(4):38--46, 1998. • use summary: Theodora Tsikrika and Mounia Lalmas. Merging techniques for performing data fusion on the web. In Proceedings of the tenth international conference on Information and knowledge management, Atlanta, Georgia, USA, October 05 - 10 2001. • Model two distributions: R. Manmatha, T. Rath, and F. Feng. Modeling score distributions for combining the outputs of search engines. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans, Louisiana, United States, 2001. • Rank and score: Javed A. Aslam and Mark Montague. Models for metasearch. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, New Orleans, Louisiana, United States, 2001. • Meta-search: Cynthia Dwork, Ravi Kumar, Moni Naor, and D. Sivakumar. Rank aggregation methods for the web. In Proceedings of the 10th International World Wide Web Conference, Hong Kong, May 01 - 05 2001. 2003-09-23 11-743 Advanced Information Retreival Seminar and Lab (Initial Project Presentation)