See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/240053042 Intelligent Information Retrieval and Presentation with Multimedia Databases Article · January 2003 CITATION READS 1 103 6 authors , including: Floris Wiesman Boban Arsenijevic Academisch Medisch Centrum Universiteit van Amsterdam Karl-Franzens-Universität Graz 37 PUBLICATIONS 348 CITATIONS 43 PUBLICATIONS 250 CITATIONS SEE PROFILE SEE PROFILE Nico Roos Maastricht University 89 PUBLICATIONS 734 CITATIONS SEE PROFILE Some of the authors of this publication are also working on these related projects: Game AI View project Argumentation Systems View project All content following this page was uploaded by Boban Arsenijevic on 05 June 2014. The user has requested enhancement of the downloaded file.
Intelligent Information Retrieval and Presentation with Multimedia Databases Floris Wiesman Stefano Bocconi Boban Arsenijevic IKAT, Universiteit Maastricht Centrum voor Wiskunde en ULCL, Leiden University P .O. Box 616, 6200 MD Informatica P .O. Box 9515, 2300 RA Maastricht, The Netherlands P .O. Box 94079, 1090 GB Leiden, The Netherlands Amsterdam, The Netherlands wiesman@cs.unimaas.nl b.arsenijevic@let.leidenuniv.nl stefano.bocconi@cwi.nl ABSTRACT the coarse extreme they show links to documents, ranked with respect to their relevance to the query. In the fine- The paper introduces a knowledge-based multimedia ap- proach to multimedia information retrieval. The approach grained extreme they provide an exact answer to the query (question-answering system). In between are the passage- uses domain knowledge to augment a user’s query, performs automatic ontology mapping to search different multimedia retrieval systems. In this paper we introduce a retrieval mode that is related to passage retrieval and question an- databases, and combines the results in a multimedia pre- sentation. The texts in the presentation are generated from swering but which is especially geared to multimedia infor- mation. the domain knowledge. Thus, the user can view a coherent multimedia presentation that contains the answer to his or In brief, our approach uses domain knowledge to augment her query. The paper describes an architecture for realiz- the query, performs automatic ontology mapping to search ing the approach. The individual parts of the architecture have been implemented, but are not yet integrated in one different multimedia databases, and combines the results in a multimedia presentation. The texts in the presentation are system. generated from the domain knowledge and the various on- tologies. Thus, the user can view a multimedia presentation Categories and Subject Descriptors that contains the answer to his or her query. H.3.3 [ Information Search and Retrieval ]: Selection process; H.3.5 [ Online Information Services ]: Web-based The remainder of this paper is organized as follows. In Sec- services; H.5.1 [ Multimedia Information Systems ]: Mis- tion 2 we elaborate on our approach and present our ar- cellaneous chitecture. Three parts of the architecture are dealt with in separate sections: presentation generation (Section 3), General Terms natural-language generation (Section 4), and ontology map- Design, Algorithms, Theory ping (Section 5). Finally Section 6 provides conclusions and directions for future work. Keywords 2. THE I 2 RP ARCHITECTURE semantic web technologies, multimedia presentations, ontol- ogy mapping, natural language generation Our approach to multimedia information retrieval can best be described on the basis of our architecture, called the I 2 RP architecture. 1 It is depicted in Figure 1. From left to right 1. FROMMULTIMEDIASEARCH TOMUL- the figure shows the course of query processing to the pre- TIMEDIA PRESENTATIONS sentation of the results. Multimedia presentations utilize a combination of several media, which results in shared load of the different per- The user starts with formulating a query. The query pro- ceptional channels [7] and reduction of cognitive memory cessor augments the query with domain knowledge by re- load [6], thus ultimately in conveying effectively informa- trieving also elements that are semantically related to the tion. Text-only information retrieval systems have retrieval query term. The domain knowledge is stored in a semantic modes that show query results in various granularities: in network, which is served by the ontology agent . This agent also has access to various multimedia databases. For each database the agent automatically creates a mapping such that items in the databases are linked to concepts in the semantic network. Thus, retrieving information from the databases becomes an inferencing task. The result is a sub- graph of the semantic network that contains the answer to 1 I 2 RP is the project acronym, which stands for Intelligent Information Retrieval and Presentation in Public Historical DIR 2003, 4th Dutch-Belgian Information Retrieval Workshop. Multimedia Databases . c � 2003 the author/owner
Figure 1: The I 2 RP architecture. the query; we call this subgraph the answer graph . edge source. Users start off specifying via a web interface the topics they are interested in. The system then accesses The next task is to generate a presentation from the answer the knowledge source to retrieve relevant information items graph. This is done by the presentation generator . A pre- (the query processor in Figure 1) and structures them in a sentation may comprise written text, sound, pictures, and presentation (the presentation generator in Figure 1). Fig- movies. The presentation generator decides how to combine ure 2 shows a multimedia player screen with an example the information from the answer graph into one presentation presentation. Each knowledge source available is described that conveys the information to the user as well as possible. by an ontology, here called domain ontology. The ontology Complete texts may be taken from the databases, but the agent guarantees that all information sources use the same texts may also be generated from the answer graph on the domain ontology. fly. This is the task from the natural language generator . It receives a selection of facts from the answer graph and us- Retrieving information based on a domain ontology makes ing knowledge of syntax and semantics it generates natural it possible to retrieve items which might not contain infor- language texts that are incorporated in the presentation. mation about the main topic of the query but are semanti- cally related (sometimes indirectly via multiple steps) to it. Finally, the presentation can be viewed by the user with a For example, a presentation about Rembrandt’s biography multimedia player. Since the presentation does not contain might include a description of his student Jan Lievens even actual multimedia items but only links to them, the player if this information item does not contain any reference to accesses the databases while playing the presentation. Rembrandt, but is annotated with a semantic relation ‘stu- dentOf’. Inferencing on the semantic relations can also help Although our project is limited to museums as test domain, to discover relevant items; for example, if A is spouseOf B the architecture has a wider scope. This explains the generic and B is sonOf C, then A and C are also relatives. character of the natural language generator and the ontology agent. This is not the only way the domain ontology can serve the purpose of information retrieval: if elements are retrieved The different parts of the architecture are developed by dif- because of their semantic relations with the topics of the ferent groups. The query processor and presentation gen- presentation and with each other, these semantic relations erator come from CWI, the natural language generator is a should be preserved when presenting the results to the users, product of Leiden University, and the ontology agent is from translating the semantic relations in spatio-temporal rela- the Universiteit Maastricht. They are further discussed be- tions (related items are presented in spacial or temporal low. proximity). A ranked list will very likely not preserve se- mantic relations among the retrieved items. Again using the example of Rembrandt, in a list a relevant information 3. QUERY PROCESSING AND PRESENTA- item (e.g., text or image) about Rembrandt can be ranked TION GENERATION as first, while a less relevant information item about Rem- brandt’s son can be ranked much lower (or excluded from The I 2 RP system generates multimedia presentations about the list). It would be better to have an ordered presentation a user-specified subject using a semantically annotated knowl-
Recommend
More recommend
