Computer-Aided Semantic Annotation of Multimedia Project Presentation June 2008
Presentation Outline 1. Introduction 7. Partner Roles 2. Consortium 8. Research Areas 3. Motivation 9. Project Expected Results 4. Project Main Target 10. Potential Impact 5. Project Objectives 11. Contact Information 6. Approach CASAM Project Presentation June 2008 2
Introduction � Project Acronym / Number: CASAM / FP7 – 217061 � Project Title: Computer-Aided Semantic Annotation of Multimedia � Project Website: www.casam-project.eu � Project Logo: � Total Cost / EC Funding: 4.24 m€ / 3.03 m€ � Duration: 1 April 2008 – 31 March 2011 CASAM Project Presentation June 2008 3
Consortium 1 INTRASOFT International SA (INTRASOFT) – Project Coordinator 2 National Centre of Scientific Research “Demokritos” (NCSR) 3 University of Birmingham (UNI-BHAM) 4 Technische Universität Hamburg (TUHH) 5 Athens Technology Center SA (ATC) 6 Deutsche Welle (DW) 7 Agência de Notícias de Portugal (LUSA) 8 European Journalism Centre (EJC) CASAM Project Presentation June 2008 4
Motivation � Pressing need for annotated Multimedia Content multimedia content Annotation � Immature machine-only (fully automated) annotation � Cost-prohibitive manual Machine Manual (human) annotation Annotation Annotation (fully automated) � Computer-Aided Semantic Annotation of Multimedia CASAM � User effort optimization through human-machine synergy � Human and machine Annotation of multimedia by employing human-machine synergy that optimizes intelligence combination user effort. CASAM Project Presentation June 2008 5
Project Main Target To design, implement and validate an Annotation Toolkit that will augment machine knowledge with human input , towards the target of minimizing user effort and bridging the gap between machine-derived and human annotation 3-phases Platform Validation in terms of easiness and quality of annotation: � Initial System Architecture � First Prototype � Second – Final Prototype CASAM Project Presentation June 2008 6
Project Objectives � To provide the technology for Multimedia Analysis � Knowledge-driven analysis � Reassessment of context and concepts based on user feedback and new knowledge � To develop novel methods of Reasoning for Multimedia Interpretation � Built around information exchange between multimedia analysis and human � To design unique intelligent Human-Computer Interaction methods � Maximization of the expected information gain from the user’s input � Cooperative environment to guide the knowledge aggregation process CASAM Project Presentation June 2008 7
Approach (I) � Introduction of the effort-optimized knowledge aggregation concept � Annotation toolkit featuring a human-machine interaction loop for fast convergence of human and machine knowledge � Annotation task into 3 main operations: � Reasoning for Multimedia Interpretation ( RMI ) � Knowledge-Driven Multimedia Analysis ( KDMA ) � Human-Computer Interaction ( HCI ) � Demonstration Domain: News Production in News Agencies and Broadcasters � Methods also applicable to other domains CASAM Project Presentation June 2008 8
Approach (II) CASAM Project Presentation June 2008 9
Approach (III) CASAM information flow (interaction loop) 1. KDMA analyzes the multimedia content (video, image, natural language) and extracts the low level information . 2. User enters a small number of keywords that describe the high-level concepts present in the multimedia content. 3. RMI augments the KDMA-derived and user-provided information instantiating an ontology , infers new concept instances and reassesses the context and previous input from KDMA. Results are fed back to the KDMA for multimedia analysis driven by the renewed information until nothing more can be inferred by RMI or recognized by KDMA. 4. RMI reasons about what is needed in order to add missing instances or resolve any ambiguities. If the annotation target has been achieved the loop exits, otherwise the information requirements are fed to HCI. 5. HCI transforms the information requirements to input requests towards the user, optimizing the user interaction . The user interface also provides the user with the opportunity to alter the knowledge acquisition path devised by the system. 6. HCI input is passed to the RMI and the loop continues from step 3 . CASAM Project Presentation June 2008 10
Partner Roles Dissemination & Exploitation (ATC) Platform Project Management (INTRASOFT) EJC Evaluation Platform TUHH Integration RMI INTRASOFT End Users UNI-BHAM NCSR DW, KDMA HCI LUSA User Requirements & ATC Knowledge Representation CASAM Project Presentation June 2008 11
Research Areas (I) Knowledge-Driven Multimedia Analysis � Text Processing � Image Processing � Video Processing and Video-OCR � Audio Processing � Ontology-based Information Extraction CASAM Project Presentation June 2008 12
Research Areas (II) Knowledge Representation and Reasoning for Multimedia Interpretation � Probabilistic Reasoning � Machine Learning � Multimedia Interpretation � Generation of Requests for Information CASAM Project Presentation June 2008 13
Research Areas (III) Interactive Interface Design, Personalization and Optimization � Effective User Interfaces for Multimedia Tagging � Adaptive Interfaces � Domain Models � User Models CASAM Project Presentation June 2008 14
Project Expected Results � Unified representation of related domain knowledge � Methodology for knowledge-driven multimedia analysis � Methodology for reasoning for multimedia interpretation � Methodology for intelligent human-computer interface � The final toolkit for computer-aided semantic annotation of multimedia content CASAM Project Presentation June 2008 15
Potential Impact (I) Commercial Benefits � Creators to be able to design more participative and communicative forms of content � Publishers in creative industries, enterprises and professional sectors to increase their productivity with innovative content of greater complexity and ease of repurposing � Organizations to be able to automate the collection and distribution of digital content and machine traceable knowledge and share them with partner organizations CASAM Project Presentation June 2008 16
Potential Impact (II) Scientific Impact Deployment of: � Self-adaptive, self-aware, non-isolated and knowledge- guided Multimedia Analysis methods � Intelligent Reasoning methods, guided by multiple targets � Human-Computer Interaction methods, maintaining the user model and customize the queries CASAM Project Presentation June 2008 17
Contact Information � Coordinator contact details: Dr. Antonis Ramfos INTRASOFT International S.A. E-Mail: Antonis.Ramfos@intrasoft-intl.com Tel: +30 210 68 76 482 Fax: +30 210 68 59 166 � Dissemination responsible: Dr. George Kountourakis Athens Technology Center S.A. E-Mail: g.kountourakis@atc.gr Tel: +30 210 68 74 300 Fax: +30 210 68 55 564 � Project Website: www.casam-project.eu CASAM Project Presentation June 2008 18
CASAM Project Presentation June 2008 19
Recommend
More recommend
