Image organization, annotation, Image organization, annotation, and retrieval from a human- - and retrieval from a human centered perspective centered perspective Dr. Oge Marques Associate Professor Computer Science and Engineering Florida Atlantic University Boca Raton, FL (USA) May 2008
Research Team Research Team � Prof. Humberto R. Gamba, Ph.D. � Liam M. Mayron, Ph.D. � Gustavo B. Borba (Ph.D. candidate) (FAU, Boca Raton, FL - USA) (UTFPR, Curitiba, PR - Brazil) � Dipl.-Ing. Dr. Mathias Lux � Vladimir Nedovic (Ph.D. candidate) � Arthur Pitman (UvA, Amsterdam, NL) (Klagenfurt University, Austria) May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 2
Background and Motivation Background and Motivation � Image retrieval is (still) an open research question – Content-based approaches only work well in narrow domains – Keyword-based solutions have limitations, too – Hybrid (semantically capable) solutions are the ‘holy grail’ � The landscape for image annotation has changed – From ‘expensive’ to ‘fun’ � Image organization tools have evolved and become popular – Picasa, iPhoto, etc. help users organize their ‘digital shoeboxes’ May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 3
Our belief Our belief � We believe that image organization, annotation, and retrieval are highly related tasks. – Consequently, there is a market (and research potential) for solutions that integrate those tasks. � Such integration must have the human user at the center of the design process. � The question, of course, is “How to do it in a principled way?” � Our “sources of inspiration”: – Cognitive (Vision) Science – Web developments and best practices – Social (Web 2.0) interactions – Games! May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 4
Human- -Centered Computing (HCC) Centered Computing (HCC) Human � Research area or buzzword? – Recent (May 2007) IEEE Computer special issue – Representative publication: � A. Jaimes, N. Sebe, and ; D. Gatica-Perez (2006). "Human- Centered Computing: A Multimedia Perspective”, Proc. of the 14th ACM international conference on Multimedia : 855- 864, Santa Barbara, CA, USA: ACM Press. – NSF cluster – Several research groups (in the US): � GA Tech, Auburn, Berkeley, Maryland, […] May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 5
Putting ideas into practice… … Putting ideas into practice � Using vision science insights and models to advance image segmentation, (region-based) image retrieval, and grouping of similar images. � Representative publications: – Oge Marques, Liam M Mayron, Gustavo B Borba, and Humberto R Gamba, “An attention-driven model for grouping similar images with image retrieval applications“, EURASIP Journal on Advances in Signal Processing , Special Issue on Image Perception, 2007, Article ID 43450, 17 pages. – Oge Marques, Liam M Mayron, Gustavo B Borba, and Humberto R Gamba, “On the potential of incorporating knowledge of human visual attention into CBIR systems“, Special Session on Perceptual Visual Processing, IEEE International Conference on Multimedia & Expo (ICME) 2006 , July 9-12, 2006, Toronto, Canada. May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 6
An attention- -driven model for grouping similar driven model for grouping similar An attention images for CBIR images for CBIR May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 7
An attention- -driven model for grouping similar driven model for grouping similar An attention images for CBIR images for CBIR May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 8
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 9
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 10
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 11
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 12
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 13
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 14
Image segmentation using computational Image segmentation using computational models of visual attention models of visual attention May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 15
Region- -based image retrieval based image retrieval Region May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 16
Putting ideas into practice… … Putting ideas into practice � Designing an Ajax-based web solution for image organization, annotation, (keyword- and content-based) image retrieval, with collaborative filtering capabilities: PRISM. � Representative publications: – Liam M Mayron, Gustavo B Borba, Vladimir Nedovic, Oge Marques and Humberto R Gamba, “A forward-looking user interface for CBIR and CFIR systems“, IEEE International Symposium on Multimedia (ISM2006), December 11-13, 2006, San Diego, CA. – Liam M Mayron and Oge Marques, “Design of a web-based interface for image retrieval“, International Conference on Web Information Systems and Technologies (WEBIST), March 3-6, 2007, Barcelona, Spain. � Provisional US patent filed through the university’s OTT May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 17
PRISM PRISM May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 18
PRISM PRISM May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 19
PRISM PRISM May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 20
Putting ideas into practice… … Putting ideas into practice � Testing PRISM with a game metaphor. � Representative publication: – Liam M. Mayron and Oge Marques, “Using a game to evaluate image retrieval, organization, and annotation”, First ICIP Workshop on "Multimedia Information Retrieval: New Trends and Challenges" , co-located with IEEE International Conference on Image Processing (ICIP) 2008 , October 12-15, 2008, San Diego, California. ( accepted ) May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 21
PRISM Game PRISM Game May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 22
PRISM Game PRISM Game May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 23
PRISM Game PRISM Game May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 24
Putting ideas into practice… … Putting ideas into practice � Using visual analysis techniques to improve tag suggestions in social media sharing sites. � Representative publication: – Mathias Lux, Oge Marques, and Arthur Pitman, “Using Visual Features to Improve Tag Suggestions in Image Sharing Sites”, International Workshop on Knowledge Acquisition from the Social Web (KASW'08), September 3, 2008, Graz, Austria. ( submitted ) May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 25
Concluding remarks Concluding remarks � If we keep asking (the right) questions, better human-centered solutions (for image organization, annotation, and retrieval) will emerge… � I’m willing to work with some of you on joint collaborative efforts � If you’re interested in any of this, please contact me. May 2008 - Dr. Oge Marques Image organization, annotation, and retrieval from a human-centered perspective 26
Recommend
More recommend
