LOD Stories { Learning About Art by Building Multimedia Stories - PowerPoint PPT Presentation

LOD Stories { Learning About Art by Building Multimedia Stories Hao Zhang, Jianliang Chen, Yuting Liu, Dipanwita Maulik, Linda Xu Dr. Craig A. Knoblock, Dr. Pedro Szekely University of Southern California, United States MielVander Sande Ghent University, Belgium

Mr. Bean (2007)

Rowan Atkinson Mr. Bean (2007)

Zazu The Lion King (1994)

Main Main Actor Character Voice

The Law of Nations Carnegie institute of George Technology Washington Emerich de Vattel Pittsburgh Andy Wharhol

LOD Stories

Related Work Everything is Connected http://everthingisconnected.be

Related Work WikiLinks

LOD Stories Relationship Learning User Exploration Using Engagement Multimedia

A Slide Demo…

? Andy Warhol

Node Navigation Bar: Nodes as in the path

Multimedia Collection: images and videos from the web

Multimedia Collection: images and videos from the web

Multimedia Preview: Preview images, videos and voices

Narration Editor: Edit text description of the node

Movie Navigation: Drag image and video here to make the movie

Switch to node: Pittsburgh

Puzzle Pieces Topic Transition Text to Speech Software Google Images and Videos

How it works…

Project Architecture Multimedia Storyline Story Path Explorer Story Editor Story Player Interesting Search Connection s Story Elements Request Generators Path Finder Machine Learning Image Video Profile Date Voice Module Multimedia Data Path Data Triple Store Images & Videos Smithsonian DBpedia from Web Datasets

Path Finder Machine Learning Training Datasets Feature 1 Feature 2 Feature 3 Feature 4 … Interestingness Triple 1 0.95 0.45 0.13 0.87 … 4 … … … … … … … Machine Learning Formula for Interesting Relationships

Path Finder Author Andy Campbell’s Subject Degree Warhol Soup Can Object Degree ? * Andy Subject Warhol Normalized Extension Rarity Feature Value ? * Person (Subj. Deg.) Object Extension Rarity

Path Finder Author Andy Campbell’s Subject Degree Warhol Soup Can Object Degree ? * Soup Subject Can Normalized Extension Rarity Feature Value ? * Work (Obj. Deg.) Object Extension Rarity

Path Finder Author Andy Campbell’s Subject Degree Warhol Soup Can Object Degree Author * Andy Subject Warhol Normalized Extension Rarity Feature Value Author * * (Subj. Ext. Object Rarity) Extension Rarity

Path Finder Author Andy Campbell’s Subject Degree Warhol Soup Can Object Degree Author * Soup Subject Can Normalized Extension Rarity Feature Value Author * * (Obj. Ext. Object Rarity) Extension Rarity

Path Finder Author Andy Campbell’s Subject Degree Warhol Soup Can Object Degree Rank = 3.9595 * Subj. Extension Rarity + 0.8631 * Obj. Extension Rarity Subject - 0.1643 * Subj. Degree + 0.0014 * Obj. Degree Extension Rarity + 1.2353 Object Extension Rarity

Path Finder Author Andy Campbell’s Warhol Soup Can Subj. Ext. Obj. Ext. Subj. Obj. Interestingness Rarity Rarity Degree Degree 1.46E-05 1.60E-04 5.62E-05 4.65E-05 4 Hmmm… Pretty Interesting

Path Finder User Feedback

Discussion

Project Features Path Multimedia Flexible Recommendation Elements Editing Mechanism Mash-up

Future Work Improve Machine Enrich Path Fully Integrate with Learning Model Explorer and Story Smithsonian Editor Museum Datasets Functionalities

Thanks! http://lodstories.isi.edu