Advene: active reading through hypervideo Olivier Aubert -Yannick - PowerPoint PPT Presentation

ACM Hypertext 2005 – Sep. 8, 2005 Advene: active reading through hypervideo Olivier Aubert -Yannick Prié LIRIS - Université Claude Bernard Lyon 1 LIRIS – Lyon Research Center for Images and Intelligent Information Systems UMR 5205 – CNRS/INSA-Lyon/Université Lyon 1/Université Lyon 2/Centrale Lyon Advene: active reading through hypervideo – ACM HT 05 1 http://liris.cnrs.fr/

Outline ● Audiovisual Information Systems for scholars ● The hypervideo model ● The Advene project – The Advene model – The Advene prototype – Views in Advene – Active Reading ● Ongoing work – Conclusion Advene: active reading through hypervideo – ACM HT 05 2

Audiovisual Information Systems for Scholars ● Various uses of AV documents – personal notes / work documents, pedagogical tools, testimony of the past, work of art, communication acts, ... ● Basic interaction – text transcriptions and notes – simple video navigation ● Tools should – accommodate the variety of needs in metadata structure and visualisation modalities – allow to share and adapt both structured metadata and visualisations Advene: active reading through hypervideo – ACM HT 05 3

A hypervideo model to... ● Analyse existing innovative uses of audiovisual document ● Provide guidelines to build new hypermedia systems Advene: active reading through hypervideo – ACM HT 05 4

The hypervideo model Audiovisual Document AVD t Advene: active reading through hypervideo – ACM HT 05 5

The hypervideo model Audiovisual Document AVD t f1 f3 f2 Fragments Annotation a3 Annotation a1 Annotations Annotation a2 Structure Relation structure S Annotation Structure AS Advene: active reading through hypervideo – ACM HT 05 6

The hypervideo model Audiovisual Document AVD t f1 f3 Views f2 Fragments View Annotation a3 definitions Annotation a1 Annotations + tools Annotation a2 Structure Relation structure S Annotation Structure AS Advene: active reading through hypervideo – ACM HT 05 7

The hypervideo model Audiovisual Document AVD t f1 f3 Views f2 Fragments Non- View hypervideo Annotation a3 definitions Annotation a1 Annotations + tools Annotation a2 Hypervideo Structure Relation structure S Annotation Structure AS Hypervideo : view that 1/ uses information from both the AV document and the annotation structure and 2/ gives access to the AV document temporality Advene: active reading through hypervideo – ACM HT 05 8

A DVD seen as a hypervideo Movie t f2 f1 f3 Views f4 Fragments DVD Non- f5 f6 hypervideo player Subtitle 3 Subtitle 1 Subtitle 2 Hypervideo Annotations Alt. soundtrack Chapter 2 Chapter 1 Annotation Structure Non-hypervideo views: plain movie, printed chapter list Hypervideo views: subtitled movie, direct access to chapters Advene: active reading through hypervideo – ACM HT 05 9

The Advene project  « advene: to be added to something or become a part of it, though not essential » (Webster 1913) ● Annotate Digital Videos/DVds, Exchange on the NEt ● Data model + prototype – Open-source, cross-platform – Reuse off-the-shelf components (VLC, ZPT, web browser) – Rapid development (python) – Testbed for experiments ● Research platform – new audiovisual interaction modalities – user-lead innovation Advene: active reading through hypervideo – ACM HT 05 10

Principle AV Document Package T1 T2 Type Contenu 0 10 Plan KF 10 25 Plan 5 31 Obj Voiture 10 44 Lieu Route .... ... ..... ............ .... ... ..... ............ .... ... ..... ............ Package creation (definition of annotations and views) ● Annotate videos Advene: active reading through hypervideo – ACM HT 05 11

Principle Package sharing : AV Document - copy through webserver, e-mail, etc. Package - reference through imports T1 T2 Type Contenu 0 10 Plan KF 10 25 Plan 5 31 Obj Voiture 10 44 Lieu Route .... ... ..... ............ .... ... ..... ............ .... ... ..... ............ Package creation (definition of annotations and views) ● Annotate videos ● Share metadata and analyses Advene: active reading through hypervideo – ACM HT 05 12

Principle Package sharing : AV Document - copy through webserver, e-mail, etc. Package - reference through imports T1 T2 Type Contenu 0 10 Plan KF 10 25 Plan 5 31 Obj Voiture 10 44 Lieu Route .... ... ..... ............ .... ... ..... ............ .... ... ..... ............ Package creation poursuivi (definition of annotations and views) Package ● Annotate videos Enriched video views ● Share metadata and Ce film, réalisé en 1972, ne plut pas aux analyses distributeurs lors de sa présentation au AV marché du film à Cannes. Alors que sévissait "L'exorciste" sur les écrans, le producteur demanda au réalisateur (grand nom du fantastique italien) de tourner de nouvelles scènes pour les insérer dans le film et ainsi de Document l'adapter à ce "courant exorciste". Le résultat ● Visualise the metadata in gagne en atouts commerciaux (scènes-chocs, vomi...) ce qu'il perd en poésie! Ce diable amateur de sucettes ne méritait pas ça! Ce film, réalisé en 1972, ne plut pas aux distributeurs lors de sa présentation au marché du film à Cannes. Alors que sévissait "L'exorciste" sur les écrans, user-specifiable ways Ad-hoc views Hypertext views Package utilisation (visualisation, enrichment) Advene: active reading through hypervideo – ACM HT 05 13

The Advene data model Example : Annotations Schema: Summary Schemas Relations Annotation-type: PartOfSpeech Content-type: text/plain Annotation-type: People View Queries Content-type: text/x-vcard definitions Relation-type: NextPart Content-type: None Resources Package Introduction Summary Chairman Element addressing with TALES (ZPT) : package/annotationTypes/PartOfSpeech/annotations/first annotation/content/data annotation/fragment/duration Advene: active reading through hypervideo – ACM HT 05 14

Static (hypertext) views <h2>The different parts of the speech:</h2> <ul> <li tal:repeat="a here/annotationTypes/PartOfSpeech/annotations"><a tal:attributes="href a/player_url"><img alt="" tal:attributes="src a/snapshot_url" /><span tal:content="a/content/data">Name</span></a> (<a tal:attributes="href string:${a/absolute_url}/ view/S_part_details">Details</a>) </li> </ul> Template-based X(HT)ML generation (ZPT) Links to the video and views (ad-hoc, static) Images extracted on the fly Advene: active reading through hypervideo – ACM HT 05 15

Dynamic (enriched video) views ● Event-Condition-Action paradigm ● View = set of rules ● Example: if the event Beginning of annotation occurs (event) and the annotation type is PartOfSpeech (condition), then display the part title as a caption and a navigation popup to go to the next and previous part (action). Events Conditions Actions - Annotation start/end - Allen conditions - Basic player control - Player start/pause - Order, inclusion - Advanced player control - ... - ... - GUI actions (popups, ...) Advene: active reading through hypervideo – ACM HT 05 16

Dynamic view rendering Advene: active reading through hypervideo – ACM HT 05 17

Active Reading with Advene ● Note taking – On-the-fly note taking (timestamped) – Annotation assistants (automatic/semi-automatic generation) ● Annotation visualisation – Ad-hoc views, user-specified static/dynamic views – Document production ● Annotation processing – Schema evolution – Queries ● Collaboration – Package sharing (by copy or reference) Advene: active reading through hypervideo – ACM HT 05 18

Conclusion ● Main Advene features : – flexible metadata structure – user-specifiable visualisations – metadata and visualisation specifications exchange ● Analysing and building systems with the hypervideo model Advene: active reading through hypervideo – ACM HT 05 19

Ongoing work ● Prototype : – Usability issues (helpers, WYSIWYG editor, ...) – Common, reusable views and schemas definition (emergent discovery) – Package repositories creation – Schema evolution ● Collaborations (Incorporactions, CRDP) ● Theoretical analyses (document temporalities, hypervideo perception, nature of document) Advene: active reading through hypervideo – ACM HT 05 20

The end... Thank you for your attention The prototype is available from http://liris.cnrs.fr/advene/ Advene: active reading through hypervideo – ACM HT 05 21

Views and hypervideos ● Annotated AV document: – movie + shot/sequence decomposition ● Non-hypervideo views AVD – the movie alone – a plain table of contents AS ● Hypervideo views – Movie with ● sequence title inserted on the video ● links to the next/previous shot or sequence – Hypertext table of contents linked with the movie ● Hypervideo concept uses – Analysis of existing hypervideos – Guidelines to build new systems Advene: active reading through hypervideo – ACM HT 05 22