Condensed Movies: Story Based Retrieval with Contextual Embeddings - PowerPoint PPT Presentation

Aug 26, 2023 •143 likes •282 views

Condensed Movies: Story Based Retrieval with Contextual Embeddings Max Bain, Arsha Nagrani, Andrew Brown, Andrew Zisserman Motivation Long term video understanding Context Billy reveals the truth to Louis about the Duke's bet which

Condensed Movies: Story Based Retrieval with Contextual Embeddings Max Bain, Arsha Nagrani, Andrew Brown, Andrew Zisserman
Motivation • Long term video understanding • Context Billy reveals the truth to Louis about the Duke's bet which changed both their lives.
Motivation • Long term video understanding • Context Billy reveals the truth to Louis about the Duke's bet which changed both their lives.
Motivation • Long term video & narrative understanding • Context ��
Text-Video Retrieval ��
Contextual Text-Video Retrieval
Condensed Movies • 34K captioned clips, from 3.4K movies • 1.3K hours video • Freely available • Pre extracted 6 expert features • 400K facetracks, 8K characters • Ordered 2 minute clips. 1 2 3 … 8 9 10
Semantic Captions
Despite illegal moves from the Cobra Kai that bring him to the ground, Daniel (Ralph Macchio) delivers one final kick and wins the championship.
Overview • Context boosts retrieval • Contextual task utilizing long term narratives
Future work • Improved temporal aggregation for better 2-minute clip embeddings • Other tasks to test long-term video understanding
Thank you • Paper, dataset, features and code can be found at: • https://www.robots.ox.ac.uk/~vgg/research/condensed-movies

Recommend

Story-based Video Retrieval in TV series using Plot Synopses Makarand Tapaswi, Martin Buml,

Story-based Video Retrieval in TV series using Plot Synopses Makarand Tapaswi, Martin Buml, Rainer Stiefelhagen Karlsruhe Institute of Technology, Germany 03 April, ACM ICMR 2014 Computer Vision for Human-Computer Interaction Lab KIT

421 views • 25 slides

Contextual Media Retrieval Using Natural Language Queries IMPRS-CS PhD Application Talk Sreyasi

Contextual Media Retrieval Using Natural Language Queries IMPRS-CS PhD Application Talk Sreyasi Nag Chowdhury Masters Thesis Supervisors Dr. Mario Fritz Dr. Andreas Bulling Adviser M.Sc. Mateusz Malinowski 1 Outline Motivation and

1.22k views • 103 slides

Contextual Inquiry Take Aways Overview of Contextual Design Contextual inquiry

Contextual Inquiry Take Aways Overview of Contextual Design Contextual inquiry Interviewing techniques Contextual Design Contextual design is: An established process for analyzing tasks people do and designing technology to

456 views • 43 slides

God in Movies True Story the question Why Why Does God Allow Some Doors to Close and

God in Movies True Story the question Why Why Does God Allow Some Doors to Close and Others to Open? Rev 3:7 What the Lord opens, no one can close; and what he closes, no one can open Background of Jimmy Adams

510 views • 15 slides

Voice Based Information Retrieval System How far is it from text based retrieval system? PRAJNA

Voice Based Information Retrieval System How far is it from text based retrieval system? PRAJNA BHANDARY CMSC 676 MOTIVATION The ever increasing Internet bandwidth, the ever-decreasing storage costs and the fast development of multimedia

324 views • 15 slides

Content-Based Image Retrieval Queries Commercial Systems Retrieval Features

Content-Based Image Retrieval Queries Commercial Systems Retrieval Features Indexing in the FIDS System Lead-in to Object Recognition 1 Content-based Image Retrieval (CBIR) Searching a large database for images that match a

507 views • 48 slides

Information Retrieval Based on Extraction of Domain Specific Information Retrieval Based on

Information Retrieval Based on Extraction of Domain Specific Information Retrieval Based on Extraction of Domain Specific Significant Keywords and Other Relevant Phrases from a Conceptual Semantic Network Structure Mohammad Moinul Hoque,

572 views • 19 slides

The Sword & Sorcery Movies of the 1980s The Sword & Sorcery Movies of the 1980s - -

So bad, theyre good The Sword & Sorcery Movies of the 1980s The Sword & Sorcery Movies of the 1980s - - Not covering other films like Kull the Conquerer because it was 1990s - Super cut of S & S movies Poster Promises

655 views • 61 slides

Neural Contextual Bandits with UCB-based Exploration Dongruo Zhou 1 Lihong Li 2 Quanquan Gu 1 1

Neural Contextual Bandits with UCB-based Exploration Dongruo Zhou 1 Lihong Li 2 Quanquan Gu 1 1 Department of Computer Science, UCLA 2 Google Research 1 / 49 Outline Background Contextual bandit problem Deep neural networks 2 / 49

739 views • 49 slides

1 QC STORY -32 QC STORY -32 QC STORY -32 QC Story-1 QC Story-1 QC Story-1 Awards and

QC STORY -32 QC STORY -32 QC STORY -32 Shop Floor view QC Story-1 QC Story-1 FACTORY OVERVIEW QC Story-1 On Behalf of Susira Industries Ltd., Chennai. We Welcome the Delegates for the QCI DL SHAH NATIONAL AWARDS ON ECONOMICS OF

976 views • 13 slides

Contextual Analysis SWEN-444 Contextual analysis Systematic analysis of contextual user work

Contextual Analysis SWEN-444 Contextual analysis Systematic analysis of contextual user work activity data Identification, sorting, organization, interpretation, consolidation, and communication For purpose of understanding work context for

449 views • 26 slides

XML Retrieval XML Retrieval XML Retrieval XML Retrieval DB/IR in DB/IR in Theory Theory Web

XML Retrieval XML Retrieval XML Retrieval XML Retrieval DB/IR in DB/IR in Theory Theory Web in Web in Practice Practice Sihem Amer-Yahia Mariano Consens Yahoo! Research University of Toronto In collaboration with: Ricardo Baeza-Yates

1.06k views • 59 slides

Contextual Inquiry Tim Clark (488232) March 21, 2011 Tim Clark (488232) Contextual Inquiry

Contextual Inquiry Tim Clark (488232) March 21, 2011 Tim Clark (488232) Contextual Inquiry What is is? Method of investigating users in the wild. Based on watching users use an existing system. Allows you to learn what tasks are performed

257 views • 7 slides

Music Recommendation Based on Multiple Contextual Similarity

Music Recommendation Based on Multiple Contextual Similarity Information Chih-Ming Chen, Ming-Feng Tsai Department of Computer Science & Program in Digital

326 views • 20 slides

Serving Contextual Communities Serving Contextual Communities The Evangelical Theological

Rochina, Rio Serving Contextual Communities Serving Contextual Communities The Evangelical Theological Seminary of Cairo Serving Contextual Communities Are our seminaries conscious that they are functioning in societies that have crucial

319 views • 30 slides

Contextual Advertising: Contextual Advertising: Semantic Approach Semantic Approach Ekaterina

Contextual Advertising: Contextual Advertising: Semantic Approach Semantic Approach Ekaterina Biehl Ekaterina Biehl Overview: Overview: based on * A. Broder et al.: A Semantic Approach to Contextual Advertising . SIGIR Conference, 2007

602 views • 23 slides

Content-Based Retrieval (CBR) - In Multimedia Systems, a mini-handbook Author: Chao Cai ID:

Content-Based Retrieval (CBR) - In Multimedia Systems, a mini-handbook Author: Chao Cai ID: 0227216 Date: 03/31/2006 Table of Contents 1.0 Background 1.1 History & Overview 1.2 Digital Library 1.3 Metadata 2.0 Content-Based Retrieval of

1.99k views • 29 slides

Content Based Image Retrieval Techniques Ambrose Tuscano (atuscan1@umbc.edu) University of

Content Based Image Retrieval Techniques Ambrose Tuscano (atuscan1@umbc.edu) University of Maryland Baltimore County, CMSC 676 Information Retrieval Introduction Image retrieval systems aim to find similar images to a query image among an

438 views • 21 slides

Brownstone contextual facades adjacent to 340 Decatur St Original Facade Tax

Brownstone contextual facades adjacent to 340 Decatur St Original Facade Tax photo from 1945 Mansarde-roof t op story burned off in fire during the 1970s 340 Decatur

734 views • 28 slides

Passage Based Retrieval (COSC 488) Nazli Goharian nazli@cs.georgetown.edu 1 Passage Based

Passage Based Retrieval (COSC 488) Nazli Goharian nazli@cs.georgetown.edu 1 Passage Based Retrieval Motivation: Only small section of a relevant document contains the information relevant to the query. Example: book chapter.

252 views • 6 slides

LDA-Based Document Models for Ad-hoc Retrieval (Xing Wei and W. Bruce Croft) Haiyun Jin, Lei Ju,

LDA-Based Document Models for Ad-hoc Retrieval (Xing Wei and W. Bruce Croft) Haiyun Jin, Lei Ju, Ruize Qin Background Text documents representation is a critical part of information retrieval (IR). Topic model can capture the

1.34k views • 28 slides

Models for Retrieval Models for Retrieval 1. HMM/N-gram-based 2. Latent Semantic Indexing (LSI)

Models for Retrieval Models for Retrieval 1. HMM/N-gram-based 2. Latent Semantic Indexing (LSI) 3. Probabilistic Latent Semantic Analysis (PLSA) Berlin Chen 2003 References: 1. Berlin Chen et al., An HMM/N-gram-based Linguistic Processing

723 views • 47 slides

Contextual Inquiry SWEN-444 Contextual Inquiry is the process of discovering what users cannot

Contextual Inquiry SWEN-444 Contextual Inquiry is the process of discovering what users cannot tell you. Interview + Observation The Flashlight, an Example of How Understanding Work Practice Paid off CI Preparation Do your homework

347 views • 13 slides

Contextual Inquiry SWEN-445 Contextual Inquiry is the process of discovering what users cannot

Contextual Inquiry SWEN-445 Contextual Inquiry is the process of discovering what users cannot tell you. Interview + Observation The Flashlight, an Example of How Understanding Work Practice Paid off CI Preparation Do your homework

239 views • 13 slides