CSM-EE727 6 Seconds of Sound and Vision: Creativity in Micro-Videos Authors Miriam Redi, Neil O’Hare, Rossano Schifanella, Michele Trevisiol, Alejandro Jaimes Published CVPR14 Lausanne - May 11th 2020 * https://www.thedrum.com/news/2016/10/28/where-twitter-went-wrong-with-vine
Introduction 2 Study the notion of “creativity”. ● Use short/micro-videos (eg: Vine, Instagram, Facebook). ● Vine emergence (google trends) ● ○ 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS 1st peak: July 2013. ○ CVPR14 submission: November 2013.
Introduction 3 Research questions: ● ○ Can we understand/generate features that are linked to a creative video ? 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS ○ Can we automatically detect it ? Contributions: ● ○ Microvideo labelled dataset. ○ New set of features (audio-visual) to describe novelty and aesthetic value of videos. ○ Correlation between generated features and creativity.
Definition 4 What is creativity ? ● “ All who study creativity agree that for something to be creative, it is not enough for it to be novel : it must have value , or be appropriate to the cognitive demands of the 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS situation. ” In this paper: ●
Dataset 5 Christian Abbet 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS Only a small fraction of video are creative (1.9%). ● 4’000 videos ● ○ a) 1’000: contains #vineart, #artwork, #vineartist, #vineartgallery ○ b) 200: mentioned in online article about vine creativity ○ c) 2’300: video authored by creators identified in previous criteria. ○ d) 500: random video from streamline
Dataset 6 Christian Abbet Labelled with Crowdflower (now Figure Eight). ● Quality checked with “G old Standard” . ● 5 labels (yes, no, idk) per video. ● 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS
6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS Features Modelling Christian Abbet 7
Experimental Results - 8 Christian Abbet Features Analysis on set D-100 (clean version) ● Use 7 sets as described in feature ● modelling 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS Use Multiple Correlation Coefficient ● (MPC) that link correlation between multidimensional variable and unidimensional one. Both aesthetic ( ) and novelty ( ) are ● important. Emotional less correlated than ● sensory. Audio not so important. ●
Experimental Results - 9 Christian Abbet Features Correlation (pearson) between individual features and labels. Homogeneity ( ) ● Video warm and bright colors ( ) ● Uniqueness ( ) ● 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS No presence of people, skin color ( ) ● Pleasant emotions ( ) ● No loop and shake ( ) ●
Experimental Results - 10 Christian Abbet Classification 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS Classification on 3 sets (D-60, D-80, D-100). ● ⅔ Training, ⅓ Testing and balanced sets ● Use of SVM (RBF kernel) on all (7) feature sets. ● Data augmentation with feature that rely on one frame. ● Use 1000 unlabeled video to compute KMeans clustering. ●
6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS Experimental Results Christian Abbet 11
Conclusion 12 Christian Abbet Defining creative videos as videos that are novel and have aesthetic ● value. If sampling is random, only 1.9% f video are considered as creative ● Dataset of > 3800 micro-videos. 6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS ● New features (aesthetic and visual) proposed. ● Positive correlation for homogeneity, warm colors, pleasant ● emotions. Negative correlation for presence of people, loop and shaky video. ● 80% accuracy on test D-100 set. ● Future work: ● ○ Feature to model intellectual aspect. ○ Include meta-data (non audiovisual), such as tags, tweet, user profile. ○ Include other video platforms.
6 SECONDS OF SOUND AND VISION: CREATIVITY IN MICRO-VIDEOS Thank you :) 13 Christian Abbet
Recommend
More recommend