Learning Better Object Models using Video Data Patrick Li, Inmar - PowerPoint PPT Presentation

Learning Better Object Models using Video Data Patrick Li, Inmar Givoni, Brendan Frey

Motivation Training on a collection of static monocular images is unnatural. Labelled Training Images are hard to get. And the lack of is becoming a problem. Tere is a wealth of video data available.

First Attempt: Learning Bags of Features Models for Image Classification Goal: Represent Objects as Bags of SIFT Features Use unsupervised learning to learn models of objects Use learned models for image classification

Image Classification INPUT: OUTPUT: “Cow” TRAINING: ... “Boat” “Car” “Sofa”

Overview of the Technique Unsupervised Training from Video ... PART 1 PART 2 PART 3 PART 60 Supervised Training on Labelled Images PART 2 PART 3 “Cow” Testing PART 1 “Sofa” PART 8

Bags of Features Models PART 1 PART 2 ... PART 60

Latent Dirichlet Allocation for Topic Modelling CAPITALISM T D S H R E A O M N U S O R A T E C C E C T C R KICK R O I S A O S HIT R E C N D A E BASEBALL L S Y MONEY SPORT POLITICS BANKING FROG ANIMALS G CAT O CAT D T R A N D S A E C M T M I S I O O L A T I P N G A C C S O R L SOCCER L A A D R B E D E C A E S L A FROG B Y 20% ANIMALS 40% POLITICS 39% BANKING Single Document 1% SPORTS

Latent Dirichlet Allocation for Topic Modelling ? ? ? ? 1 2 3 ... 60 Corpus of Documents

Latent Dirichlet Allocation for Topic Modelling Transactions ? ? ? Money 1 2 3 ... 60 Corpus of Documents

Latent Dirichlet Allocation for Object Modelling COW CAR BOAT SOFT DRINKS 90% SOFT DRINKS 10% CORPORATE LOGOS Single Image

Latent Dirichlet Allocation for Object Modelling ? ? ? ? 1 2 3 ... 60 Image Collection

Flow-LDA for Motion Modelling COW CAR BOAT VILLAIN Pair of Consecutive Frame Pairs 50% SWORD 50% VILLAIN

Flow-LDA for Motion Modelling ? ? ? ? 1 2 3 ... 60 Frame Pair Collection

Flow-LDA for Motion Modelling

Image Recognition Unsupervised Training from Video using FLDA PART 1 PART 2 ... PART 60 Training And Testing Images 0.8 Part 1 0.2 Part 2 0.7 Part 1 0.2 Part 3 0.1 Part 4 0.6 Part 2 0.2 Part 13 0.2 Part 24

Initial Results Naive Guesser: 8.6% Error SVM trained on SIFT histograms directly: 8.6% Error SVM trained using LDA model (no motion): 5.6% Error SVM trained using FLDA model (motion): 3.7% Error

... to continue Experiment on Real Dataset Go beyond Bags of Features models -Hierarchical Models -Account for Spatial Relations -Account for temporal relations between more than 2 frames

Tank you!