Image Registration and Motion Estimation Fabio Viola University of - PowerPoint PPT Presentation

Image Registration and Motion Estimation Fabio Viola University of Cambridge

The Goal • Produce mosaics from: – Sets of uncalibrated still images – Videos

What do we need to mosaic images? • Image mosaicing integrates information from different pictures (measurements) into a single support • In order to merge data from different inputs we need to register the input frames

What is Image Registration?

Image Registration In a Nutshell • “...given two different views of the same scene, for each image point in one view find the image point in the second view which has the same pre ‐ image, i.e. corresponds to the same actual point in the scene.”

Global transformations • We know how to model the image formation process • This fact allows – under precise hypotheses ‐ for the registration task to be performed inferring global geometrical relations between pairs of input frames

Global transformations cont. • Generalization of these properties have been exploited to mosaic still images form tunnels, assuming they came from the same ring • KEY POINT: images are registered locally in pairs, approximating the captured surfaces as planes

Mosaics from videos

Images from Rav ‐ Acha,et al, " Minimal Aspect Distorsion (MAD) Mosaicing of Long Scenes"

Images from Rav ‐ Acha,et al, " Minimal Aspect Distorsion (MAD) Mosaicing of Long Scenes"

The Challenge • Exploit the redundant information available in video to overcome current limitations – Introduce a global optimization over the whole video to increase coherence of registration (?) – Investigate Manifold Projection methods • Deal with low quality of video: – Motion blur due to hand held camera – Bad lighting conditions

Global Transformations Failures Mosaicing of Shafts • When there is a lot of 3D structure, or motion, in the scene simple geometrical global methods fail • Unfortunately this is the case of Shafts Imagery

The Idea • Define the Mosaics as a flat representation of the scene • Define each frame as a map from the mosaic to the image plane • Given the image data learn jointly the representation and the mapping so that the input frames can be regenerated from the mosaic

So, is this Image Registration?

Compute motion • The registration step is performed computing a dense motion field between the input frames • Frames could be easily roughly aligned by hand • The machine will fill the gap and generate the mosaic representation

Work Ongoing • We are investigating the motion model and its learning (tech report available on request) • We are now going to start coding the mosaicing of the provided video sequences • Results and ICCV2009 paper expected by march

Take ‐ home message • The different data sets we have been provided with require different techniques • The key task to perform is registration – When 3D structure is present accurate motion estimation tools are required to provide registration – Motion estimation is NOT a solved problem yet, and further research is needed