Mosaics of Scenes with Moving Objects James Davis Computer - PowerPoint PPT Presentation

Mosaics of Scenes with Moving Objects James Davis Computer Science Department Stanford University Mosaics of Scenes with Moving Objects - CVPR98

Motivation • Panoramic imagery • Large high resolution images Mosaics of Scenes with Moving Objects - CVPR98

Overview Registration algorithms do not account for moving objects. Use phase correlation and estimate correct projective geometry. Mosaics with many images are ruined by accumulated registration errors. Find many local registrations and solve a linear system to obtain global registration. The final mosaic is blurry in regions of motion. Segment the mosaic into disjoint regions and fill each from a single source image. Mosaics of Scenes with Moving Objects - CVPR98

✞ ✒ ✘ ☎ ✍ ✗ ✚ ✣ ✱ ✣ ✒ ✚ ☎ ✣ ✍ ✗ ☎ ✥ ✜ ✜ ✯ ✚ ✒ ✍ ✛ ☎ ✗ ✒ ☎ ✄ ✗ ✛❃ ☎ ✢ ✩ ✕ ✲ ✔ ✁ ✆ ✦ ☎ ✣ ✍ ✒ ✎ ✄ ✢ ✄☎ ✣ ✁ ☎ ✣ ✍ ✒ ✢ ✣ ✒ ✚ ✍ ❄ ✚ ✓ ✘ ✆ ✬ ☎ ✂ ✴ ✗ ✝ � ✞ ✽ ✩ ✩ ✰ ✆ ✭ ✖ ✷ ✽ ✩ ✖ ✕ ❁ ✖ ✕ ✝ ✱ ✂ ✎ ✢ ✗ ✄ ✆ ✍ ✄ ✥ ✯ ✄ ✘ ✗ ✓ ✺ ✖ ✍ ✒ ✥ ☞ ✱ ☞ ✿ ✢ ❁ ✡ ✆ ✞ ✕ ✆ ✢ ✗ ✓ ☞ ❂ ✆ ✞ ✽ ✰ ✄ ✣ ✆ ✦ ☎ ✣ ✍ ✒ ✗ ✓ ✍ ✒ ✚ ✄ ☎ ✍ ✠ ✒ ✣ ✓ ✄ ✥ ✜ ✍ ✒ ✥ ☞ ✓ ✒ ☎ ✺ ✠ ☎ ✖ ✼ ✩ ✩ ✰ ✒ ✚ ☞ ✘ ☞ ✯ ✆ ✭❅ ✮ ✻ ✠ ✰ ✮ ✵ ✫ ✮ ★ ✮ ✰ ✆ ✺ ✱ ✯ ✲ ✗ ✍ ✩ ✆ ✜ ✓ ✣ ✁ ✡ ✆ ✞ ✝ ✆ ✢ ✄ ✷ ✯ ✞ ✎ ✿ ✆ ✷ ✄ ☎ ✂ ✴ ✗ ✝ � ✞ ✽ ✩ ✗ ✒ ✓ ✤ ✣ ✛ ✂ ✘ ☞ ✢✣ ✂ ✒ ✚ ✣ ✄ ✛ ✍ ✹ ✢ ✣ ✚ ☎ ✣ ✍ ✒ ✗ ✒ ✄☎ ✚ ✄ ✢ ✜ ✄ ✚ ✄ ✕ ✂ ✘ ☞ ✯ ✆ ✭✮ ✖ ✩ ✬ ✜ ✆ ✫ ✩✪ ✗✜ ✚ ✢ ✘ ✘ ✍ ✝ ★ ✚ ✂ ✗✜ ✢ ✧ ✢ ✄ ☞ ✒ ☞ ✗ ✒ ✢ ✒ ✔ ✡ ✆ ✞ ✳ ✞ ✕ ✆ ✛ ✢ ✲ ✰ ✆ ✞ ✝ ✆ ☎ ☎ ✗ ✱ � ✞ ✪ ✩ ✩ ✒ ✜ ✗ ✗ ✂ ✎ ✗ ✢✣ ✜ ✗✜ ✛ ✄ ✗✚ ✙ ✄ ✗✘ ☎ ✖ ✣ ✔✕ ✄ ✒ ✑ ✡ ✆ ✞ ✠ ✆ ✄☎ ✂ ✁ � ✒ ✤ ✓ ☎ ✣ ✁ ✆ ✦ ☎ ✣ ✍ ✒ ✗ ✘ ✍ ✤ ✗ ✒ ✍ ✄☎ ✓ ☎ ✢✣ ✍ ✤ ☎ ✄ ✥ ✗ ☞ ✒ ✢ ☞ ✍ ✥ ☞ ✕ ✆ ✍ ✑ ✚ ✍ ✥ ✄ ✆ ✞ ✿ ✓ ✂ ✆ ✝ � ✞ ✻ ✩ ✩ ✰ ✢ ✄ ✙ ✙ ✄ ✤ ✞ ✡ ✾ ✗ ✹ ✣ ✚ ✣ ✢ ✎ ✍ ✱ ✆ ✦ ✚ ✎ ✍ ✚ ✲ ✣ ✱ ✄ ✘ ✗ ✓ ✺ ✓ ✗ ✢ ✣ ☎ ✗ ✣ ✓ ✆ ✎ ✗ ✭✼✽ ✶ ✖ ✂ ✘ ✍ ✂ ✘ ☎ ✍ ✒ ☞ ✍ ✢ ✒ ✚ ☎ ✣ ✁ ✚✵ ✴ ✣ ✥ ✥ ✄ ✥ ☞ ✦ ✆ ✁ ✺ ✣ ✲ ✩✻ ✄ ✛ ✍ ✤ ✜ ✆ ✓ ✭✼ ✢✣ ✹ ✚ ✄ ✘ ✗ ✭ ✖ ✺ Mosaics of Scenes with Moving Objects - CVPR98 ✍✏✎ ✍✏✓ ✒✸✷ Related Work • Pairwise Registration • Global Registration ✍✏✎ ✍✏✓ ✍✏✎ • Moving Objects ✍✏✓ ✍✏✎ ☛✌☞ ✝✟✞ ✝✟✞ ✍✏✎ ✝✌❀ ✝✟✞ ✍✏✎

Pairwise Registration • Fixed center of projection • Parameter estimation • Robust registration with moving objects Mosaics of Scenes with Moving Objects - CVPR98

Phase correlation recovers translation I 2 (x, y) = I 1 (x-x 0 , y-y 0 ) x 0 δ (x-x 0 , y-y 0 ) = F -1 F * [ I 2 ]F[ I 1 ] |F * [ I 2 ]F[ I 1 ]| Mosaics of Scenes with Moving Objects - CVPR98

Phase correlation is not biased by moving objects (a) (b) (c) (d) Mosaics of Scenes with Moving Objects - CVPR98

Mellin transform recovers rotation and translation • Extends phase correlation • Polar transform converts rotation into translation • Assumes orthogonal projection Mosaics of Scenes with Moving Objects - CVPR98

Finding the projection matrix • Possess 2D parameters (x 0 , y 0 , θ 0 ) f x 0 • Desire 3D Euler rotation angles ( α , β , θ ) x 0 • Assume small angular rotation f   +   f x 2 x 2   • α β θ =   θ 0 0 ( , , ) ( 2 arctan , 2 arctan , )     0 f y     0 • A = C -1 RC I 2 x A : Image plane projection matrix f 2 x C : Intrinsic camera matrix α f 1 R : 3D rotation matrix f I 1 Mosaics of Scenes with Moving Objects - CVPR98

Without global registration errors accumulate Mosaics of Scenes with Moving Objects - CVPR98

Relating pairwise and global registration Reference image I m a P 1 g e 1 Image 2 P 2 A 21 • A ij P j = P i • Given all A ij , find all P k A ij : Pairwise projection of image i onto image j P k : Projection of image k onto global reference plane Mosaics of Scenes with Moving Objects - CVPR98

Globally registered mosaic Mosaics of Scenes with Moving Objects - CVPR98

Compositing • Blending produces blurring • How do we avoid this? Mosaics of Scenes with Moving Objects - CVPR98

Segment the mosaic • A single source image per region • Avoiding artifacts along boundaries? Mosaics of Scenes with Moving Objects - CVPR98

Finding boundaries • Avoid contradictory information • Relative difference image • Minimum difference path Mosaics of Scenes with Moving Objects - CVPR98

Mosaic without blurring Mosaics of Scenes with Moving Objects - CVPR98

Global registration comparison Mosaics of Scenes with Moving Objects - CVPR98

Compositing comparison Mosaics of Scenes with Moving Objects - CVPR98

Summary • Contributions Mosaics of Scenes with Moving Objects - CVPR98

Discussion • Quantify small angle approximation • Matrix elements do not have uniform scales • Avoid segmenting into tiny regions • Moving objects must appear in one image Mosaics of Scenes with Moving Objects - CVPR98