In the name of Allah the compassionate, the merciful
Digital Video Systems S. Kasaei S. Kasaei Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei Lab. Website: http://ipl.ce.sharif.edu
Acknowledgment Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001]
Chapter 5 Video Modeling
Outline � Camera model � Object model � Shape model � Motion model � Scene model � 2-D motion model
Why Modeling? � To describe various events in a video processing system in parametric forms. � To enable estimation of these parameters.
Pinhole Camera � Perfect image if the hole is infinitely small. � Pure geometric optics. � No depth of field issue.
Simplified Pinhole Camera � Eye-image pyramid (frustum). � Note that the distance/size of image is arbitrary.
Pinhole Camera 3-D Y point X Z Y X Focal X Length Z C y F x Focal /Camera Center x y 2-D The image of an object is reversed from its x Imaging image 3-D position. The object appears smaller Plane when it is farther away.
Perspective vs. Orthographic Parallel Lines Perspective Orthographic
Pinhole Camera Model: Perspective Projection All points in this ray will have the same image. x X y Y X Y = = ⇒ = = , x F , y F F Z F Z Z Z x , y are inversely related to Z
Approximate Model: Orthographic Projection → ∞ When the object is very far ( Z ) = = x X y , Y Can be used as long as the depth variation within the object is small compared to the distance of the object.
Perspective Projection
Rigid Object Motion Rotation and translati on wrp. the object center : = − + + θ θ θ X ' [ R ]( X C ) T C ; [ R ] : , , ; T : T , T , T x y z x y z
Flexible Object Motion � Two ways to describe it: � Decompose into multiple, but connected rigid sub-objects. � Ex., Human body consists of many parts each undergoes a rigid motion. � Global motion plus local motion in sub- objects. � Ex., Global camera motion plus local object motions.
Scene Models � A scene is determined by: � Illumination � Objects in the scene (their shape, motion, & relative positions) � Camera � 3-D scene model � 2.5-D scene model � 2-D scene model
3-D Scene Model � Perspective camera (object image depends on depth). � Ambient illumination. � Objects at varying depth. � Used for 3-D motion/structure estimation.
2.5-D Scene Model � Orthographic camera (depth has no effect on object image). � Ambient illumination. � Objects at varying depth (layered objects, MPEG-4).
2-D Scene Model � Orthographic camera. � Ambient illumination. � Objects are flat & at the same depth (H.261, H.263, MPEG-1, MPEG-2).
2-D Scene Model � Projection of 3-D motion � Camera motion � Rigid object motion � Projective mapping � Approximation of projective mapping � Affine model � Bilinear model
3-D Motion -> 2-D Motion 3-D MV 2-D MV
Sample Motion Field Motion Motion Field Vector
Occlusion Effect Motion is undefined in occluded regions.
Typical Camera Motions Mounted Camera
2-D Motion Corresponding to Camera Motion Camera zoom Camera rotation around Z-axis (roll)
Linear Transformations
Affine Transformations � Preserves parallel lines. P. Angles P. Distances & Angles
Projective Transformations � Preserves lines.
Projective Mapping Sampled Perspective Affine Translation
Projective Mapping Rigid Affine Projective Nonlinear
Motion Field Corresponding to Different 2-D Motion Models Translation Affine Bilinear Perspective
Projective Mapping nonequal nonparallel Co Ch&Co Ch&Co Ch&Co Ch parallel Two features of projective mapping: � Chirping: increasing perceived spatial frequency for far away objects [ ]. � Converging (Keystone): parallel lines converge in distance [ ].
2-D Motion Corresponding to Rigid Object Motion X ' r r r X T 1 2 3 x 12-parameter � General case: = + Y ' r r r Y T 4 5 6 y Z ' r r r Z T 7 8 9 z → Perspectiv e Projection + + + ( r x r y r F ) Z T F = x ' F 1 2 3 x + + + ( r x r y r F ) Z T F 7 8 9 z + + + ( r x r y r F ) Z T F 4 5 6 y = y ' F + + + ( r x r y r F ) Z T F 7 8 9 z � Prospective mapping: = + + When the object surface is planar (Z aX bY c) : + + + + a a x a y b b x b y = = 0 1 2 0 1 2 x ' , y ' 8-parameter + + + + 1 c x c y 1 c x c y 1 2 1 2
Affine and Bilinear Models � Affine (6-parameter): + + d ( x , y ) a a x a y x 0 1 2 = + + d ( x , y ) b b x b y y 0 1 2 � Good for mapping triangles to triangles. � Cannot capture either chirping or converging effect.
Affine and Bilinear Models � Bilinear (8-parameter): + + + d ( x , y ) a a x a y a xy x 0 1 2 3 = + + + d ( x , y ) b b x b y b xy y 0 1 2 3 � Good for mapping blocks to quadrangles. � Can capture converging effect of projective mapping but not chirping effect.
Homework 3 � Reading assignment: � Chapter 5: Sec. 5.1, 5.4, & 5.5 � Written assignment: � Prob. 5.2, 5.3, 5.4, 5.5, & 5.6 � Correction to Problems: � 5.3: Show that the projected 2-D motion of a 3-D object undergoing rigid motion can be described by Eq.(5.5.13). � 5.4: Change aX+bY+cZ=1 to Z=aX+bY+c. � Other corrections: � P.125, Fig. 5.11 caption: “diffuse”->”ambient”.
The End
Recommend
More recommend
