In the name of Allah the compassionate, the merciful
Digital Video Systems S. Kasaei S. Kasaei Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei Lab. Website: http://ipl.ce.sharif.edu
Acknowledgment Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001].
Chapter 2 Fourier Analysis of Video Signals & Frequency Response of the HVS
Outline � Fourier transform over multidimensional space: � Continuous-space FT (CSFT) � Discrete-space FT (DSFT) � Frequency domain characterization of video signals: � Spatial frequency � Temporal frequency � Temporal frequency caused by motion Kasaei 6
Outline � Frequency response of the HVS: � Spatial frequency response � Temporal frequency response & flicker � Spatio-temporal response � Smooth pursuit eye movement � Video sampling (a brief discussion) Kasaei 7
Continuous-Space Signals � K-D space continuous signals: R : set of real numbers k ψ = ∈ ( x ), x [ x , x ,..., x ] R real or complex 1 2 K X : K-D � Convolution: continuous variable ∫ ψ = ψ − ( x ) * h ( x ) ( x y ) h ( y ) d y k R � Example function: � Dirac delta function: Kasaei 8
Continuous-Space Fourier Transform (CSFT) � Forward transform: ∫ T Ψ = ψ − π ( f ) ( x ) exp( j 2 f x ) d x c k R � Inverse transform: ∫ Ψ T ψ = π ( x ) ( f ) exp( j 2 f x ) d f c k R � Convolution theorem: ψ ⇔ Ψ ( x ) * h ( x ) ( f ) H ( f ) c c ψ ⇔ Ψ ( x ) h ( x ) ( f ) * H ( f ) c c Kasaei 9
Continuous-Space Fourier Transform (CSFT) � More on inverse transform: ∫ Ψ ψ = π T ( x ) ( f ) exp( j 2 f x ) d f c k R � The inverse CSFT shows that any signal can be expressed as a linear combination of complex exponential function with different frequencies. � The CSFT at a particular frequency represents the contribution of the corresponding complex exponential basis function. � The transform determines the correlation between the input signal & its projection on some defined basis function. � Orthogonal basis functions preserve the signal energy in the transform domain. Kasaei 10
Continuous-Space Systems � General system over K-D continuous space: φ = ψ ∈ k ( x ) T ( ( x )), x R � Linear & (Space) Shift-Invariant (LSI) System: α φ + α φ = α ψ + α ψ ( x ) ( x ) T ( ( x ) ( x )) 1 1 2 2 1 1 2 2 ψ + = φ + T ( ( x x )) ( x x ) 0 0 � LSI system can be completely described by its impulse response: ( ) = δ h ( x ) T ( x ) φ = ψ ⇔ Φ = Ψ ( x ) ( x ) * h ( x ) ( f ) ( f ) H ( f ) c c c Kasaei 11
Discrete-Space Signals � K-D space discrete signals: ψ = ∈ K ( n ), n [ n , n ,..., n ] Z 1 2 K � Convolution: ∑ ψ = ψ − ( n ) * h ( n ) ( n m ) h ( m ) ∈ K m Z � Example function: � Kronecker delta function: Kasaei 12
Discrete-Space Fourier Transform (DSFT) � Forward transform: ∑ Ψ = ψ − π T ( f ) ( n ) exp( j 2 f n ) d K ∈ n R Ψ ( f ) is periodic in each dimension with period of 1 d { } K = ∈ − unit freq: Fundamenta l period : I f , f ( 1 / 2 , 1 / 2 ) k hypercube repeats @ integer points � Inverse transform: ∫ T ψ = Ψ π ( n ) ( f ) exp( j 2 f n ) d f d ∈ K f I � Convolution theorem: ψ ⇔ Ψ ( n ) * h ( n ) ( f ) H ( f ) d d ψ ⇔ Ψ ( n ) h ( n ) ( f ) * H ( f ) Kasaei 13 d d
Frequency Domain Characterization of Video Signals � Spatial frequency � Temporal frequency � Temporal frequency caused by motion Kasaei 14
Spatial Frequency � Spatial frequency measures how fast the image intensity changes in the image plane. � Spatial frequency can be completely characterized by the variation frequencies in two orthogonal directions ( e.g., horizontal & vertical): � f x : cycles/horizontal unit distance. � f y : cycles/vertical unit distance. � It can also be specified by magnitude & angle of change: 2 2 = + θ = f f f , arctan( f / f ) m x y y x Kasaei 15
Illustration of Spatial Frequency Kasaei 16
Angular Frequency � Problem with previous defined spatial frequency: � Perceived speed of change depends on the viewing distance. 180 h θ = ≈ = 2 arctan( h / 2 d ) (radian) 2 h/2d(radia n) (degree) π d π f d = = s f f ( cycle/degr ee) Kasaei 17 θ s θ 180 h
Angular Frequency � For the same picture, the angular frequency increases as the viewing distance increases. � For a fixed viewing distance, a larger screen size leads to lower angular frequency. � The same picture appears to change more rapidly when viewed farther away, & it changes more slowly if viewed from larger screen. � It depends on both the spatial frequency in the signal & the viewing conditions. Kasaei 18
Temporal Frequency � Temporal frequency measures temporal variation (cycles/s). � In a video, the temporal frequency is spatial position dependent, as every point may change differently. � Temporal frequency is caused by camera or object motion. � It depends not only on the motion, but also on the spatial frequency of the object. Kasaei 19
Temporal Frequency caused by Linear Motion Kasaei 20
Relation between Motion, Spatial, & Temporal Frequency = Consider an object moving with speed ( v , v ). Assume the image pattern at t 0 x y ψ is ( x , y ), the image pattern at time t is 0 X’ y’ under ψ = ψ − − ( x , y , t ) ( x v t , y v t ) CIA 0 x y ⇔ convolution Ψ = Ψ δ + + ( f , f , f ) ( f , f ) ( f v f v f ) x y t 0 x y t x x y y 2-D CSFT Relation between motion, spatial, and temporal frequency : nonzero on plane = − + f ( v f v f ) t x x y y The temporal frequency of the image of a moving object depends on motion as well as the spatial frequency of the object (projection of v on f ). Example: A plane with vertical bar pattern, moving vertically, causes no temporal change; but moving horizontally, it causes fastest temporal change. Kasaei 21
Illustration of the Relation = = ⇒ = f f 0 f 0 x y t Kasaei 22
Frequency Response of HVS � Temporal frequency response & flicker � Spatial frequency response � Spatio-temporal response � Smooth pursuit eye movement Kasaei 23
Frequency Responses of HVS � Most of the video systems are ultimately targeted for human viewers. � It is important to understand how the human perceives a video signal. � Sensitivity of the HVS to a visual pattern depends on the spatial & temporal frequency content of the pattern. Kasaei 24
Frequency Responses of HVS � The visual sensitivity is highest at some intermediate spatial & temporal frequencies. � It then falls off & diminishes at some cut-off frequencies. � Spatial or temporal changes above these frequencies are invisible to the human eye. � They form the basis for determining the frame & line rates in video capture & display systems. Kasaei 25
Temporal Frequency Responses � The temporal frequency response of the HVS refers to the visual sensitivity to a temporally varying pattern at different frequencies. � The temporal response of an observer depends on viewing distance, display brightness, ambient lightning, & … � The temporal response of the HVS is similar to a BPF. Kasaei 26
Temporal Frequency Responses � The peak increases with the mean brightness of the image. � One reason that the eye reduces sensitivity at higher temporal frequencies is because the eye can retain the sensitivity of an image for a short time interval (even when the actual image has been removed). � This phenomenon is known as the persistence of vision. Kasaei 27
Flicker Perception � It causes temporal blurring, if a pattern changes in a rate faster than the refresh rate of the HVS. � It allows the display of a video signal as a consecutive sequence of frames. � As long as the frame interval is shorter than the visual persistence period, the eye perceives a continuously varying image. � Otherwise, the eye will observe frame flicker. Kasaei 28
Flicker Perception � The lowest frame rate at which the eye does not perceive flicker is known as the critical flicker frequency . � The brighter the display, the higher the critical flicker frequency. � The motion picture industry uses 24 frames/sec. Kasaei 29
Flicker Perception � The TV industry uses 50/60 fields/sec. � The computer display uses 72 frames/sec. � A troland is the unit used to describe the intensity of light entered the retina. Kasaei 30
Recommend
More recommend
