In the name of Allah f the compassionate, the merciful p ,
Digital Video Processing g g S. Kasaei S. Kasaei Room: CE 307 Room: CE 307 Department of Computer Engineering Sharif University of Technology E-Mail: skasaei@sharif.edu a s asae @s a edu Webpage: http://sharif.edu/~skasaei Lab. Website: http://ipl.ce.sharif.edu
Acknowledgment g Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 .2 .W36 2001].
Chapter 9 p Waveform-Based Coding: g Transform & Predictive Coding
Outline � Overview of video coding systems � Overview of video coding systems � Transform coding � Predictive coding Predictive coding 6 Kasaei
Components in a Coding System Focus of this lecture. 7 Kasaei
Video Coding � Reduces the data rate of a video sequence by Reduces the data rate of a video sequence by exploiting the spatial & spectral correlation between neighboring pixels as well as temporal correlation b t between consecutive frames of the sequence. ti f f th � Spatial & spectral correlation is due to the fact that color values of adjacent pixels in the same video frame usually j p y change smoothly. � Temporal correlation refers to the fact that consecutive frames usually show the same physical scene occupied by frames usually show the same physical scene, occupied by the same object that may have moved. 8 Kasaei
Transform Coding � Motivation: � Motivation: � Represent a vector ( e.g., a block of image pels) as the superposition of some typical vectors p p yp (basic patterns, transform basic functions). � Quantize & code the coefficients. � Can be thought of as a constrained vector quantizer. transform basis transform coefficients + t 1 t 2 t 3 t 4 9 Kasaei
Transform Coding Block Diagram 10 Kasaei
Which Transform Basis to Use? � The transform should: � The transform should: � Minimize the correlation among resulting coefficients, so that scalar quantization can be , q employed without losing too much in coding efficiency (compared to vector quantization). � Compact the energy into as few coefficients as possible. 11 Kasaei
Which Transform Basis to Use? � Optimal transform: Optimal transform: � Karhunen-Loeve transform (KLT): signal statistics dependent (ensemble dependent, 2 nd order statistics). t ti ti ) � Suboptimal transform: � Discrete cosine transform (DCT): nearly as good as Discrete cosine transform (DCT): nearly as good as the KLT for common image signals (highly correlated signals). � Wavelet transform (WT): a powerful multiresolution tool that utilizes space-frequency properties of non- stationary signals stationary signals. 12 Kasaei
General Linear Transform � Linearly independent basis vectors (or blocks): � Linearly independent basis vectors (or blocks): � Inverse transform represents a vector (or � Inverse transform represents a vector (or block) as the superposition of basis vectors (or blocks): blocks): � Forward transform determines the contribution � Forward transform determines the contribution (weight) of each basis vector: s t s V U 13 Kasaei
Unitary Transform � Unitary (orthonormal) basis: � Unitary (orthonormal) basis: � Basis vectors are orthogonal to each other & each has length 1: g � Transform coefficient associated with a basis vector is simply the projection of the input vector is simply the projection of the input vector onto the basis vector: 14 Kasaei
Discrete Cosine Transform Basis Images 15 Kasaei
Energy Distribution of DCT Coefficients in Typical Images 16 Kasaei
Images Approximated by Different Number of DCT Coefficients With 4/64 Original Original Coefficients Image With 8/64 With 164/64 Coefficients Coefficients 17 Kasaei
Demos � Use Matlab demo to demonstrate � Use Matlab demo to demonstrate approximation by using different number of DCT coefficients. (dctdemo.m) 18 Kasaei
Distortion in Transform Coding � Distortion in sample domain: � Distortion in sample domain: � Distortion in coefficient domain: � These two distortions equal when using � These two distortions equal when using unitary transform: 19 Kasaei
Modeling of Distortion due to Coefficient Quantization � High Resolution Approximation of Scalar � High Resolution Approximation of Scalar Quantization: � With the MMSE quantizer, when each coefficient � With the MMSE quantizer, when each coefficient is scalar quantized with sufficiently high rates, so that the pdf in each quantization bin is approximately flat we have: i l fl h Depends on the pdf of the k th coefficient Depends on the pdf of the k -th coefficient. 20 Kasaei
Optimal Bit Allocation among Coefficients � How many bits to use for each coefficient? � How many bits to use for each coefficient? � Can be formulated as an constrained optimization problem: p Minimize: Subject to: � The constrained problem can be converted to unconstrained one using the Lagrange multiplier method: Minimize: 21 Kasaei
Derivation & Result 22 Kasaei
Implication of Optimal Bit Allocation � Bitrate for a coefficient proportional to its variance Bitrate for a coefficient proportional to its variance (energy): geometric mean � Larger variance, more bits. g � Distortion is equalized among all coefficients & depends on the geometric mean of the coefficient variances: i 23 Kasaei
Transform Coding Gain over PCM � Distortion for PCM (SQ samples) if each sample is Distortion for PCM (SQ samples) if each sample is quantized to R bit: � Gain over PCM: arithmetic mean geometric mean 24 Kasaei
Transform Coding Gain over PCM � For any arbitrary set of values, the arithmetic mean For any arbitrary set of values, the arithmetic mean is equal to (or larger than) the geometric mean ( � G >=1). � For Gaussian sources: � If each sample is Gaussian, then coefficients are also p , Gaussian (& are all the same). Then: 25 Kasaei
Example � Determine the optimal bit allocation & corresponding TC p p g gain for coding 2x2 image block, using 2x2 DCT. Assume that the image is a stationary Gaussian process with inter-sample correlation as shown below. ith i t l l ti h b l 26 Kasaei
Example (cnt’d) � Correlation matrix: zero-mean data. Correlation matrix: zero mean data. � DCT basis images: � Equivalent transform matrix: 27 Kasaei
Example (cnt’d) covar. matrix geo. mean g 0 For R=2 : For R=2 : 28 Kasaei
Optimal Transform Design � Optimal transform: p � Should minimize the distortion for a given average bitrate. � Equivalent to minimize the geometric mean of the coefficient variances variances. � When the source is Gaussian, the optimal transform is the KLT (which depends on the covariance matrix of the samples), ensemble dependent. � Basis vectors are the eigenvectors of the covariance matrix, the coefficient variances are the eigenvalues: k- th coeff. var. KLT unit. minimal geo. mean maximize TC 29 Kasaei
Example � Determine the KLT for the 2x2 image block in the g previous example. Determine the eigenvalues by solving: (Same as the coefficient variances with DCT!) Determine the eigenvectors by solving: Resulting transform is the DCT! Resulting transform is the DCT! 30 Kasaei
Data Compression Symbol Data Quantization Encoding Transform O t Output t Original O i i l Bit Stream Image Transform coding s stem Transform coding system. 31 4/14/2008 S. Kasaei
JPEG-98 Compression Run- 8 × 8 Scalar Length DCT Pixel Pixel Quantizer Quantizer & & Block Huffman Original Output I Image Bitstream JPEG 98 encoder JPEG-98 encoder. 32 4/14/2008 S. Kasaei
DCT Transform DCT transform: � + + C C 7 7 2 x 1 2 y 1 ∑∑ = π π u v T S (cos u )(cos v ) u , v x , y 4 4 16 16 16 16 = = y 0 0 x 0 0 2 u, v = 0 = = 2 2 C C u C , C v otherwise 1 S: source block, T: transformed block. T has the same size as S with real coefficients. 33 4/14/2008 S. Kasaei
DCT Transform � In matrix T, elements , Lower Lower containing high energy Frequency levels (with low frequencies) are placed in frequencies) are placed in upper left corner. � When going towards lower Higher right corner, energy level Frequency q y goes down (& frequency goes down (& frequency goes up). DCT coefficients. 34 4/14/2008 S. Kasaei
Scalar Quantization Quantization tables used by JPEG (entries are the stepsizes) Quantization tables used by JPEG (entries are the stepsizes). 35 4/14/2008 S. Kasaei
Scalar Quantization � Quantized matrix has the same size as S & T � Quantized matrix has the same size as S & T, with elements of: T T QT = Q Q: quantization matrix, QT: quantized T. 36 4/14/2008 S. Kasaei
Recommend
More recommend
