Contents: 1. Introduction 1.1 Hybrid Video Coding 1.2 Object - PDF document

The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 ICNNSP’2008: IEEE International Conference on Neural Networks and Signal Processing, 6-10 June 2008, Zhenjiang China On Video Transcoding to Super-Resolution Videos Keynote Speaker: Professor W.C. Siu Chair Professor and Director Centre for Signal Processing Department of Electronic and Information Engineering Hong Kong Polytechnic University Siu_...OnVideoTranscoding 1 The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 Contents: 1. Introduction 1.1 Hybrid Video Coding 1.2 Object Oriented Coding 1.3 Advanced Video Concepts 1.4 A highlight of others of our studies 2. Motion Estimation Algorithms 2.1 Sample studies: Fast Adaptive Search Algorithm, Fast Pixel Decimation 2.2 Sample studies: Fast Exhaustive Full Search, Novel Directional Search 3. Video Transcoding 3.1 Video Transcoding (frame skipping) 3.2 Video Transcoding (H.264 to H.264 conversion) 4. Extending to video Enlargement Super-resolution Videos 4.1 New Edge-Directed Interpolation 4.2 Modified Edge-Directed Interpolation 4.3 SR video Construction 4.4 SR video re-encoding 6. Summary and Conclusion Siu_...OnVideoTranscoding 2 1

The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 1. Introduction 1.1 Hybrid Video Coding In the recent years, there is a remarkable progress in Video Coding. In this talk we mainly concentrate on predictive Hybrid Video Coding. Predictive Coding: Instead of transmitting a frame (called current frame ), its motion vectors with reference to a previous frame (called reference frame ) are transmitted. This will produce the motion compensated frame which consists of prediction errors. Most videos nowadays are coded by using the Hybrid Video Coding which makes use of the Predictive Coding. Siu_...OnVideoTranscoding 3 The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 In order to recover the signal without the motion estimation errors, a motion compensated Residual Frame is constructed, such that Residual frame = Current frame – Motion compensated frame Current Residual frame frame Motion compensated Reference frame frame This predicted residual frame is subsequently coded in a similar manner as an intra frame i.e. through DCT  Quantization  Entropy coding Siu_...OnVideoTranscoding 4 2

The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 Hybrid Video Coding Regulator Regulator Source Compressed video bit stream + Frame VLC VLC Frame Quantizer Quantizer 2D-DCT 2D-DCT Buffer Buffer 011010…111 011010…111 Memory Memory coder coder - Dequantizer Dequantizer Inverse Inverse 2D-DCT 2D-DCT + Predicted frame + Motion Motion Frame Frame Compensation Compensation Memory Memory Motion vectors Motion Motion Estimation Estimation Siu_...OnVideoTranscoding 5 The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 Hybrid Video Coding Error Regulator Source Frame Compressed video bit stream + Frame VLC Quantizer 2D-DCT Buffer 011010…111 Memory coder - Dequantizer Inverse 2D-DCT + Predictive frame + Motion Motion Motion Frame Compensation Compensation Compensation Memory Motion vectors Motion Estimation Siu_...OnVideoTranscoding 6 3

The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 1.2 Object Oriented Coding DM4 • To divide the scene into Background object(s): Hybrid Coding, or Sprite Generation Technique Object • and foreground objects: Extraction Multiple objects Hybrid Coding Object based coding (complete object) Time, position, motion manipulations, etc. • Segmentation is still a problem, since the definition of an object can never make clear to computers. • Object boundaries always merge with the background, etc. Siu_...OnVideoTranscoding 7 The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 1.3 Advanced Video Concepts - from HVC to Advanced Video Coding (H.264) Regulator Regulator Regulator Source Source Source Compressed Compressed Compressed video video video bit stream bit stream bit stream + + + Frame Frame Frame Integer Transform, Scaling, VLC VLC VLC Quantizer Quantizer Quantizer 2D-DCT 2D-DCT 2D-DCT Buffer Buffer Buffer 011010…111 011010…111 011010…111 Memory Memory Memory Quantization coder coder coder - - - Dequantizer Dequantizer Dequantizer Scaling and Intra-frame Inverse Prediction Transform Inverse Inverse Inverse 2D-DCT 2D-DCT 2D-DCT + + + Predictive frame Predictive frame Predictive frame + + + Motion Motion Motion Motion Motion Frame Frame Frame Frame Frame Compensation Compensation Compensation Compensation Compensation Memory Memory Memory Memory Memory Motion vectors Motion vectors Motion vectors Motion Motion Motion Estimation Estimation Estimation Siu_...OnVideoTranscoding 8 4

The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 Advanced Video Coding Regulator Source Compressed video bit stream + Frame Transform, Scaling, VLC Buffer 011010…111 Memory Quantization coder - Digital Transform Scaling and Intra-frame Inverse Prediction Transform + Predictive frame + Variable Block Size Multi- Motion Motion Frame Frame Frame Frame Ref. Frame Compensation Compensation Memory Memory Memory Memory Motion vectors Motion Estimation Siu_...OnVideoTranscoding 9 The Hong Kong Polytechnic University Department of Electronic and Information Engineering, Centre for Multimedia Signal Processing Prof. W.C. Siu, Chair Professor and Centre Director 8 June, 2008 Video Composition: (Object Oriented Processing) 1. Background: Hybrid Coding, or Sprite Generation Technique 2. Foreground: Multiple objects Hybrid Coding Object based coding (complete object) Time, position, motion manipulations, etc. Techniques developed: 1. Motion estimation/Sprite Generation Our own fast hybrid Coding making use new concepts of fast motion estimation , and sprite generation techniques. Siu_...OnVideoTranscoding 10 5