Exercise 8: Preparation 1 Download test video sequences from course - PowerPoint PPT Presentation

Exercise 8: Preparation 1 Download test video sequences from course web-site http://www.inf.fu-berlin.de/lehre/SS20/ImageVideoCoding/ivc.htm version 1: YCbCr 4:2:0 format version 2: Gray images (will use this format for now) 2 Install a player for YUV images and videos, for example: pYUV player (free): (recommended) http://dsplab.diei.unipg.it/software YUVToolkit (free): http://www.yuvtoolkit.com/ vooya (free for Linux): http://www.offminor.de/ Note: The pYUV player can detect the format based on the filename (test sequences are named in a way that they can be directly opened) 3 Download and compile the PSNR tool for video sequences Gray videos: ./psnr (width) (height) 400 (org) (rec) [ (stream) (frame rate) ] YCbCr videos: ./psnr (width) (height) 420 (org) (rec) [ (stream) (frame rate) ] Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 8a: Prepare Encoder/Decoder Framework (Step 1) Modified Raw Data Format : Gray video sequences (may later use YCbCr 4:2:0) Raw video sequences do not contain any header information Contain 8-bit samples (0..255) in binary form (picture by picture, line by line inside each picture) Encoder Modification Add command line parameter that specifies picture size (something like “ -size 416x240 ” ) Read raw sample data (using picture size specified in command line) Decoder Modification Write raw sample data without header data Suggestion: Keep PGM image format alive Encoder: Use conventional PGM if no parameter “ -size WxH ” is given in command line Decoder: Add command line option “ -pgm ” that specifies whether output is written as PGM image or raw data samples Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 8b: Implement Multi-Frame Encoder/Decoder (Step 2) Goal : Encoding and decoding of video sequences (without any additional coding tools) Encoder Modification Add command line parameter that specifies number of pictures to be encoded (e.g., “ -n 100 ” ) Code the number of specified pictures using our current image encoding Write all data to the same bitstream (just picture after picture) Decoder Modification Decode all pictures included in bitstream (detect by checking for end of file) Write all decoded samples to the same file (raw video data in gray format) Test Coding Efficiency ( version A : intra-only coding) Determine rate-PSNR curve for two selected video sequences (e.g., “Johnny” and “BQSquare”) Use 50 pictures of each video and QP values 12, 16, 20, 24, 28 Use new PSNR tool for calculating average PSNR and bit rate Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 8c: Simple Version of Frame Difference Coding (Step 3) Implement First Video Codec (using inter-picture dependencies) First picture is coded using intra-picture coding (no changes to our image codec) All blocks of all following pictures are coded as follows Prediction signal ˆ s n [ x , y ] is given by co-located samples in reconstructed previous picture s ′ n − 1 [ x , y ] Prediction error u n [ x , y ] = s n [ x , y ] − ˆ s n [ x , y ] is transmitted using transform coding (as usual) Encoder/Decode Modification Store last reconstructed picture in encoder and decoder Implement function for copying samples from last reconstructed picture (inter prediction) Note: Intra prediction modes are only coded for first picture Test Coding Efficiency ( version B : simple frame difference coding) Determine rate-PSNR curve in same way as for Exercise 8b Compare rate-PSNR curves for versions A and B by plotting them into the same diagram Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding

Exercise 8d: Improved Version of Frame Difference Coding (Step 4) Improved Codec Concept (first picture unchanged) Block-adaptive decision between intra and difference coding Coding mode signaled by a flag (new probability model) block syntax : if( frame_idx != 0 ) Encoder Modification coding_mode // new flag if( coding_mode == 0 ) // intra Test difference mode in addition to intra modes intra_prediction_mode coded_block_flag Lagrangian mode decision (use correct bits !) if( coded_block_flag == 1 ) { last_pos_x last_pos_y { quantization indexes } Decoder Modification } Read coding mode at start of a block Apply intra or inter prediction depending on coded data Test Coding Efficiency ( version C : improved frame difference coding) Determine rate-PSNR curve in same way as for Exercises 8a and 8b Compare rate-PSNR curves for versions A, B, and C by plotting them into the same diagram Heiko Schwarz (Freie Universität Berlin) — Image and Video Coding