In the name of Allah the compassionate, the merciful
D IGITAL V IDEO S YSTEMS S. Kasaei Room: CE 307 Department of Computer Engineering Sh Sharif University of Technology if U i it f T h l E-Mail: skasaei@sharif.edu Webpage: http://sharif.edu/~skasaei Webpage: http://sharif.edu/ skasaei Lab. Website: http://ipl.ce.sharif.edu
A CKNOWLEDGMENT A CKNOWLEDGMENT Most of the slides used in this course have been provided by: Prof Yao Wang (Polytechnic provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications g by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: 0130175471. [SUT Code: TK 5105 2 W36 2001] [SUT Code: TK 5105 .2 .W36 2001].
C HAPTER 14 Error Control in Video Communications
O UTLINE O UTLINE � Necessity & challenge for error control y g Kas saei � Characteristics of typical applications & networks � Overview of techniques � Error resilient encoding � Error concealment � Encoder-decoder-network interactive error control 6
V IDEO C OMMUNICATION S YSTEM V IDEO C OMMUNICATION S YSTEM Kasaei A typical video communication system. 7
C HALLENGE FOR V IDEO C OMMUNICATIONS C OMMUNICATIONS � Effective video communications include: Kas saei � Reduction of video data rate. � Handling errors & losses in communication networks. � Data communications are not usually subject to � Data communications are not usually subject to strict delay constraints. � Thus can be handled using network protocols (that use resubmission to ensure error free delivery. resubmission to ensure error-free delivery. � Transmission error categories are: � Random bit errors, caused by imperfections of physical , y p p y channels; bit inversion/insertion/deletion. � Erasure errors, caused by packet lost in packet networks, long burst errors in storage media due to physical defects, system failures/link downs random bit errors in VLC system failures/link downs, random bit errors in VLC 8 coded streams.
C HALLENGE FOR V IDEO C OMMUNICATIONS C OMMUNICATIONS � Real networks are unreliable � Kas saei � Wireless networks: random bit errors, long burst errors, & possibly link downs. � Internet: packet loss & variable delay due to Internet: packet loss & variable delay due to network congestion. � Excessive delay = loss for real-time applications. � Real networks are heterogeneous in bandwidth & reliability. � Video data are delay-sensitive: � One cannot rely on retransmission for error control because of the stringent delay control because of the stringent delay 9 requirement!
C ONVENTIONAL S OURCE C ODING T ECHNIQUE IS NOT G OOD E NOUGH IS NOT G OOD E NOUGH � Conventional source coding techniques g q Kas saei difficulties: � Optimal performance is obtained only for fixed rate & perfect channels. � It results in a poor reconstruction quality when parts of the coded data are lost. 10
C ONVENTIONAL S OURCE C ODING T ECHNIQUE IS NOT G OOD E NOUGH IS NOT G OOD E NOUGH � Compressed video data is very sensitive to Kas transmission errors because of: saei � Variable length coding. � Temporal predictive coding. � Spatial predictive coding. S ti l di ti di � All contribute to error propagation within the same frame as well as the following frames: � 1 bit error or packet loss can render the following � 1 bit error or packet loss can render the following received data useless. 11
S PATIAL /T EMPORAL E RROR P ROPAGATION S PATIAL /T EMPORAL E RROR P ROPAGATION Kasaei Illustration of spatiotemporal error propagation Illustration of spatiotemporal error propagation. 12
D RIFT P ROBLEM (R EFERENCE M ISMATCH ) (R EFERENCE M ISMATCH ) Kas � Motion compensated temporal prediction � Motion compensated temporal prediction saei should be retained to preserve the coding efficiency. efficiency. � Loss in a previous frame can cause mismatch between the reference frame used in the encoder & that in the decoder: � Encoder & decoder out of sync. 13
D RIFT P ROBLEM (R EFERENCE M ISMATCH ) (R EFERENCE M ISMATCH ) Kasaei Distortion Satellite dish referen refere nce ence transmission 14 encoder decoder
E FFECT OF T RANSMISSION E RRORS E FFECT OF T RANSMISSION E RRORS Kasaei 3% Coded, no loss 10% 5% 15 Reconstructed video frames using a H.263 coded sequence, subject to packet losses.
C HANNEL C ODING B ASICS C HANNEL C ODING B ASICS � Channel coding; forward error correction C g; f Kas saei (FEC): � Adding redundancy bits on compressed source bit to e able e bits to enable error detection & correction. o detectio & co ectio � Simple example: � Adding a parity check bit at the end of a block of data stream, can detect all single bit errors. � Channel coding rate: � For every k source bits, add l channel bits, to create n=k+l bits channel coding rate r=k/n . � Well designed code ( e.g., Reed-Solomon (RS) code) can correct t=l/2 error bits in each n -bit block. 16
C HANNEL C ODING B ASICS C HANNEL C ODING B ASICS � Classical Shannon information theory states that Kas one can separately design the source & channel l d i h & h l saei coders, to achieve error-free delivery of a compressed bit stream, as long as the source is represented by a rate below the channel capacity represented by a rate below the channel capacity. � Source coding minimizes the bit rate necessary to satisfy a distortion criterion (Shannon rate-distortion theory). � Channel coding adds just enough redundancy bits to g j g y reduce the raw channel error rate to the permitted level. � Such ideal error-free delivery requires infinite delays in implementing (FEC). d l i i l i (FEC) � Only valid for stationary source & channel, requires processing of infinitely long blocks of data (delay = infinity!) infinity!). 17
FEC FOR V IDEO T RANSMISSION FEC FOR V IDEO T RANSMISSION � For wireless networks, FEC is necessary to reduce Kas raw bit error rates. saei � For the Internet, errors are mainly due to congestion- caused packet losses, FEC can be applied across packets to correct/detect packet losses. � Unequal error protection (UEP): using stronger channel codes ( r=k/n smaller) for more “important” bit streams (base layer). Best implemented with rate- bit t (b l ) B t i l t d ith t 18 compatible punctured convolutional (RCPC) code.
T RANSMISSION E RROR R ESILIENCE T RANSMISSION E RROR R ESILIENCE � Joint source & channel coding is often a g Kas more viable scheme (which allocates a saei total amount of redundancy between the source & channel coding). so ce & c a e co g). � All error resilient encoding methods work under this promise. d thi i � This is usually accomplished by carefully � This is usually accomplished by carefully designing both the predictive coding loop & the VLC, to limit the extend of error propagation propagation. 19
E RROR C ONCEALMENT E RROR C ONCEALMENT � When a part of image is missing due to Kas transmission errors, the decoder estimates them i i h d d i h saei based on surrounding received samples (using the inherent correlation among spatially & temporally adjacent samples) adjacent samples). � It does not employ any additional bit rate. ☺ � But adds computational complexity at the decoder � � But adds computational complexity at the decoder. � � To facilitate error concealment in the decoder, the codec & the network transmission protocol must p cooperate. For example: � Important parts should be assigned a more stringent set of QoS parameters. � Network may provide a feedback channel (so the encoder N t k id f db k h l ( th d 20 does not use this part for prediction).
T YPICAL V IDEO A PPLICATIONS & C OMMUNICATION N ETWORKS & C OMMUNICATION N ETWORKS � Some basic definitions: Kas saei � Latency: maximally allowed average end-to-end delay. � Jitter: delay variation. � Real-time delivery: compressed data are transferred at R l ti d li d d t t f d t a speed that matches the coded video source rate. 21
Q O S R EQUIREMENTS OF T YPICAL V IDEO A PPLICATIONS OF T YPICAL V IDEO A PPLICATIONS � Video applications consist of: pp Kas saei � Interactive two-way visual communications. � One-way video streaming. � One-way video downloading. O id d l di � No difference from file downloading. 22
I NTERACTIVE T WO -W AY V ISUAL C OMMUNICATIONS C OMMUNICATIONS � Examples include: teleconferencing, video Kas telephony virtual classrooms telephony, virtual classrooms. saei � They have very stringent delay requirements. � The latency & jitter must be kept within a certain limit. � For intercontinental telephone conversations (& video conferencing), the ITU-T G.114 standard recommends: � <=150 ms (one way) desired � <=150 ms (one way) desired. � 150-400 ms can be acceptable. � >400 ms not acceptable. � Audio & video must be in sync to maintain lip synchronization. h i ti 23 � Both encoding & decoding must be completed in real-time.
Recommend
More recommend
