Typical Network Traffic Using Redundancy and Interleaving to Ameliorate the • Majority is text-based Effects of Packet Loss in a Video – File transfer, Email, Web… • Reliability is critical Stream • Latency is not critical • Transmitted Using TCP Yali Zhu, Mark Claypool and Yanlin Liu – Provide reliable service where all bytes arrive Department of Computer Science Worcester Polytechnic Institute CS Technical Report TR-01-03 Multimedia Network Traffic Multimedia Over Internet • High bandwidth • Often suffer from delay, loss and jitter – Can induce congestion � packet loss – degrading multimedia quality • Latency is critical • Loss has the most severe effects on quality • Loss is not critical • Use loss recovery techniques to – Can tolerate some – Improve multimedia quality • Transmitted using UDP – Keep latency low – Provide unreliable service where some packets may be lost Media Specific FEC Multimedia Repair Taxonomy Repair Sender Based Receiver Based Forward Error Interpolation Insertion Regeneration Retransmission Interleaving Correction Repetition • Multiple copies of data • Lower quality of secondary frames 1
Idea of Interleaving Audio Interleaving • Without Interleaving • With Interleaving Encode Interleave WorcesterPolytechnicInstitute otlhnuWsocItreynstcrtiteePeci Transmit sterPolytechnicInstitute WrceserPoytecnicIstitute Decode Goal Groupwork • Above techniques have been done primarily • What are the issues with applying video to audio only redundancy to video vs. audio? • Our goal: • What are the issues with applying interleaving to video vs. audio? – Apply Interleaving and Redundancy to Video • What would be a methodology for evaluating – Evaluate effects on perceptual quality – Evaluate system overhead the benefit? • What performance metrics should you have? MPEG Encoding Techniques Coding Dependency within GOP • Intra-frame encoding – Based on current frame only • Inter-frame encoding I B B P B B P B B I – Based on similarity among frames • Frame types – I-frame (Intra-coded frame) •I frames ( Intra-coded frames) – P-frame (Predictive-coded frame) •P frames ( Predictive-coded frames) – B-frame (Bi-directionally predictive frames) Require information on previous I- or P- frames •B frames ( Bi-directionally predictive-coded frames) Require information on frames before and after 2
Loss Propagation Outline • Loss of one single frame result in multiple • Introduction • Background losses • Approach – Loss of P-frame – Loss of I-frame • System Overhead – B-frame loss has no propagation • Perceptual Quality • Conclusions I B B P B B P B B I Loss Propagation of Second P-frame Repairing a Video Stream Approach • Apply interleaving and redundancy to video .mpg file – Hypothesize will improve perceptual quality mpeg decoder – Measure system overhead 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Original Stream (interleave, redundancy) Interleaver • Build movies (next slide) 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16 18 Interleaved Stream mpeg encoder – With loss, interleaved repair, redundancy repair I B B P B B P B B I B B P B B P B B Encoded Stream • Evaluate with user study (apply loss) Transmitted over Network • Measure system overhead mpeg decoder • Analysis Decoded Stream 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16 18 (apply repair) Reconstructor Reconstructed 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Stream Effect of Two Frames with Different Video Redundancy Compression Rates 1 1 2 2 3 3 4 Encode Transmit 1 3 4 1 1 3 4 Decode Repetition in the case of consecutive loss - (if I frame, bad news) Two Frames with Different Compression Rates Propagation in the case of I, P frame loss - of quality or previous frame 3
Interleaving Approaches Partial Video Interleaving • Partial-Interleaving approach A1 A2 B1 B2 C1 C2 D1 D2 Original Stream – Use sub-frame as basic interleaving unit A3 A4 B3 B4 C3 C4 D3 D4 + sub-frame consists of macro blocks frame A frame B frame C frame D – (Next slide) • Whole-Interleaving approach A1 B1 A2 B2 A3 B3 A4 B4 – Use whole frame as basic interleaving unit Interleaved Stream • Focus on Whole-Interleaving C1 D1 C2 D2 C3 D3 C4 D4 frame A frame B frame C frame D – (Rest of work) Repetition and Partial Video Whole Interleaving Interleaving 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Original Stream 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16 18 Interleaved Stream I B B P B B P B B I B B P B B P B B Encoded Stream (4 repair pictures here) frame loss during Propagation Loss transmission I I-frame loss Reconstructed 1 3 5 7 9 11 13 15 17 Stream 1 1 3 3 5 5 7 7 9 9 11 11 13 13 15 15 17 17 Apply Repetition Effects of lost I-frame with whole-interleaving /distance=2 Effects of Whole-Interleaving Whole Interleaving Distance Propagation Loss Original Stream (GOP Size = 9) I-frame loss 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18... I Reconstructed 1 3 5 7 9 11 13 15 17 Results of Stream Interleaving Distance = 2 Interleaved Streams Apply Repetition 1 1 3 3 5 5 7 7 9 9 11 11 13 13 15 15 17 17 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16 18... Interleaving Distance = 5 Stream with 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 one I frame loss Results of 1 6 11 16 21 26 31 36 41 2 7 12 17 22 27 32 37 42 … Non-interleaved Streams Apply Repetition 1 2 3 4 5 6 7 8 9 9 9 9 9 9 9 9 9 9 4
A Possible Negative Effect of Outline Whole-Interleaving • Introduction • Background 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16 18 Interleaved Stream • Approach • System Overhead Single Losses • Perceptual Quality 1 2 5 6 7 8 9 10 13 14 15 16 17 18 Reconstructed Stream • Conclusions A Special Case of Single Losses in the Interleaved Stream, distance=2 Redundancy Overhead per Frame MPEG Quality Vs. File Size Redundancy Overhead per Movie Interleaving Overhead Type • About 15% bandwidth overhead Frame Size (Kbytes) 20 • Reason 15 – Intra-frame encoding based on similarity among Primary 10 frames Redundancy 5 – Interleaving 0 + Decrease similarities among consecutive frames n s s m t w o r o i o e + Result in bigger B- and P- frames t c a p N t m i S S i n A Video Clips 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Original Stream 1 3 5 7 9 11 13 15 17 2 4 6 8 10 12 14 16 18 Interleaved Stream 5
Proposed Solution to Bandwidth Outline Overhead • Encode using different MPEG qualities • Introduction • Background – Original stream with MPEG quality 1 • Approach – Stream with added repair with MPEG quality 2 • System Overhead – File size decreases exponentially – Video quality slightly decreased • Perceptual Quality + may be undistinguishable by users • Conclusions Perceptual Quality for User Study for Redundancy Redundancy • We had over 40 users watch 22 video clips • Video are from variety of television shows • A video clip without loss is first displayed • Video clips are of various loss rate and loss pattern. – Loss Rate: 1 10 20 20 20 – Loss Pattern: 1 1 1 2 4 Consecutive Loss and Redundancy User Study for Interleaving • Parameters to be tested – MPEG Quality 1 & 2 – Loss Rate: no loss, 2%, 5%, 10%, 20% – Movie type: hockey game & CNN news + frequency of scene changes and + intensity of object actions – Distance Value: 2 & 5 • Totally 24 movie clips – 20 seconds / clips 6
Perceptual Quality for Interleaving User Study for Interleaving (II) (hockey) 100 Interleaved hockey Clips Non-interleaved hockey Clips • Parameters not to be tested 90 – Frame Rate: 30 frames/sec 80 – Size of movie: 320 x 240 pixels Average Scores 70 – Hardware difference 60 + All tests on one machine 50 + One user each time – Human interaction: one same assistant (me) 40 30 20 5% 10 % 15% 20% 0 Loss Rate in % Percentage (hockey) Perceptual Quality for Interleaving – Perceptual Quality for Interleaving Movie Type (CNN) 100 Interleaved CNN clips Non-interleaved CNN clips 90 16 80 14 12 Average Scores 70 Average Increased Points 10 60 H o c k e y G a m e C l i p s 8 C N N N e w s C l i p s 50 6 4 40 2 30 0 2% 5% 1 0 % 2 0 % 20 0 5% 10% 15% 20% Loss Rate in Percentage Loss Rate in % Pecentage (CNN) Perceptual Quality for Interleaving – Perceptual Quality vs. MPEG Interleaving Distance Quality 1 0 0 80 90 70 80 60 70 50 Non-interleaved Clips 60 Average Score Average Score quality number 1 40 Distance=5 50 quality number 2 40 30 Distance=2 30 20 20 10 10 0 0 5% 2 0 % 0% 1 0 % Loss Rate in Percentage Loss Rate in Percentage (hockey) 7
Recommend
More recommend
