Order two, pick the one that fits! • Salsify’s functional video codec can explore different execution paths without committing to them. • For each frame, codec presents the transport with three options: A slightly-higher-quality version, B K 0 5 A slightly-lower-quality version, better ❌ Discarding the frame. worse 1 0 K B � 60
Salsify’s architecture: Unified control loop transport protocol & video codec � 61
Codec → Transport “Here’s two versions of the current frame.” B K 0 5 r e t t e b w o r s e 2 5 K B target frame size 30 KB � 62
Transport → Codec “I picked option 2. Base the next frame on its exiting state.” 2 5 K B target frame size 30 KB � 63
Codec → Transport “Here’s two versions of the latest frame.” B K 0 5 r e t t e b w o r s e 2 5 K B target frame size 55 KB � 64
Transport → Codec “I picked option 1. Base the next frame on its exiting state.” B K 0 5 target frame size 55 KB � 65
Codec → Transport “Here’s two versions of the latest frame.” B K 0 7 r e t t e b w o r s e 5 2 5 0 K K B B target frame size 5 KB � 66
Transport → Codec “I cannot send any frames right now. Sorry, but discard them.” target frame size 5 KB � 67
Codec → Transport “Fine. Here’s two versions of the latest frame.” 45 KB better worse 20 KB target frame size 50 KB � 68
Transport → Codec “I picked option 1. Base the next frame on its exiting state.” 45 KB target frame size 50 KB � 69
There’s no notion of frame rate or bit rate in the system. Frames are sent when the network can accommodate them.
Loss recovery • Option 1: Ignore dropped packets. • Option 2: Retransmit dropped packets. • Option 3: Restart the video stream with a keyframe or a “key-slice.” � 71
Loss corrupts the current frame... frame corrupted frame � 72
Loss corrupts the current frame... and the rest! frame corrupted frame frame frame � 73
Loss recovery • Option 1: Ignore dropped packets. • Option 2: Retransmit dropped packets. • Option 3: Restart the video stream with a keyframe or a “key-slice.” � 74
Option 5, Salsify’s way: Jump back to the last correct state � 75
Measurement Testbed
Goals for the measurement testbed • A system with reproducible input video and reproducible network traces that runs unmodified version of the system-under-test. • Target QoE metrics: image quality and video delay . � 77
Goals for the measurement testbed • A system with reproducible input video and reproducible network traces that runs unmodified version of the system-under-test. • Target QoE metrics: image quality and video delay . � 78
Goals for the measurement testbed • A system with reproducible input video and reproducible network traces that runs unmodified version of the system-under-test. • Target QoE metrics: image quality and video delay . � 79
� � � � � � � � � � � � � Video delay ! ! ! ! ! ! sender receiver ! time � 80
Sender Receiver Network Simulator Video AV.io System Measurement System
� 82
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
emulated network receiver HDMI output barcoded video video in/out (HDMI) HDMI to USB camera
Sent Image Timestamp: T+0.000s Received Image Timestamp: T+0.765s Quality: 9.76 dB SSIM
Evaluation of Salsify
Evaluation results: Verizon LTE Trace 18 Video Quality (SSIM dB) 16 Skype WebRTC (VP9-SVC) 14 FaceTime WebRTC 12 Better 10 Hangouts 8 7000 5000 2000 1000 700 500 Video Delay (95th percentile ms)
Evaluation results: Verizon LTE Trace 18 Video Quality (SSIM dB) 16 Skype WebRTC (VP9-SVC) 14 FaceTime WebRTC 12 Status Quo (conventional transport 10 and codec) Hangouts 8 7000 5000 2000 1000 700 500 Video Delay (95th percentile ms)
Evaluation results: Verizon LTE Trace 18 Video Quality (SSIM dB) 16 Skype WebRTC (VP9-SVC) 14 FaceTime WebRTC Salsify (conventional codec) 12 Status Quo (conventional transport 10 and codec) Hangouts 8 7000 5000 2000 1000 700 500 Video Delay (95th percentile ms)
Evaluation results: Verizon LTE Trace 18 Salsify Video Quality (SSIM dB) 16 Skype WebRTC (VP9-SVC) 14 FaceTime WebRTC Salsify (conventional codec) 12 Status Quo (conventional transport 10 and codec) Hangouts 8 7000 5000 2000 1000 700 500 Video Delay (95th percentile ms)
Evaluation results: Grace Period 18 Salsify Video Quality (SSIM dB) 16 Skype WebRTC (VP9-SVC) 14 FaceTime WebRTC Salsify (conventional codec) 12 Status Quo (conventional transport 10 and codec) Hangouts Salsify (no grace period) 8 7000 5000 2000 1000 700 500 Video Delay (95th percentile ms)
Evaluation results: AT&T LTE Trace 16 Salsify 15 WebRTC (VP9-SVC) Video Quality (SSIM dB) FaceTime 14 13 WebRTC 12 Hangouts 11 r e t t 10 e B Skype 9 8 5000 2000 1000 700 500 300 200 Video Delay (95th percentile ms)
Evaluation results: T-Mobile UMTS Trace 14 Salsify Video Quality (SSIM dB) 13 WebRTC (VP9-SVC) Skype WebRTC 12 11 FaceTime r e t t e B 10 Hangouts 9 18000 14000 10000 7000 5000 3500 Video Delay (95th percentile ms)
Network Variations Watch the video at: https://snr.stanford.edu/salsify Salsify WebRTC (Google Chrome 65.0 dev)
Network Outages Watch the video at: https://snr.stanford.edu/salsify Salsify WebRTC (Google Chrome 65.0 dev)
Recommend
More recommend
