Power and Bandwidth Optimization in 360-Degree Immersive Mobile Video Streaming Sheng Wei, Assistant Professor Rutgers ECE May 2019
Context: Research Paths Hardware Security . . Mobile Video Systems Assistant Professor Assistant Professor PhD Student Intern/Research Scientist 2008 2010 2015 2018 2 6/25/2019
All about Videos (AR) (VR) Volumetric Immersive 2D Video 3D Video Video Video 3 6/25/2019
The 360-degree Challenges Power Bandwidth 4 6/25/2019
Power Profiling 5 6/25/2019
Power Optimization Solution Key Idea: Reducing the framerate The Problem: Video quality degradation Solution: Reducing framerate during switching only (Courtesy of F. Qian et al.) 6 6/25/2019
Power Optimization Framework: QuRate Video Quality Framerate 7 6/25/2019
Power Optimization Results 6-video, 59-user head movement dataset Frequency of View Switching 8 6/25/2019
Power Optimization Results Power Quality Battery Life 9 6/25/2019
Future Work (x, y, z) Content Distribution Network Edge Node Remote Media Server Hardware Viewport Prediction Edge Computing Acceleration 10 6/25/2019
Viewport Prediction Bandwidth/Power Optimization Opportunity 11 6/25/2019
Problem Definition Scenario: Live 360-degree Video Streaming Problem: Predict the user viewport for a few seconds 12 6/25/2019
Solution Space 13 6/25/2019
Challenges 14 6/25/2019
[Ubicomp 19] LiveMotion: Motion-tracking-based Viewport Prediction Key Idea: User watches moving objects, so let’s track the motion 15 6/25/2019
LiveMotion Results 16 6/25/2019
LiveMotion Limitations Does not work for videos shot by moving cameras Does not work for complicated moving background User may want to watch non-moving objects Solution: Let’s understand the users better 17 6/25/2019
LiveObj: Object-semantics-based viewport prediction Key Idea: user watches meaningful objects 18 6/25/2019
LiveObj Design 19 6/25/2019
LiveObj Results Public head movement dataset involving 48 users watching 10 videos 20 6/25/2019
Future work in viewport prediction Accomplished User watches moving objects → LiveMotion [ubicomp19] In Progress User watches meaningful objects → LiveObj Future Work User watches meaningful actions → LiveROI User watches video content based on audio guide → LiveAudio User/Content-based adaptive viewport algorithm selection LiveMotion LiveRoI ? LiveObj LiveAudio 21 6/25/2019
Future work: Volumetric Video Security Security/Privacy concerns Threat model Bypassing face ID authentication Solution Encryption (too much overhead) Proposed: Opposite use of adversarial attack Volumetric Video 22 6/25/2019
Acknowledgement Project Repositories: gitlab.com/hwsel 23 6/25/2019
Recommend
More recommend
