Popularity Prediction of Facebook Videos for Higher Quality Streaming ∗ Linpeng Tang ♭ ♭ Qi Huang , Amit Puntambekar ∗ ‡ † ♭ Ymir Vigfusson , Wyatt Lloyd , Kai Li ∗ † ‡ ♭ 1
Videos are Central to Facebook 8 billion views per day 9-year old singing on Black bear roaming Small shop making America’s Got Talent in Princeton frozen yogurt 44M views 3.8K views 122 views 2
Workflow of Videos on Facebook Streaming Video Engine Original Encoded Backend Storage Upload CDN ABR streams the best quality version of the video that fits! Intensive processing needed to create ABR Streaming multiple video versions for ABR streaming 3
Better Video Streaming from More Processing • Better compression at the same quality • QuickFire: 20% size reduction using 20X computation • More users can view the high quality versions Video Quality Bob Alice BaQdwith 4
Better Video Streaming from More Processing • Better compression at the same quality • QuickFire: 20% size reduction using 20X computation • More users can view the high quality versions Video Quality Better Compression Bob Alice BaQdwith 5
How to apply QuickFire for FB videos • Infeasible to encode all videos with QuickFire – Increase by 20X the already large processing fleet • High skew in popularity – Reap most benefit with modest processing? 6
Opportunity: High Skew in Popularity • Access logs of 1 million videos randomly sampled by ID • Watch time: total time users spent watching a video watch time ratio 1.0 Cumulative 0.8 0.6 0.4 0.2 0.0 10 0 10 1 10 2 10 3 10 4 10 5 10 6 Video rank 7
Opportunity: High Skew in Popularity • We can serve most watch time even with a small fraction of videos encoded with QuickFire • Can we predict these videos for more processing? 80%+ watch time watch time ratio 1.0 Cumulative 0.8 0.6 0.4 0.2 0.0 10 0 10 1 10 2 10 3 10 4 10 5 10 6 Video rank 8
CHESS Video Prediction System • Popularity prediction is important for higher quality streaming – Direct encoding on videos with the largest benefit • Goal of CHESS video prediction system – Identify videos with highest future watch time – Maximize watch-time ratio with budgeted processing 9
CHESS Video Prediction System Streaming Video Engine Predicted Popular Videos Backend Storage CHESS-VPS CDN Access logs Social signals Facebook Graph Serving System 10
CHESS Video Prediction System Streaming Video Engine Predicted Popular Videos QuickFire Original Encoded Backend Storage CHESS-VPS CDN Access logs Social signals Facebook Graph Serving System 11
CHESS Video Prediction System Streaming Video Engine Predicted Popular Videos QuickFire Original Encoded Backend Storage CHESS-VPS CDN Access logs Social signals Facebook Graph Serving QuickFire-encoded versions! Serving System 12
Requirements of CHESS-VPS • Handle working set of ~80 million videos • Generate new predictions every few minutes • Requires a new prediction algorithm: CHESS! 13
CHESS Key Insights • Efficiently model influence of past accesses as the basis for scalable prediction • Combine multiple predictors to boost accuracy 14
Efficiently model past access influence • Self exciting process – A past access makes future accesses more probable, i.e. provides some influence on future popularity 8 8 7 7 InIluence InIluence Influence of past accesses 6 6 5 5 4 4 3 3 2 2 1 1 0 0 0 0 1 1 2 2 3 3 4 4 5 5 7ime 7ime 15
Efficiently model past access influence • Self exciting process – A past access makes future accesses more probable, i.e. provides some influence on future popularity – Prediction: sum up total future influence of all past accesses now 8 7 InIluence 6 5 4 3 Total future influence 2 1 0 0 1 2 3 4 5 7ime 16
Efficiently model past access influence • Influence modeled with kernel function • Power-law kernel used by prior works – Provides high accuracy – Scan all past accesses, O(N) time/space not scalable 2.0 3ower Law InIluence 1.5 1.0 0.5 y = ( x + β ) − α 0.0 0 1 2 3 4 5 TLPe 17
Efficiently model past access influence • Influence modeled with kernel function • Power-law kernel used by prior works • Key insight: use exponential kernel for scalability 1.0 y = exp( − x/w ) 3ower Law InIluence 0.8 ExponentLal 0.6 0.4 y = ( x + β ) − α 0.2 0.0 0 1 2 3 4 5 TLPe 18
Efficiently model past access influence • Self exciting process with the exponential kernel ✓ − ( t − u ) ◆ F ( t ) = x ˜ ˜ w + exp F ( u ) w Exponential Previous Current Access + x Decay Prediction Watch-time 19
Efficiently model past access influence • Single exponential kernel is less accurate than power-law kernel – 10% lower watch time ratio • O(1) space/time to maintain Single exponential kernel is less accurate yet scalable 20
Combining Efficient Features in a Model • Key insight: maintain multiple exponential kernels • O(1) space/time Exp Exponential ke kernels ls Actual acces Ac access WaWch Time pa pattern Mod Modeled by Time Combining multiple exponential kernels is as accurate as a power-law kernel 21
Combining Efficient Features in a Model Raw features Past access watch-time likes Multiple Future Popularity Kernels Neural comments Network shares owner likes Directly-used Features video age Social signals further boosts accuracy 22
CHESS Video Prediction System Prediction Aggregated NN Models Access logs workers top videos Streaming Shard 1 Worker 1 Model Aggr Client Shard 2 Worker 2 Shard 3 Worker 3 Model Client Aggr Shard 4 Worker 4 23
Evaluation • What is the accuracy of CHESS? • How do our design decisions on CHESS affect its accuracy and resource consumption? • What is CHESS’s impact on video processing and watch time ratio of QuickFire? 24
Evaluation • What is the accuracy of CHESS? • How do our design decisions on CHESS affect its accuracy and resource consumption? • What is CHESS’s impact on video processing and watch time ratio of QuickFire? 25
Metrics • Watch time ratio – Ratio of watch time from better encoded videos – Directly proportional to benefits of better encoding • Processing time 26
Metrics • Watch time ratio – Ratio of watch time from better encoded videos – Directly proportional to benefits of better encoding • Processing time (infeasible to encode all videos) – Video length processing time ∝ – Video length ratio ≈ computation overhead 27
CHESS is Accurate • Vary video length ratio (proxy for processing overhead) • Observe watch time ratio of better encoded videos WaWch Wime raWio 1.0 0.8 0.6 0.4 0.2 0.0 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Video lengWh raWio 28
CHESS is Accurate • Initial(1d): initial watch time up to 1 day after upload WaWch Wime raWio 1.0 0.8 0.6 0.4 0.2 IniWial(1d) (CAC0'10) 0.0 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Video lengWh raWio 29
CHESS is Accurate • Initial(1d): initial watch time up to 1 day after upload • SESIMIC: handcrafted power-law kernel WDWcK Wime rDWio 1.0 0.8 0.6 0.4 6(I60IC (.'''15) 0.2 IniWiDl(1d) (CAC0'10) 0.0 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 Video lengWK rDWio 30
CHESS is Accurate • Initial(1d): initial watch time up to 1 day after upload • SESIMIC: handcrafted power-law kernel WDWcK WimH rDWio 1.0 0.8 0.6 CH(66 0.4 6(I60IC (.'''15) 0.2 IniWiDl(1d) (CAC0'10) 0.0 CHESS provides higher accuracy than even 10 -6 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 the non-scalable state of the art VidHo lHngWK rDWio 31
CHESS Reduces Encoding Processing • Predict on whole Facebook video workload in real-time • Sample 0.5% videos for actual encoding WaWch 7iPH 5aWiR 0.9 0.8 0.7 0.6 CH(66 0.5 0.4 2wnHU OikHs 0.3 10 20 30 40 50 60 3URcHssing C38 (%) CHESS reduces CPU by 3x (54% to 17%) for 80% watch time ratio 32
Related Work Popularity Prediction Hawkes'71, Crane'08, Szabo'10, Cheng'14, SEISMIC'15 CHESS is scalable and accurate Video QoE Optimization Liu'12, Aaron'15, Huang'15, Jiang'16, QuickFire'16 Optimize encoding with access feedback Caching LFU‘93, LRU’94, SLRU‘94, GDS’97, GDSF‘98, MQ’01 Identify hot items to improve efficiency 33
Conclusion • Popularity prediction can direct encoding for higher quality streaming • CHESS: first scalable and accurate popularity predictor – Model influence of past accesses with O(1) time/space – Combine multiple kernels & social signals to boost accuracy • Evaluation on Facebook video workload – More accurate than non-scalable state of the art method – Serve 80% user watch time with 3x reduction in processing 34
Recommend
More recommend
