Real-time Serverless: Enabling Application Performance Guarantee Hai - PowerPoint PPT Presentation

Real-time Serverless: Enabling Application Performance Guarantee Hai Duc Nguyen 1 , Chaojie Zhang 1 , Zhujun Xiao 1 , and Andrew A. Chien 1,2 1 University of Chicago 2 Argonne NaKonal Lab

Serverless has Limitation • Function-as-a-Service (FaaS) aka Serverless is the fastest growing element of cloud workload But • Best-effort invocations • Long-tail latency h"ps://serverless-benchmark.com 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Bursty, Real-.me Applica.ons Computation demand surges when interest events happen ◦ A “wanted” person appears ◦ A cyber attack Timely response to these events Demand Time 4 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Serverless vs. Bursty, Real-time Apps Resource Bursty Serverless invoca-ons are Load best-effort Serverless ❌ No way to guarantee when an invoca-on will run Time 5 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Real-&me Serverless Real-time Serverless (RTS) = Serverless + Guaranteed Invocation Rate Resource guaranteed invoca6on rate Deploy Real-time 10 inv. per sec Serverless Function description: 30 inv. - Maximum Runtime (timeout) 10 inv. - Handler - … - Guaranteed invocation rate ( ! "#$ ): at least 1 Time 1 sec invocation per period of ( %/! "#$ ) seconds. - … 3 sec 6 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Real-&me Serverless Resource Resource Bursty Load Real-5me Bursty Serverless Serverless Load invoca5on Time Time ✓ Timely resource access 7 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Analy&c Model: Video Monitoring Guaranteed Invocation Burst Duration Burst arrival Video Monitoring Rate ( 9 .:; ) Burst Height Response Camera Real-Bme Serverless Framework Frame Value Value is represented as: - ./01230/ 4/567 !"#$%&#'(% = *#+&#'(% ⋅ % 8 Response Delay 8 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

RTS Guarantees Statistics for Frame Value n o i t u b 100 i Guarantee Invocation Rates r t s i D ■ ! "#$ = 0.0 ()*+,-.(+)/0. e u 30 l a V ■ ! "#$ = 0.1 ()*+,-.(+)/0. Percentage of Burst Frames r e t t e ■ ! "#$ = 0.3 ()*+,-.(+)/0. B ■ ! "#$ = 0.9 ()*+,-.(+)/0. 3 ■ ! "#$ = 1.0 ()*+,-.(+)/0. ft = frame-time = 1/30 sec 0.3 ü High guaranteed invoca8on rate à high value 0.03 ü Guarantee Sta8s8cs for 0 0.2 0.4 0.6 0.8 1 Frame Value Normalized Frame Value 9 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Ra#onal Design for Value 0.92 100% 100% 50% of max. value 90% of max. value 70% of max. value 90% 90% 80% 80% Application can adjust Percentage of Burst Frame Percentage of Burst Frame 70% 70% guaranteed invocation rate to Higher is better 60% 60% meet any value target 50% 50% 40% 40% 30% 30% ü Enable applicaJon to engineer 20% 20% the value distribuJon 10% 10% 0.86 0% 0% 0 0 0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 Guarantee InvocaJon Rate (instance per frame-Jme) Guarantee Invocation Rate (instance per frame-time) 10 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

RTS with Burst Interference 1 0.9 ■ Duty factor = 1% 0.8 ■ Duty factor = 10% 0.7 ■ Duty factor = 25% 0.6 Probability Time Burst interference 0.5 0.4 duty factor ↔ burst interference 0.3 0.2 2.5% 0.2% 0.1 ü For realistic bursty applications, 0 the interference probability is low 0 1 2 3 4 5 Burst Interfering 11 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

RTS can support Mul0ple Bursts Bursts can happen simultaneously ü Real-time Serverless can support Higher is multiple bursts be,er ü Approach is simple – just increase the guaranteed invocation rate 12 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Implementation Real-&me serverless interface Working prototype ◦ Compa'ble with serverless ◦ Leverage OpenFaaS ◦ Admission control at func'on registra'on <function name> lang : <Language of function body> handler : <Location of function body> image : <Docker image reference> realtime: <Guaranteed invocation rate> timeout : <Runtime limit> limits : <Maximum resource use> requests : <Minimum resource use> 13 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Case Study: Traffic Monitoring • Traces from real video over Glimpse • Low-level monitor for vehicle presence Rush • Bursts arise when vehicles appears. hours DayAme Night time 14 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Simple Frame Value Model (Success/Fail) Vary guaranteed invocation rate (large background load) 1 Successful Frame Rate ■ ApplicaHon Requests 0.8 ■ Serverless/OpenFaaS ( ! "#$ = 0 ) (normalized) 0.6 ■ Real-Hme Serverless, ! "#$ = 0.3 ■ Real-Hme Serverless, ! "#$ = 1.0 0.4 Serverless cannot respond to demand 0.2 changes ✓ RTS’ guarantee invocation rate enables 0 it to respond to application demand 0 5 10 15 20 despite competition from background load Hour of Day ✓ Higher RTS invocation rate improves for success rate for multiple bursts 15 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Related Work • Traditional Serverless with fast, dynamic invocation • Amazon Lambda, Google Cloud Function, OpenFaaS, Knative, etc. • Minimizing FaaS invocation overhead • SAND (ATC’18), SOCK (ATC’18), Kim et. al. (CLUSTER’18). • Extension for improving FaaS performance • Jonas et. al. (SoCC’17), Hellerstein et. al. (CIDR), Jonas et. al. (Berkeley, 2019) None focus on performance guarantees / real-time. 16 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Summary • Current serverless interface cannot support real-2me, bursty applica2ons. • Real-2me serverless = Serverless + Guaranteed invoca2on rate. Guarantee sta*s*cs for value. • Enable ra*onal design. • • A prototype shows 2mely response for a video monitoring applica2on • Future work Efficient implementa*on for RTS interface • Explore the benefits of RTS interface for other applica*on classes • 17 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Q&A Acknowledgement. This work supported by Na3onal Sci- ence Founda3on Grants CNS-1405959, CMMI-1832230, and CNS- 1901466. We gratefully acknowledge support from Intel, Google, and Samsung. 18 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Backup Slides 19 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

A Big Picture Infrastructure Computa,on Cloud Data Center Edge Providers Applications IT Server Bursty Real-,me Our focus! 20 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Validate Analy,cal Results with Simula,on 22 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Supporting Multiple Applications 23 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Case Study: Sta,s,cs Burst StaLsLcs Glimpse Pipeline Architecture 1 1 Tiffany Yu-Han Chen et. al., Glimpse: Continuous, Real-time Object Recognition on Mobile Devices, SenSys’15 24 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

RTS for Video Analysis ü Guaranteed invoca.on rate enables value guarantee Realistic Workload Synthe.c Workload 25 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

RTS for Video Analysis ü Enable ra+onal design for value guarantee Realis+c Workload Synthetic Workload 26 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Robust against Burst Shape • Fixed total demand per burst • Vary burst dura4on (and height) Higher is better ü Any value are achievable at an appropriate ! "#$ ü Maximum value is achieved at ! "#$ = 1 , regardless burst shape 27 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Robust against Burst Variability Change variability by varying burst duration standard deviation Variability causes value drop Higher duty factor creates more damage ✓ Increase ! "#$ Higher is cancels variability better 0.25 effect 0.35 ✓ RTS value can be 27% maintained for 40% wide burst variance Duty factor = 1% Duty factor = 25% 28 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Mul$ple Real-$me, Bursty Apps. 3000 ✓ RTS resource cost ■ Total demand 2500 Lower is ■ UI allocaEon scales with actual Resource (instances) beJer ■ RTS allocaEon demand 2000 ■ RTS avg. allocaEon ✓ RTS resource 1500 consumption is 2.2x 1000 to 5x lower than UI 500 ✓ RTS helps cloud provider save 0 resource to serve 0 200 400 600 800 1000 1200 0 200 400 600 800 1000 1200 Time (min) Time (min) more applications 10 Apps 100 Apps 29 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

Application Cost: UI vs. RTS Resource cost for maximizing burst • value with different duty factors Lower is Vary RTS vs. UI cost ra@o • better ✓ RTS resource value per unit cost is 16-24x higher than UI ✓ RTS enables low cost solu@ons for real-@me, bursty applica@ons 30 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)

UI Cost at Different Duty Factors 31 12/09/2019 5TH WORKSHOP ON SERVERLESS COMPUTING (WOSC’19)