The State of Serverless Computing or, Fixing Dysfunction-as-a-Service Chenggang Wu RISE Lab, UC Berkeley QCon New York 06/24/2019
UC Berkeley: CS 61A Fall 2018 The State of Serverless Computing QCon New York 06/24/2019
The State of Serverless Computing The State of Serverless Computing QCon New York 06/24/2019 QCon New York 06/24/2019
Making Programmers Productive Key Question: Where will code be run? In the ☁ ! The State of Serverless Computing QCon New York 06/24/2019
Background: Serverless Computing The State of Serverless Computing QCon New York 06/24/2019
What is serverless computing? Serverless computing is a programming abstraction that enables users to upload programs, run them at any scale, and pay only for resources used. The State of Serverless Computing QCon New York 06/24/2019
Functions-as-a-Service (FaaS) • AWS Lambda, Google Cloud Functions, OpenWhisk (IBM), Azure Functions, OpenLambda, OpenFaaS, kNative… • Optimized for simplicity – register functions, enable triggers, and scale transparently The State of Serverless Computing QCon New York 06/24/2019
Academic Interest in Serverless The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
Industrial Interest in Serverless The State of Serverless Computing Craft Conference 2019 – Budapest, Hungary
What is FaaS good at today? f() f() f() f() Embarrassingly parallel tasks Workflow orchestration The State of Serverless Computing QCon New York 06/24/2019
Wait! What about… • There are other serverless services, too! • e.g., Google Cloud Dataflow, AWS Athena, Snowflake… • Often referred to as Backend-as-a-Service (BaaS) We’re primarily interested in generality. The State of Serverless Computing QCon New York 06/24/2019
Limitations on FaaS Today Limited execution lifetimes x No inbound network connections IO is a bottleneck No specialized hardware The State of Serverless Computing QCon New York 06/24/2019
But that’s okay: Everything is functional! • Functional programs don’t have side effects or mutable state! • And it is called AWS Lambda The State of Serverless Computing QCon New York 06/24/2019
Dysfunction-as-a-Service • FaaS is not designed for functional programming because real applications share state • FaaS is poorly suited for all of these The State of Serverless Computing QCon New York 06/24/2019
Quantifying The Pain of FaaS How FaaS Disappoints Famous Computer Scientists The State of Serverless Computing QCon New York 06/24/2019
Even Functional Programming is Slow! 1000 Latency (ms) 737 573 285 239 100 10 1 Lambda Lambda Lambda AWS (S3) (Dynamo) (Direct) Step FNs Median and 99 th percentile latencies for composing two arithmetic functions on AWS Lambda. The State of Serverless Computing QCon New York 06/24/2019
Even Functional Programming is Slow! 1000 Latency (ms) 737 573 178 285 239 100 79.6 10 1 Lambda Lambda Lambda AWS (S3) (Dynamo) (Direct) Step FNs Median and 99 th percentile latencies for composing two arithmetic functions on AWS Lambda. The State of Serverless Computing QCon New York 06/24/2019
Even Functional Programming is Slow! 3346 1000 Latency (ms) 737 573 569 178 285 239 100 79.6 10 1 Lambda Lambda Lambda AWS (S3) (Dynamo) (Direct) Step Fns Median and 99 th percentile latencies for composing two arithmetic functions on AWS Lambda. The State of Serverless Computing QCon New York 06/24/2019
Shared Mutable State The State of Serverless Computing QCon New York 06/24/2019
Shared Mutable Storage The State of Serverless Computing QCon New York 06/24/2019
Ideal Serverless Storage Autoscaling Low Latency The State of Serverless Computing QCon New York 06/24/2019
(In)Consistency Guarantees ? Shared Counter The State of Serverless Computing QCon New York 06/24/2019
No Inbound Network Connections Enables Process Easy Fault Migration Tolerance The State of Serverless Computing QCon New York 06/24/2019
Indirect Communication Write Read The State of Serverless Computing QCon New York 06/24/2019
We can fix that! The State of Serverless Computing QCon New York 06/24/2019
A Platform for Stateful Serverless Computing The State of Serverless Computing QCon New York 06/24/2019
Background: Anna • High performance across orders of magnitude in scale ü 10x faster than Redis/Cassandra in a geo-distributed deployment • Autoscaling & cost-efficient ü 500x faster than Amazon DynamoDB for the same cost The State of Serverless Computing Chenggang Wu, Jose Faleiro, Yihan Lin, and Joseph M. Hellerstein. "Anna: A KVS for Any Scale." IEEE Transactions on Knowledge and Data Engineering (2019). QCon New York 06/24/2019 Chenggang Wu, Vikram Sreekanti, and Joseph M. Hellerstein. "Autoscaling Tiered Cloud Storage in Anna." Proceedings of the VLDB Endowment 12, no. 6 (2019): 624-638.
Fluent: FaaS-over-Anna • Maintain disaggregation of compute & state • Make serverless a viable option for stateful applications • Use Anna for both storage and communication Anna The State of Serverless Computing QCon New York 06/24/2019
Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing QCon New York 06/24/2019
Logical disaggregation with physical colocation The State of Serverless Computing QCon New York 06/24/2019
Fluent: FaaS-over-Anna Network Boundary Anna The State of Serverless Computing QCon New York 06/24/2019
Key Idea: Caching • Enable low-latency data access by caching data close to code execution • Communication (and composition) is achieved via a fast-path on top of KVS puts and gets The State of Serverless Computing QCon New York 06/24/2019
Challenge: Cache Consistency • tl;dr: we can provide a variety of coordination-avoiding consistency modes – which is better than S3 or DynamoDB! • This is done by encapsulating program state in lattices The State of Serverless Computing QCon New York 06/24/2019
Lattice • Data structure that accepts incoming update in a way that is associative, commutative, and idempotent (ACI). • Achieves eventual replica convergence The State of Serverless Computing QCon New York 06/24/2019
Causal Consistency • Strongest consistency level that doesn’t require coordination • Causally-related updates will be revealed in an order that respects causality • In addition, guarantee • Repeatable read • Atomic visibility The State of Serverless Computing QCon New York 06/24/2019
Function Composition, Revisited 3346 1000 Latency (ms) 737 573 569 178 285 239 100 79.6 10 6.37 3.59 1 Fluent Lambda Lambda Lambda AWS (S3) (Dynamo) (Direct) Step Fns Median and 99 th percentile latencies for composing two arithmetic functions on AWS Lambda and Fluent. The State of Serverless Computing QCon New York 06/24/2019
Case Study: Prediction Serving The State of Serverless Computing QCon New York 06/24/2019
Prediction Serving • Generate predictions from pretrained machine learning models Model Replica Model Join with Combine Clean Input Reference Data Replica Results Model Replica • At first blush, a great fit for serverless infrastructure The State of Serverless Computing QCon New York 06/24/2019
The State of Serverless Computing QCon New York 06/24/2019
Background: SqueezeNet • State-of-the-art image classification model (developed at Berkeley!) The State of Serverless Computing QCon New York 06/24/2019
Prediction Serving 400 361.7 350 Latency (ms) 300 282.2 250 200 156.3 150 153.8 122.9 100 90.6 50 0 Fluent Python AWS SageMaker Median and 99 th percentile latencies for SqueezeNet on Fluent and AWS SageMaker. The State of Serverless Computing QCon New York 06/24/2019
The Future of Cloud Programming The State of Serverless Computing QCon New York 06/24/2019
Looking Back: Disappointed Computer Scientists • Functional programming is slow • Communication through slow storage • Poor consistency guarantees The State of Serverless Computing QCon New York 06/24/2019
Making FaaS Functional • Embrace state • Easy things become better • Hard things become easy A step on our road towards a programmable cloud. The State of Serverless Computing QCon New York 06/24/2019
Our Vision • Serverless will change the way that we write software and the way that programming infrastructure works • Cloud-native programming models • Enable users to take advantage of millions of cores and petabytes of RAM The State of Serverless Computing QCon New York 06/24/2019
Moving Forward from FaaS Building Developer Tools The State of Serverless Computing QCon New York 06/24/2019
UC Berkeley: CS 61A Fall 2018 The State of Serverless Computing QCon New York 06/24/2019
Moving Forward from FaaS Building Developing Developer Autoscaling Tools Policy The State of Serverless Computing QCon New York 06/24/2019
Moving Forward from FaaS Designing Building Developing Developer Autoscaling SLOs & SLAs Tools Policy The State of Serverless Computing QCon New York 06/24/2019
Recommend
More recommend
