BurScale: Using Burstable Instances for Cost-Effective Autoscaling - PowerPoint PPT Presentation

BurScale: Using Burstable Instances for Cost-Effective Autoscaling in the Public Cloud Ata Fatahi , Timothy Zhu, Bhuvan Urgaonkar 1

Problem and Motivation 1 day Wikipedia access trace • Context: Autoscaling in the cloud Load Variability • Problem: Uses expensive regular instances 30 second Wikipedia access trace SALE Short-term • Solution: Burstiness UP TO Use cheaper burstable instances 95% OFF Goal: Cost-effective autoscaling using burstable instances 2

Burstable Instances r • CPU capacity rate-limited by a token (credit) bucket mechanism • Credits accrue at baseline rate up to max bucket size (24x baseline rate) 24 × r • 1 credit = 100% CPU utilization for 1 min = 50% CPU utilization for 2 min • Example: AWS t3.small accrues 24 credits/hour = 0.4 credits/min = 40% baseline CPU utilization Burstable instance = “Fractional” instance with burst capability 3

Burstable Instances r • CPU capacity rate-limited by a token (credit) bucket mechanism • Credits accrue at baseline rate up to max bucket size (24x baseline rate) 24 × r • 1 credit = 100% CPU utilization for 1 min = 50% CPU utilization for 2 min • Example: AWS t3.small accrues 24 credits/hour = 0.4 credits/min = 40% baseline CPU utilization Pros Cons • • Performance is rate limited Cheaper (up to 95%) • Ability to burst • More expensive than regular for performance Burstable instance = “Fractional” instance with burst capability 4

How to Effectively Use Burstable Instances? 1. How many burstable/regular instances to provision? 2. How to avoid running out of credits? 3. How to handle flash crowds? 5

Resource Provisioning • Scaling policy Arrival rate ( λ ) 40 req/s Determines # of instances (k) Load Balancer • What is the minimum # instances? R = λ / µ • k > R for latency SLOs Service rate µ µ µ µ µ µ µ 10 req/s R instances k instances k - R instances • Square Root Staffing Rule Scaling Policy: k = R + c √ R Idea: Use burstable instances for standby variable capacity 6

How to Avoid Running Out of Credits? • Problem: Arrival rate ( λ ) 40 req/s Burstable instances overused à run out of credits Load Balancer Load Balancer Weighted Join the Shortest Queue 1 1 1 1 w w w • Solution: Unbalance the load Service rate µ µ µ µ µ µ µ 10 req/s R instances k - R instances • How to set weight w? k instances Solution: Monitor credits & Dynamically adjust weight to earn credits 7

Flash Crowds m • Flash crowds are unpredictable sudden load increases Provisioning delay • Challenge: Delay in acquiring and Normal warming up new resources Provisioning Overprovisioning ( 𝝁 , R) Flash Crowd • Solution: Provisioning Overprovision capacity ( 𝐧𝝁 , 𝒏𝑺 ) (e.g., Netflix Project Nimble) BurScale ( 𝐧𝝁 , 𝑺 ) Idea: Use burstable instances for standby capacity 8

BurScale Design and Implementation BurScale • Monitor: Scaling Policy • Collects system stats # of instances Controller Metrics • Scaling Policy: Monitor Metrics • Determines cluster size Allocation Metrics Metrics Allocation Request Config • Controller: • Determines # burstable/regular instances Burstable Regular Load Cloud • Allocates/deallocates instances Pool Pool Balancer Watch • Detects flash crowds Amazon EC2 Instances • Adjusts load balancer weights Amazon Web Services 9

Evaluation • Workload: WikiMedia application using Wikipedia access traces • Regular instances: m5.large, 2 vCPUs, $0.096 / hr • Burstable instances: t3.small, 2 vCPUs, $0.0208 / hr • Moderate cluster size ranging from 20 to 70 instances • Comparisons • Reg-Only: Cluster of only regular instances • BurScale: Combines burstable and regular instances 10

Handling Transient Queueing BurScale saves 16.8% in costs 11

Handling Flash Crowds BurScale saves 46.3% in costs 12

Conclusion Arrival rate ( λ ) Load Balancer • Goal: Cost-effective autoscaling using burstable instances Load Balancer Weighted Join the Shortest Queue 1 1 1 1 w w w • Challenge: avoid running out of CPU credits • Solution: BurScale Service rate µ µ µ µ µ µ µ • Selects appropriate number of burstable instances • Dynamically adjusts load balancer weights R instances k - R instances k instances • Results: BurScale saves cost while maintaining performance • Evaluated under web applications, flash crowds, and stateful caches BurScale is open-sourced at: https://github.com/psu-cloud/BurScale 13