Bursting with Possibilities An Empirical Study of Credit-Based - PowerPoint PPT Presentation

software evolution & architecture lab Department of Informatics – s.e.a.l. Bursting with Possibilities An Empirical Study of Credit-Based Bursting Cloud Instance Types Dr. Philipp Leitner, Joel Scheuner leitner@ifi.uzh.ch, joel.scheuner@uzh.ch 2015-12-09 Page 1

software evolution & architecture lab Department of Informatics – s.e.a.l. A new Type of Cloud Instances Credit-Based Bursting Instances ➔ Behave fundamentally different than any other existing instance type 2015-12-09 Page 2 Icons from the Noun Project: Rabbit by Hayden Kerrisk, Stopwatch by Nørgaard Andersen, Snail by Jems Mayor

software evolution & architecture lab Department of Informatics – s.e.a.l. Context • Infrastructure-as-a-Service (IaaS) • Virtual Machines (VMs) on a pay-per-use basis • Different performance characteristics CPU Memory I/O 2015-12-09 Page 3 Icons from the Noun Project: CPU by iconsmind.com, ram by Bryn Bodayle, cloud-storage by Matthew Hawdon

software evolution & architecture lab Department of Informatics – s.e.a.l. Credit-Based CPU Bursting Peak Baseline 2015-12-09 Page 4 Icons from the Noun Project: Rabbit by Hayden Kerrisk, Snail by Jems Mayor

software evolution & architecture lab Department of Informatics – s.e.a.l. Bursting Instance Types in Industry “The burstable model has proven to be extremely popular with our customers.” AWS Official Blog Oct 2015 Announced a new instance type in the burstable T2 family “f1-micro machine types offer bursting capabilities that allow instances to use additional physical CPU for short periods of time” 2015-12-09 Page 5

software evolution & architecture lab Department of Informatics – s.e.a.l. Related Work Cloud Benchmarking • S. Ostermann, A. Iosup, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema, “ A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing ,” in Cloud Computing , ser. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering. Springer, 2010, vol. 34, pp. 115–131. • K. R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H. J. Wasserman, and N. J. Wright, “ Performance analysis of high performance computing applications on the amazon web services cloud ,” in Proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science , ser. CLOUDCOM ’10, 2010, pp. 159–168. • A. Iosup, S. Ostermann, N. Yigitbasi, R. Prodan, T. Fahringer, and D. Epema, “ Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing ,” IEEE Transactions on Parallel and Distributed Systems , vol. 22, no. 6, pp. 931–945, Jun. 2011. Burstable Instances • J. Wen, L. Lu, G. Casale, and E. Smirni, “ Less can be More: micro-Managing VMs in Amazon EC2 ,” in Proceedings of the 2015 IEEE International Conference on Cloud Computing (CLOUD’15) , 2015. 2015-12-09 Page 6

software evolution & architecture lab Department of Informatics – s.e.a.l. Credit-Based CPU Bursting – Explained (1) 300 ● ● ● ● ● ● ● ● ● ● ● ● ● ● Execution Time (s) ● ● ● ● 200 10x Baseline Peak ● 100 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 18:10 18:20 18:30 18:40 18:50 19:00 19:10 19:20 19:30 19:40 19:50 20:00 20:10 20:20 20:30 Experiment Duration 30 ● 1 CPU Credit full CPU core performance for 1 minute = ˆ ● CPU Credit Balance ● 20 ● ● 10 ● ● ● 0 ● ● ● ● ● ● ● ● ● ● ● ● 18:10 18:20 18:30 18:40 18:50 19:00 19:10 19:20 19:30 19:40 19:50 20:00 20:10 20:20 20:30 Experiment Duration 2015-12-09 Page 7

software evolution & architecture lab Department of Informatics – s.e.a.l. Credit-Based CPU Bursting – Explained (2) 300 ● ● ● ● ● ● ● ● ● ● ● ● ● ● Execution Time (s) ● ● ● ● 200 10x Baseline Peak ● 100 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 18:10 18:20 18:30 18:40 18:50 19:00 19:10 19:20 19:30 19:40 19:50 20:00 20:10 20:20 20:30 100 Experiment Duration 75 CPU Time (%) CPU Time user 50 steal idle 25 0 18:10 18:20 18:30 18:40 18:50 19:00 19:10 19:20 19:30 19:40 19:50 20:00 20:10 20:20 20:30 Experiment Duration 2015-12-09 Page 8

software evolution & architecture lab Department of Informatics – s.e.a.l. Research Questions 1. How do t2 bursting instance types perform in terms of CPU and IO speed in comparison to other instances? 2. When are t2 bursting instance types more cost-efficient than other instance types? 3. How do t2 instance types perform in comparison to the previous generation ( t1 ) types? 2015-12-09 Page 9 Icons from the Noun Project: CPU by iconsmind.com, cloud-storage by Matthew Hawdon, Dashboard by Björn Andersson, Coins by hunotika, History by Joe Mortell

software evolution & architecture lab Department of Informatics – s.e.a.l. Empirical Study Setup Region Ireland (eu-west-1) All T2 bursting instance types in May 2015 (t2.micro, t2.small, t2.medium) Sysbench measures CPU and I/O performance 1.-15. May 2015 50 data points for each configuration (~1000 in total) Automated execution with Cloud WorkBench (CWB) [1] Benchmark definitions and data publicly available: https://github.com/sealuzh/bursting-cloud-instances [1] Scheuner, Leitner, Cito, Gall: Cloud WorkBench - Infrastructure-as-Code Based Cloud Benchmarking. CloudCom‘14 2015-12-09 Page 10 Icons from the Noun Project: Month by Rohit Arun Rao, Shapes by Chananan, Tool Presets by Fabiano Coelho, Repeat by Dimitry Sunseifer, Gears by Rigo Peter

software evolution & architecture lab Department of Informatics – s.e.a.l. Results – T2 vs. Other Instance Types t2.micro m3.medium m3.large c4.large t1.micro Medium − Instance Equivalents 4 3 2 1 0 t2.micro − Peak t2.micro − Base m3.medium m3.large c4.large t1.micro − Peak Instance Types 2015-12-09 Page 11

software evolution & architecture lab Department of Informatics – s.e.a.l. Results – T2 Bursting Instances t2.micro t2.small t2.medium Medium − Instance Equivalents 4 3 2 1 t2.micro − Peak t2.micro − Base t2.small − Peak t2.small − Base t2.medium − Peak t2.medium − Base Instance Types 2015-12-09 Page 12

software evolution & architecture lab Department of Informatics – s.e.a.l. Results – Performance-Cost Ratio (1) c4.large 32 medium-instance m3.large 23 hours t2.small − Base 15 Instance Types t2.micro − Peak 147 = ˆ t2.micro − Base 15 t2.medium − Base 15 t2.small − Peak 71 t2.medium − Peak 71 medium-instance m3.medium 13 equivalents per 0 50 100 150 USD and hour Performance / Cost Ratio (pcr) 2015-12-09 Page 13 Icons from the Noun Project: 24 hour by iconsmind.com, Ruler by Arthur Shlain, dollar by Simple Icons

software evolution & architecture lab Department of Informatics – s.e.a.l. Results – Performance-Cost Ratio (2) c4.large 32 c4.large 32 m3.large 23 m3.large 23 t2.small − Base 15 Base 15 Instance Types t2.micro − Peak 147 eak 15 t2.micro − Base 15 Base 15 t2.medium − Base 15 Base 15 t2.small − Peak 71 eak 14 t2.medium − Peak 71 eak 14 m3.medium 13 m3.medium 13 0 50 100 150 0 10 20 30 Performance / Cost Ratio (pcr) Full − Utilization Equivalent pcr 2015-12-09 Page 14

software evolution & architecture lab Department of Informatics – s.e.a.l. Usage Scenarios – Low or Irregular Load • Identify the cutoff point for each T2 instance • Where does higher avg. utilization (u) make them less cost efficient • Assumptions: Service is CPU-bound + always requires peak performance 50 Config Utilization − Normalised PCR 40 c4.large m3.large m3.medium 30 t2.medium − Peak t2.micro − Peak t2.small − Peak 20 40 60 80 100 Utilization (%) 2015-12-09 Page 15

software evolution & architecture lab Department of Informatics – s.e.a.l. Usage Scenarios – Boosting Performance-Cost Ratio Idea Exploit initial CPU credit balance on VM startup Implementation Systematically restart VM instances when they run out of CPU credits Effect Improved (utilization normalized) performance cost ratio up to 4x 2015-12-09 Page 16