ks q
play

KS & Q Q UE ANKS UESTIONS TIONS Microservices introduce new - PowerPoint PPT Presentation

Aug 24, 2022 •681 likes •1.95k views

Q S IM : E NABLING A CCURATE AND S CALABLE S IMULATION FOR I NTERACTIVE M ICROSERVICES Yanqi Zhang, Yu Gan, Christina Delimitrou Cornell University Session: Datacenters and Cloud Computing, March 26 th M AR TO M IC ARCHI CHING NG TO ICRO

  1. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write 5

  2. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write 5

  3. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write 1 Connection A 2 Connection B 3 5

  4. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write 1 2 Connection A Connection B 3 5

  5. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write Connection A 1 2 Connection B 3 5

  6. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write Connection A 2 1 Connection B 3 5

  7. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write Connection A 1 2 Connection B 3 5

  8. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write Connection A 1 2 Connection B 3 5

  9. M OD ELING I NDIVI AL M IC ODELING IVIDU DUAL ICRO ROSERVIC ERVICES ES  𝝂 qSim microservice model • Multiple stages per microservice • Provided models for common OS queueing utilities (e.g., epoll & socket read) • Each stage optionally coupled with a queue • Multiple execution paths (e.g., Memcached read/write) per microservice  Memcached Mem$ read Socket Socke TCP RX Epoll TCP TX read t Mem$ send write Connection A 1 2 Connection B 3 5

  10. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW 6

  11. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Microservice dependency graph Memcached NGINX MongoDB 6

  12. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Microservice dependency graph Memcached  Dataflow NGINX • Sequence of microservices to execute MongoDB • Differ across request types • Dataflow node encodes blocking/synchronization operation 6

  13. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Microservice dependency graph Memcached  Dataflow NGINX • Sequence of microservices to execute MongoDB • Differ across request types • Dataflow node encodes blocking/synchronization operation  Deployment • Available servers and hardware resources • Service to server mapping • Services on the same server share network stack & disk I/O 6

  14. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW 7

  15. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol 7

  16. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached 7

  17. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection 7

  18. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 2 1 NGINX Memcached Connection B 3 7

  19. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 2 1 NGINX Memcached 3 Connection B 7

  20. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 2 1 NGINX Memcached 3 Connection B 7

  21. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 2 1 NGINX Memcached 3 Connection B 7

  22. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 2 1 NGINX Memcached 3 Connection B 7

  23. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 1 2 NGINX Memcached 3 Connection B 7

  24. M OD ELING D EP CY G RA RAPH & D ATA ODELING EPENDEN ENDENCY ATAFLOW FLOW  Example: 2-tier application & http 1/1.1 protocol • Dependency graph NGINX Memcached NGINX Memcached NGINX • Dataflow Block recv Unblock recv connection connection Connection A 1 2 NGINX Memcached 3 Connection B 7

  25. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write MongoDB Graph NGINX Text Read Memcached UserTag Post MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  26. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write R MongoDB Graph NGINX Text Read Memcached UserTag Post MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  27. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL r Shorten Memcached Posts Unique r Storage MongoDB ID Write r Video Timeline Compose Memcached r Post Image Write MongoDB Graph NGINX r Text Read Memcached r UserTag Post MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  28. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique Storage MongoDB ID Write Video Timeline r r r r Compose r r Memcached Post Image Write MongoDB Graph NGINX Text Read Memcached UserTag Post MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  29. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video r Timeline Compose Memcached Post Image Write r MongoDB Graph NGINX Text Read Memcached UserTag Post MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  30. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write r MongoDB Graph NGINX Text Read Memcached UserTag Post r MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  31. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write r MongoDB Graph NGINX Text R Read Memcached UserTag Post r MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  32. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write r MongoDB Graph NGINX Text Read Memcached UserTag Post r MongoDB Read R Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  33. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video Timeline Compose R Memcached Post Image Write r MongoDB Graph NGINX Text Read Memcached UserTag Post r MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  34. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write r MongoDB Graph NGINX Text Read Memcached UserTag Post r MongoDB Read R Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  35. D EP CY G RA OR C OMPL EX M IC EPENDEN ENDENCY RAPH FOR OMPLEX ICRO ROSERVICES SERVICES  Social network application URL Shorten Memcached Posts Unique r Storage MongoDB ID Write Video Timeline Compose Memcached Post Image Write r MongoDB Graph NGINX Text R Read Memcached UserTag Post r MongoDB Read Timeline Memcached Follow User User Info MongoDB Source: http://www.csl.cornell.edu/~delimitrou/papers/2019.asplos.microservices.pdf 8

  36. S IM IMULATOR ULATOR IN INPUTS UTS 9

  37. S IM IMULATOR ULATOR IN INPUTS UTS  Microservice queueing model • Identifying sources of queueing • Typical queues: network, epoll/kqueue, socket read, disk I/O… • Microservice models reusable (open source community) 9

  38. S IM IMULATOR ULATOR IN INPUTS UTS  Microservice queueing model • Identifying sources of queueing • Typical queues: network, epoll/kqueue, socket read, disk I/O… • Microservice models reusable (open source community)  Processing time distribution • Instrumenting applications and profiling on real servers • Instrumentations reusable 9

  39. S IM IMULATOR ULATOR IN INPUTS UTS  Microservice queueing model • Identifying sources of queueing • Typical queues: network, epoll/kqueue, socket read, disk I/O… • Microservice models reusable (open source community)  Processing time distribution • Instrumenting applications and profiling on real servers • Instrumentations reusable  Microservice dependencies & dataflow paths • Obtained from app developers 9

  40. S IM IMULATOR ULATOR IN INPUTS UTS  Microservice queueing model • Identifying sources of queueing • Typical queues: network, epoll/kqueue, socket read, disk I/O… • Microservice models reusable (open source community)  Processing time distribution • Instrumenting applications and profiling on real servers • Instrumentations reusable  Microservice dependencies & dataflow paths • Obtained from app developers  Server & system resources 9

  41. V AL ALIDATIO IDATION  Validation experiments • Multi-tier microservices: 2/3-tier application • Load balancing & fanout effects • Microservices based on RPC • Comparison with BigHouse  Server platform for trace collection • 10-server cluster • Intel(R) Xeon(R) CPU E5-2660 v3 • 2 sockets, 10 cores/socket, 2 threads/core • Min/max DVFS frequency: 1.2GHz/2.6GHz • Network bandwidth: 1Gbps 10

  42. 2/3-T IER IER A PPLICA ICATION TION  2-tier application NGINX Memcached 11

  43. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached 11

  44. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached 11

  45. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached 11

  46. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached 11

  47. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached NGINX MongoDB 11

  48. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R NGINX MongoDB 11

  49. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application R Memcached NGINX MongoDB 11

  50. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R NGINX MongoDB 11

  51. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R R NGINX MongoDB 11

  52. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application R Memcached R NGINX MongoDB 11

  53. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R R NGINX MongoDB 11

  54. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R NGINX R MongoDB 11

  55. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R R NGINX MongoDB 11

  56. 2/3-T IER IER A PPLICA ICATION TION  2-tier application R NGINX Memcached  3-tier application Memcached R R NGINX MongoDB 11

  57. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  58. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  59. L OA OAD B AL ALAN ANCING CING R Load Balancer R NGINX NGINX NGINX Web Server Web Server Web Server 12

  60. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  61. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  62. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  63. L OA OAD B AL ALAN ANCING CING R Load Balancer R NGINX NGINX NGINX Web Server Web Server Web Server 12

  64. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  65. L OA OAD B AL ALAN ANCING CING R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

  66. L OA OAD B AL ALAN ANCING CING  Linear throughput increase from cluster size of 4 to 8  Sub-linear increase from cluster size of 8 to 16 R R Load Balancer NGINX NGINX NGINX Web Server Web Server Web Server 12

Recommend

More recommend