PARTIES: Q O S-A WARE R ESOURCE P ARTITIONING FOR M ULTIPLE I - PowerPoint PPT Presentation

PARTIES: Q O S-A WARE R ESOURCE P ARTITIONING FOR M ULTIPLE I NTERACTIVE S ERVICES Shuang Chen , Christina Delimitrou, José F. Martínez Cornell University

C OLOCATION OF A PPLICATIONS Best- Latency effort -critical … P P P P Private Private Private Private caches caches caches caches Last-level Cache Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 1 of 15

P RIOR W ORK § Interference during colocation § Scheduling [Nathuji’10, Mars’13, Delimitrou’14] • Avoid co-scheduling of apps that may interfere - May require offline knowledge - Limit colocation options § Resource partitioning [Sanchez’11, Lo’15] • Partition shared resources - At most 1 LC app + multiple best-effort jobs Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 2 of 15

T RENDS IN D ATACENTERS 1 LC + many BE Monolith Best- effort Latency -critical many LC + many BE More LC jobs All have QoS targets Microservices Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 3 of 15

M AIN C ONTRIBUTIONS § Workload characterization • The impact of resource sharing • The effectiveness of resource isolation • Relationship between different resources § PARTIES: First QoS-aware resource manager for colocation of many LC services • Dynamic partitioning of 9 shared resources • No a priori application knowledge • 61% higher throughput under QoS constraints • Adapts to varying load patterns Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 1 of 15

I NTERACTIVE LC A PPLICATIONS Application Memcached Xapian NGINX Moses MongoDB Sphinx Key-value Web Web Real-time Persistent Speech Domain store search server translation database recognition Target QoS 600us 5ms 10ms 15ms 300ms 2.5s Max Load 1,280,000 8,000 560,000 2,800 240 14 User / Sys / 13 / 78 / 0 42 / 23 / 0 20 / 50 / 0 50 / 14 / 0 0.3 / 0.2 / 57 85 / 0.6 / 0 IO CPU% LLC MPKI 0.55 0.03 0.06 10.48 0.01 6.28 Memory 9.3 GB 0.02 GB 1.9 GB 2.5 GB 18 GB 1.4 GB Capacity Memory 0.6 GB/s 0.01 GB/s 0.6 GB/s 26 GB/s 0.03 GB/s 3.1 GB/s Bandwidth Disk 0 MB/s 0 MB/s 0 MB/s 0 MB/s 5 MB/s 0 MB/s Bandwidth Network 3.0 Gbps 0.07 Gbps 6.2 Gbps 0.001 Gbps 0.01 Gbps 0.001 Gbps Bandwidth Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 5 of 15

I NTERACTIVE LC A PPLICATIONS Application Memcached Xapian NGINX Moses MongoDB Sphinx Key-value Web Web Real-time Persistent Speech Domain store search server translation database recognition Target QoS 600us 5ms 10ms 15ms 300ms 2.5s Max Load 1,280,000 8,000 560,000 2,800 240 14 User / Sys / 13 / 78 / 0 42 / 23 / 0 20 / 50 / 0 50 / 14 / 0 0.3 / 0.2 / 57 85 / 0.6 / 0 IO CPU% LLC MPKI 0.55 0.03 0.06 10.48 0.01 6.28 Memory 9.3 GB 0.02 GB 1.9 GB 2.5 GB 18 GB 1.4 GB Capacity Memory 0.6 GB/s 0.01 GB/s 0.6 GB/s 26 GB/s 0.03 GB/s 3.1 GB/s Bandwidth Disk 0 MB/s 0 MB/s 0 MB/s 0 MB/s 5 MB/s 0 MB/s Bandwidth Network 3.0 Gbps 0.07 Gbps 6.2 Gbps 0.001 Gbps 0.01 Gbps 0.001 Gbps Bandwidth Max load: max RPS under QoS target when running alone Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 6 of 15

I NTERFERENCE S TUDY Memcached Xapian NGINX Moses MongoDB Sphinx Hyperthread CPU Power LLC Capacity LLC Bandwidth Memory Bandwidth Memory Capacity Disk Bandwidth Network Bandwidth 0% 100% % of max load under QoS target Extremely sensitive Not sensitive at all • Applications are sensitive to resources with high usage • Applications with strict QoS targets are more sensitive Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 7 of 15

I SOLATION M ECHANISMS • Core mapping » Hyperthreads » Core counts … P P P P • Memory capacity Private Private Private Private • Disk bandwidth caches caches caches caches • Core frequency Last-level Cache » Power • LLC capacity » Cache capacity » Cache bandwidth » Memory bandwidth cgroup ACPI frequency driver • Network bandwidth qdisc Intel CAT Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 8 of 15

R ESOURCE F UNGIBILITY Stand-alone With memory interference Xapian Xapian 1 1 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 3 X X X X X X X X X X X X X X X X X X X X 3 X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 5 X 5 X X X X X X X X X 7 X X X X X X X X 7 X X X X X X 9 X X X X 9 X X X 11 X 11 X 13 13 1 3 5 7 9 1113151719 1 3 5 7 9 1113151719 Cache ways Cache ways § Resources are fungible • More flexibility in resource allocation • Simplifies resource manager Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 9 of 15

PARTIES: D ESIGN P RINCIPLES § PARTIES • PARTitioning for multiple InteractivE Services § Design principles • LC apps are equally important • Allocation should be dynamic and fine-grained • No a priori application knowledge or offline profiling • Recover quickly from incorrect decisions • Migration is used as a last resort Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 10 of 15

PARTIES Client side • 5 knobs organized into 2 wheels Latency • Start from a random resource Monitor • Follow the wheels to visit all resources Poll latency Unallocated pool every 100ms Main Function QoS violations? Upsize! Slack Excess resources? App 1 C Downsize! 20% 0 time $ C C C App 2 $ $ Upsizing App 1… Downsizing App 2… Server side F F $ F $ $ Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 11 of 15

M ETHODOLOGY § Platform: Intel E5-2699 v4 • Single socket with 22 cores (8 IRQ cores) § Virtualization • LXC 2.0.7 § Load generators • Open loop • Request inter-arrival distribution: exponential • Request popularity: Zipfian § Testing strategy • Constant load: 30s warmup, 1m measurement (x5) • Varying load simulates diurnal load patterns Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 12 of 15

C ONSTANT L OADS : M EMCACHED , X APIAN & N GINX Oracle Unmanaged Oracle Max Load of Memcached(%) 100 Max Load of NGINX(%) 10 50 30 20 10 10 80 70 50 40 30 30 20 10 - Offline profiling 80 20 30 20 70 50 40 30 20 10 60 30 10 60 40 30 20 10 - Always finds the 40 10 40 30 20 40 global optimum 50 20 10 20 60 10 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 Heracles Max Load of Xapian(%) Max Load of Xapian(%) Heracles PARTIES - No partitioning Max Load of Memcached(%) 100 Max Load of NGINX(%) 10 60 50 40 30 10 70 60 50 40 30 20 20 10 between BE jobs 80 20 50 40 20 10 70 50 40 20 20 10 60 30 40 30 10 - Suspend BE upon 60 40 30 10 10 40 30 10 30 20 10 40 QoS violation 50 20 10 20 - No interaction 60 10 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 between resources Max Load of Xapian(%) Max Load of Xapian(%) Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 13 of 15

M ORE E VALUATION Constant loads § All 2- and 3-app mixes under PARTIES § Comparison with Heracles for 2- to 6-app mixes Diurnal load pattern § Colocation of Memcached, Xapian and Moses PARTIES overhead § Convergence time for 2- to 6-app mixes Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 14 of 15

C ONCLUSIONS § Need to manage multiple LC apps § Insights • Resource partitioning • Resource fungibility § PARTIES • Partition 9 shared resources • No offline knowledge required • 61% higher throughput under QoS targets • Adapts to varying load patterns Motivation• Characterization• PARTIES• Evaluation • Conclusions Page 15 of 15

PARTIES: Q O S-A WARE R ESOURCE P ARTITIONING FOR M ULTIPLE I NTERACTIVE S ERVICES http://tiny.cc/parties Shuang Chen , Christina Delimitrou, José F. Martínez Cornell University