2DFQ: Two-Dimensional Fair Queuing for Multi-Tenant Cloud Services - PowerPoint PPT Presentation

2DFQ: Two-Dimensional Fair Queuing for Multi-Tenant Cloud Services Jona nathan an Ma Mace Peter Bodik Madanlal Musuvathi Brown University Microsoft Microsoft Rodrigo Fonseca Krishnan Varadarajan Brown University Microsoft

Containers / VMs 2

Containers / VMs Shared Systems: Storage, Database, Queueing, etc. 2

Containers / VMs Shared Systems: Storage, Database, Queueing, etc. 2

Shared Process 3

Tenants Shared Process Request Queues … 3

Tenants Shared Process Request Queues Threads time … 3

Tenants Shared Process Request Queues Threads time … 3

Tenants Shared Process Request Queues FIFO or ad-hoc Threads time … 3

Tenants Shared Process Request Queues Threads time … 4

Tenants Packet Flows Shared Process Request Queues Threads time … 4

Tenants Switch Packet Flows Shared Process Request Queues Link Threads time … 4

Tenants Switch Packet Flows Shared Process Request Queues Weighted Fair Queueing (WFQ) Link Threads time … 4

Tenants Shared Process Request Queues Weighted Fair Queueing (WFQ) Threads time … Multiple threads Multiple concurrent requests 4

… 5

… 5

Two-Dimensional Fair Queueing (2DFQ) … 5

6

A size =1 (fast database read) 6

A D size =1 size =10 (fast database read) (slow database scan) 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: time 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: time 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: time 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: … time 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: … time throughput zero-throughput lull high-throughput burst 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: … time throughput zero-throughput lull high-throughput burst Ideal: 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: … time throughput zero-throughput lull high-throughput burst Ideal: 6

A D size =1 size =10 (fast database read) (slow database scan) One thread: … time throughput zero-throughput lull high-throughput burst Ideal: 6

A D size =1 size =10 7

C B A D size =1 size =10 7

C B A D size =1 size =10 time 7

C B A D size =1 size =10 time 7

C B A D size =1 size =10 time 7

C B A D size =1 size =10 time time 7

C B A D size =1 size =10 time time 7

C B A D size =1 size =10 time time 7

C B A D size =1 size =10 time time 7

C B A D size =1 size =10 time throughput time throughput 7

C B A D size =1 size =10 WFQ, SFQ, … time throughput Weighted Fair Queuing Start-time Fair Queueing time throughput 7

C B A D size =1 size =10 WFQ, SFQ, … time throughput Weighted Fair Queuing Start-time Fair Queueing WF²Q time Worst-Case Fair throughput Weighted Fair Queuing 7

C B A D size =1 size =10 8

C B A D size =1 size =10 time 8

C B A D size =1 size =10 time 8

C B A D size =1 size =10 time 8

C B A D size =1 size =10 time 8

C B A D size =1 size =10 time time 8

C B A D size =1 size =10 time time 8

C B A D size =1 size =10 time time 8

C B A D size =1 size =10 time time 8

C B A D size =1 size =10 time Ideal: time 8

C B A D size =1 size =10 time time More threads → Opportunity to reduce burstiness 8

C B A D size =1 size =10 WFQ, SFQ, WF²Q, … Weighted Fair Queueing Start-time Fair Queueing Worst-case Fair Weighted Fair time Queueing time More threads → Opportunity to reduce burstiness 8

C B A D size =1 size =10 WFQ, SFQ, WF²Q, … Weighted Fair Queueing Start-time Fair Queueing Worst-case Fair Weighted Fair time Queueing 2DFQ Two-dimensional Fair Queueing time More threads → Opportunity to reduce burstiness 8

Challenges 9

A size =1 Tenants with small requests are affected 10

A C size =1 size =10 Tenants with small requests are affected Burstiness is proportional to size of large requests 10

A C size =1 size =10 Tenants with small requests are affected Burstiness is proportional to size of large requests Cloud services: 4+ orders of magnitude variation in cost 10

A C size =1 size =10 Tenants with small requests are affected Burstiness is proportional to size of large requests Cloud services: 4+ orders of magnitude variation in cost A C size =1 size =10 Size is used by scheduler to make scheduling decisions 10

A C size =1 size =10 Tenants with small requests are affected Burstiness is proportional to size of large requests Cloud services: 4+ orders of magnitude variation in cost A C size =? size =? Size is used by scheduler to make scheduling decisions 10

A C size =1 size =10 Tenants with small requests are affected Burstiness is proportional to size of large requests Cloud services: 4+ orders of magnitude variation in cost A C size =? size =? Size is used by scheduler to make scheduling decisions Cloud services: Estimation using model or moving averages 10

Key 1 st quartile 3 rd quartile outliers 1 st percentile median 99 th percentile distribution shape 11

Key 1 st quartile 3 rd quartile outliers 1 st percentile median 99 th percentile distribution shape 11

12

Quality Of Service Unpredictability Cost Variation 12

Quality Of Service Unpredictability Cost Variation 12

Quality Of Service Unpredictability Cost Variation 12

Two-Dimensional Fair Queueing 13

C B A D size =1 size =10 WFQ time 14

C B A D size =1 size =10 WFQ time 14

C B A D size =1 size =10 WFQ time 14 0 0 0 0

C B A D size =1 size =10 WFQ time 15 0 0 0 0

C B A D size =1 size =10 WFQ time 15 0 0 0 0

C B A D size =1 size =10 WFQ time 15 0 0 0 0

C B A D size =1 size =10 WFQ time 15 0 0 0 0

C B A D size =1 size =10 WFQ time 16 1 0 0 0

C B A D size =1 size =10 WFQ time 17 1 1 0 0

C B A D size =1 size =10 WFQ time 18 1 1 0 0

C B A D size =1 size =10 WFQ time 18 v(t)= ½ 1 1 0 0

C B A D size =1 size =10 WFQ time 19 v(t)=1 2 2 0 0

C B A D size =1 size =10 WFQ time 20 v(t)=1 ½ 3 3 0 0

C B A D size =1 size =10 WFQ time 21 v(t)=2 4 4 0 0

C B A D size =1 size =10 WFQ time 22 v(t)=2½ 5 5 0 0

C B A D size =1 size =10 WFQ time 23 v(t)=3 6 6 0 0

C B A D size =1 size =10 WFQ time 24 v(t)=3½ 7 7 0 0

C B A D size =1 size =10 WFQ time 25 v(t)=4 8 8 0 0

C B A D size =1 size =10 WFQ time 26 v(t)=4½ 9 9 0 0

C B A D size =1 size =10 WFQ time 27 v(t)=5 10 10 0 0

C B A D size =1 size =10 WFQ time 27 v(t)=5 10 10 0 0

C B A D size =1 size =10 WFQ time 28 v(t)=10 10 10 10 10

C B A D size =1 size =10 WF²Q time 29 v(t)=0 0 0 0 0

C B A D size =1 size =10 WF²Q time 29 v(t)=0 0 0 0 0

C B A D size =1 size =10 WF²Q time 30 v(t)= ½ 1 1 0 0

C B A D size =1 size =10 WF²Q time 30 v(t)= ½ 1 1 0 0

C B A D size =1 size =10 WF²Q time 30 v(t)= ½ 1 1 0 0

C B A D size =1 size =10 WF²Q time 31 v(t)=5½ 1 1 10 10

C B A D size =1 size =10 WF²Q time 32 v(t)=6 2 2 10 10

C B A D size =1 size =10 WF²Q time 33 v(t)=6 ½ 3 3 10 10

C B A D size =1 size =10 WF²Q time 34 v(t)=7 4 4 10 10

C B A D size =1 size =10 WF²Q time 35 v(t)=7 ½ 5 5 10 10

C B A D size =1 size =10 WF²Q time 36 v(t)=8 6 6 10 10