Strawman: Queue Tenants By Single Resource Tenant A Tenant B VM VM VM VM VM VM PP Bandwidth limited Request Limited Controller WA W A2 W B2 RR RS FQ W A2 < W B2 out req out req 21
Strawman: Queue Tenants By Single Resource Tenant A Tenant B VM VM VM VM VM VM PP Bandwidth limited Request Limited Controller WA W A2 W B2 bottleneck resource (out bytes) fair share RR RS FQ W A2 < W B2 out req out req 21
Pisces: Queue Tenants By Dominant Resource Tenant A Tenant B VM VM VM VM VM VM PP Bandwidth limited Request Limited Controller WA W A2 W B2 RR RS FQ Track per-tenant W A2 < W B2 resource vector out req out req 22
Pisces: Queue Tenants By Dominant Resource Tenant A Tenant B VM VM VM VM VM VM PP Bandwidth limited Request Limited Controller WA W A2 W B2 dominant resource fair share RR RS FQ Track per-tenant W A2 < W B2 resource vector out req out req 22
Pisces: Queue Tenants By Dominant Resource Tenant A Tenant B VM VM VM VM VM VM PP Bandwidth limited Request Limited Controller WA W A2 W B2 dominant resource fair share RR RS FQ Track per-tenant W A2 < W B2 resource vector out req out req 22
Pisces Mechanisms Solve For Global Fairness PP WA W A2 W B2 System Visibility RS d FQ Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility RS d FQ minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility RS d FQ seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility RR RS ... RR d SS local ... FQ SS microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility RR Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility RR Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility RR Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness PP Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness Maximum bottleneck PP flow weight exchange Controller global WA W A2 W B2 System Visibility fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness Maximum bottleneck PP flow weight exchange Controller global WA W A2 W B2 System Visibility FAST -TCP based replica selection fairness and capacity RR constraints Weight Allocations RS ... RR d SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Pisces Mechanisms Solve For Global Fairness Maximum bottleneck PP flow weight exchange Controller global WA W A2 W B2 System Visibility FAST -TCP based replica selection fairness and capacity RR constraints Weight Allocations RS ... RR DRR token-based d DRFQ scheduler SS local ... Replica Selection FQ SS Policies microseconds seconds minutes Timescale 23
Evaluation 24
Evaluation • Does Pisces achieve (even) system-wide fairness? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? • Does Pisces handle mixed workloads? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? • Does Pisces provide local dominant resource fairness? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? • Does Pisces provide local dominant resource fairness? • Does Pisces handle dynamic demand? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? • Does Pisces provide local dominant resource fairness? • Does Pisces handle dynamic demand? • Does Pisces adapt to changes in object popularity? 24
Evaluation • Does Pisces achieve (even) system-wide fairness? - Is each Pisces mechanism necessary for fairness? - What is the overhead of using Pisces? • Does Pisces handle mixed workloads? • Does Pisces provide weighted system-wide fairness? • Does Pisces provide local dominant resource fairness? • Does Pisces handle dynamic demand? • Does Pisces adapt to changes in object popularity? 25
Pisces Achieves System-wide Per-tenant Fairness Ideal fair share: 110 kreq/s (1kB requests) Unmodified Membase 180 GET Requests (kreq/s) 160 140 120 100 80 60 40 0.57 MMR 20 0 10 20 30 40 50 60 70 80 Time (s) 8 Tenants - 8 Client - 8 Storage Nodes Zipfian object popularity distribution Min-Max Ratio: min rate/max rate (0,1]
Pisces Achieves System-wide Per-tenant Fairness Ideal fair share: 110 kreq/s (1kB requests) Unmodified Membase Pisces 180 180 GET Requests (kreq/s) GET Requests (kreq/s) 160 160 140 140 120 120 100 100 80 80 60 60 40 40 0.57 MMR 0.98 MMR 20 20 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 Time (s) Time (s) 8 Tenants - 8 Client - 8 Storage Nodes Zipfian object popularity distribution Min-Max Ratio: min rate/max rate (0,1]
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase 180 GET Requests (kreq/s) 160 140 120 100 80 60 40 0.57 MMR 20 0 10 20 30 40 50 60 70 80 Time (s) 27
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase 180 GET Requests (kreq/s) 160 140 120 100 80 60 40 0.57 MMR 20 0 10 20 30 40 50 60 70 80 Time (s) 2x vs 1x demand 180 GET Requests (kreq/s) 160 140 120 100 80 60 40 0.36 MMR 20 0 10 20 30 40 50 60 70 80 Time (s) 27
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase FQ 180 180 GET Requests (kreq/s) 160 160 140 140 120 120 100 100 80 80 60 60 40 40 0.57 MMR 0.59 MMR 20 20 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 90 Time (s) 2x vs 1x demand 180 180 GET Requests (kreq/s) 160 160 140 140 120 120 100 100 80 80 60 60 40 40 0.58 MMR 0.36 MMR 20 20 0 0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 Time (s) 27
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase FQ FQ PP 180 180 180 GET Requests (kreq/s) 160 160 160 140 140 140 120 120 120 100 100 100 80 80 80 60 60 60 40 40 40 0.57 MMR 0.59 MMR 0.64 MMR 20 20 20 0 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 Time (s) 2x vs 1x demand 180 180 180 GET Requests (kreq/s) 160 160 160 140 140 140 120 120 120 100 100 100 80 80 80 60 60 60 40 40 40 0.58 MMR 0.74 MMR 0.36 MMR 20 20 20 0 0 0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 Time (s) 27
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase FQ FQ PP WA 180 180 180 GET Requests (kreq/s) 160 160 160 140 140 140 120 120 120 100 100 100 80 80 80 60 60 60 40 40 40 0.57 MMR 0.59 MMR 0.64 MMR 0.93 MMR 20 20 20 0 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 Time (s) 2x vs 1x demand 180 180 180 GET Requests (kreq/s) 160 160 160 140 140 140 120 120 120 100 100 100 80 80 80 60 60 60 40 40 40 0.58 MMR 0.74 MMR 0.96 MMR 0.36 MMR 20 20 20 0 0 0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 Time (s) 27
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase FQ RS FQ PP FQ PP WA 180 180 180 180 GET Requests (kreq/s) 160 160 160 160 140 140 140 140 120 120 120 120 100 100 100 100 80 80 80 80 60 60 60 60 40 40 40 40 0.57 MMR 0.59 MMR 0.64 MMR 0.93 MMR 0.90 MMR 20 20 20 20 0 0 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 Time (s) 2x vs 1x demand 180 180 180 180 GET Requests (kreq/s) 160 160 160 160 140 140 140 140 120 120 120 120 100 100 100 100 80 80 80 80 60 60 60 60 40 40 40 40 0.58 MMR 0.74 MMR 0.96 MMR 0.89 MMR 0.36 MMR 20 20 20 20 0 0 0 0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 Time (s) 27
Each Pisces Mechanism Contributes to System- wide Fairness and Isolation Unmodified Membase FQ RS FQ PP FQ PP WA WA 180 180 180 180 GET Requests (kreq/s) 160 160 160 160 140 140 140 140 120 120 120 120 100 100 100 100 80 80 80 80 60 60 60 60 40 40 40 40 0.57 MMR 0.59 MMR 0.64 MMR 0.93 MMR 0.90 MMR 0.98 MMR 20 20 20 20 0 0 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 Time (s) 2x vs 1x demand 180 180 180 180 GET Requests (kreq/s) 160 160 160 160 140 140 140 140 120 120 120 120 100 100 100 100 80 80 80 80 60 60 60 60 40 40 40 40 0.58 MMR 0.74 MMR 0.96 MMR 0.89 MMR 0.97 MMR 0.36 MMR 20 20 20 20 0 0 0 0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 Time (s) 27
Pisces Imposes Low-overhead Aggregate System Throughput 3500 > 19% GET Requests (kreq/s) 2625 1750 < 5% 875 0 1kB Requests 10B Requests Unmodified Membase Pisces 28
Pisces Achieves System-wide Weighted Fairness 4 heavy hitters 20 moderate demand 40 low demand 29
Pisces Achieves System-wide Weighted Fairness 4 heavy hitters 20 moderate demand 40 low demand 160 GET Requests (kreq/s) 140 120 100 80 100x weight (4) 60 10x weight (20) 40 1x weight (40) 20 0 25 30 35 40 45 50 55 60 Time (s) 29
Pisces Achieves System-wide Weighted Fairness 0.98 MMR 0.89 MMR 0.91 MMR 4 heavy hitters 20 moderate demand 40 low demand 160 GET Requests (kreq/s) 140 120 100 80 100x weight (4) 60 10x weight (20) 40 1x weight (40) 20 0 25 30 35 40 45 50 55 60 Time (s) 29
Pisces Achieves System-wide Weighted Fairness 0.98 MMR 0.89 MMR 0.91 MMR 4 heavy hitters 20 moderate demand 40 low demand 160 GET Requests (kreq/s) 140 120 100 0.91 MMR 80 100x weight (4) 60 10x weight (20) 40 1x weight (40) 20 0 25 30 35 40 45 50 55 60 Time (s) 29
Pisces Achieves System-wide Weighted Fairness 0.98 MMR 0.89 MMR 0.91 MMR 4 heavy hitters 20 moderate demand 40 low demand 160 GET Requests (kreq/s) 140 120 100 0.91 MMR 80 100x weight (4) 60 10x weight (20) 40 1x weight (40) 20 0.56 MMR 0 25 30 35 40 45 50 55 60 Time (s) 29
Pisces Achieves Dominant Resource Fairness 1kB workload 10B workload bandwidth limited request limited 30
Pisces Achieves Dominant Resource Fairness 1kB workload 10B workload bandwidth limited request limited 800 700 Bandwidth (Mb/s) 600 500 400 300 200 1kB bandwidth limited 10B request limited 100 0 20 25 30 35 40 45 50 55 60 Time (s) 30
Pisces Achieves Dominant Resource Fairness 1kB workload 10B workload bandwidth limited request limited GET Requests (kreq/s) 800 300 700 Bandwidth (Mb/s) 250 600 200 500 400 150 300 100 200 1kB bandwidth limited 50 10B request limited 100 0 0 20 25 30 35 40 45 50 55 60 20 25 30 35 40 45 50 55 60 Time (s) Time (s) 30
Pisces Achieves Dominant Resource Fairness 1kB workload 10B workload bandwidth limited request limited GET Requests (kreq/s) 800 76% of bandwidth 76% of request rate 300 700 Bandwidth (Mb/s) 250 600 200 500 400 150 300 100 200 1kB bandwidth limited 50 10B request limited 100 0 0 20 25 30 35 40 45 50 55 60 20 25 30 35 40 45 50 55 60 Time (s) Time (s) 30
Pisces Achieves Dominant Resource Fairness 1kB workload 10B workload bandwidth limited request limited GET Requests (kreq/s) 800 76% of bandwidth 76% of request rate 300 700 Bandwidth (Mb/s) 250 600 200 500 400 150 24% of request rate 300 100 200 1kB bandwidth limited 50 10B request limited 100 0 0 20 25 30 35 40 45 50 55 60 20 25 30 35 40 45 50 55 60 Time (s) Time (s) 30
Pisces Adapts to Dynamic Demand Tenant Demand Constant Diurnal (2x wt) Bursty 31
Pisces Adapts to Dynamic Demand Tenant Demand Constant Diurnal (2x wt) Bursty 200 GET Requests (kreq/s) 150 100 50 0 0 10 20 30 40 50 60 70 80 90 Time (s) 31
Pisces Adapts to Dynamic Demand Tenant Demand Constant Diurnal (2x wt) Bursty 200 GET Requests (kreq/s) 150 ~2x 100 50 0 0 10 20 30 40 50 60 70 80 90 Time (s) 31
Pisces Adapts to Dynamic Demand Tenant Demand Constant Diurnal (2x wt) Bursty 200 GET Requests (kreq/s) even 150 ~2x 100 50 0 0 10 20 30 40 50 60 70 80 90 Time (s) 31
Recommend
More recommend