Response Time-Optimized Distributed Cloud Resource Allocation - PowerPoint PPT Presentation

Response Time-Optimized Distributed Cloud Resource Allocation Matthias Keller Holger Karl Computer Networks Group Universität Paderborn

Minimizing response times • Latency-critical service • Interactive, emergency service request t Response Time answer DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 21

Minimizing response times • Latency-critical service • Interactive, emergency service request t Time to compute Response Time the answer answer DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 22

Minimizing response times request t Queuing System: Time in Queue Response Time + Processing Time Time in System (TIS) answer DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 23

Minimizing response times • Latency-critical service • Interactive, emergency service • Decision: Spend time on RTT or TIS request t Queuing System: Time in Queue Response Time + Processing Time Time in System (TIS) answer DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 24

Minimizing response times • Latency-critical service • Interactive, emergency service • Decision: Spend time on RTT or TIS request Time in System (TIS) Response Time answer DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 25

Example: RTT + TIS • Demand assignment • Facility Location Solution with RTT only 14 RTT solution 12 average response time 10 8 Better 6 4 2 0 0 10 20 30 40 50 arrival rate DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 26

Example: RTT + TIS • Demand assignment • Facility Location Solution with RTT only 14 RTT solution 12 average response time RTT solution with TIS 10 8 Better 6 4 2 0 0 10 20 30 40 50 arrival rate DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 27

Example: RTT + TIS • Demand assignment • Facility Location Solution with RTT only 14 RTT solution 12 average response time RTT solution with TIS 10 8 Better 6 Surprise at 4 runtime 2 0 0 10 20 30 40 50 arrival rate DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 28

Example: RTT + TIS • Demand assignment • Facility Location Solution with RTT only • With RTT + TIS 14 RTT solution 12 average response time RTT solution with TIS 10 RTT+TIS solution 8 Better 6 4 2 0 0 10 20 30 40 50 arrival rate DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 29

Goal Given • Network • Data centres Objective • Minimize response time Means • Allocation of n VMs at data centres DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 30

Goal Given Characterise: • How does response time • Network depend on number n of VMs? • Data centres Objective • Minimize response time Means • Allocation of n VMs at data Response Time centres Optimal Solutions Number of VMs DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 31

Two Approaches Accurate Solution • Mixed Integer Convex Problem • Convex TIS function for each data centre 10 time in system 8 6 4 2 0 0.00 0.30 0.60 0.90 utilization • Tough to solve – slow? DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 32

Two Approaches Accurate Solution Approximate Solution • Mixed Integer Convex Problem • Reformulation: Mixed Integer Linear Problem • Convex TIS function for each • Linearization of TIS function data centre 10 10 original time in system time in system 8 8 uniform 6 6 4 4 2 2 0 0 0.00 0.30 0.60 0.90 0.00 0.30 0.60 0.90 utilization utilization • Accuracy? Speed? • Tough to solve – slow? DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 33

Improve accuracy of linearization • Objective: 10 • Minimize the maximum difference original time in system 8 uniform • Control knobs 6 4 • Number of basepoints 2 • End point at asymptote 0 • Basepoint positions utilization 2.5 difference to org. Better 2.0 1.5 1.0 0.5 0.0 0.00 0.30 0.60 0.90 34 DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation

Improve accuracy of linearization • Objective: 10 • Minimize the maximum difference original time in system 8 uniform • Control knobs 6 imamoto 4 • Number of basepoints 2 • End point at asymptote 0 • Basepoint positions utilization 2.5 difference to org. Better 2.0 • Evaluation in Paper 1.5 1.0 0.5 0.0 0.00 0.30 0.60 0.90 35 DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation

Evaluation of both approaches Convex Problem Linear Problem • Reference Solution • Approximate Solution • Tough to solve – slow? • Accuracy? Speed? Linearization DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 36

Evaluation of both approaches Convex Problem Linear Problem • Reference Solution • Approximate Solution • Tough to solve – slow? • Accuracy? Speed? Linearization Configurations • 6 topologies, 12 – 54 nodes • à 50 random demand realizations • 10 data centre fix DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 37

Evaluation of both approaches Convex Problem Linear Problem • Reference Solution • Approximate Solution • Tough to solve – slow? • Accuracy? Speed? Linearization VM limit: 5 – 10 Configurations • 6 topologies, 12 – 54 nodes • à 50 random demand realizations • 10 data centre fix DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 38

Results – Approximation Ratio approx. ratio = Resp.time Linear Resp.time Convex 1.16 Better 1.14 approximation ratio 1.12 1.10 1.08 1.06 1.04 1.02 1.00 dfn-bwin di-yuan norway atlanta zib54 ta2 topology large small DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 39

Results – Approximation Ratio approx. ratio = Resp.time Linear Resp.time Convex 1.16 Better 1.14 approximation ratio 1.12 1.10 1.08 1.06 1.04 1.02 1.00 dfn-bwin di-yuan norway atlanta zib54 ta2 topology large small DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 40

Results – Runtime Ratio runtime ratio = Runtime Linear Runtime Convex 10 0 Better 10 -1 runtime ratio 10 -2 10 -3 10 -4 dfn-bwin di-yuan atlanta norway zib54 ta2 topology large small DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 41

Results – Runtime Ratio runtime ratio = Runtime Linear Runtime Convex 10 0 Better 10 -1 runtime ratio 10 -2 10 -3 Runtime Linear Runtime Convex 10 -4 2s 0:28h dfn-bwin di-yuan atlanta norway zib54 ta2 7s 1:03h 5s topology 1:15h large small DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 42

Results – Runtime Ratio runtime ratio = Runtime Linear Runtime Convex 10 0 Better 10 -1 runtime ratio 10 -2 10 -3 Runtime Linear Runtime Convex 10 -4 2s 0:28h dfn-bwin di-yuan atlanta norway zib54 ta2 7s 1:03h 5s topology 1:15h large small DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 43

Results – Optimal Solutions • More Resources: • Shorter time in queuing system • VMs at closer data centres 350 RTT 300 Response Time (ms) TIS 250 200 150 100 50 0 5 6 7 8 9 10 5 6 7 8 9 10 5 6 7 8 9 10 di-yuan norway dfn-bwin Resource Limit (#VM) DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 44

Results – Optimal Solutions • More Resources: • Shorter time in queuing system • VMs at closer data centres 350 RTT 300 Response Time (ms) TIS 250 200 150 100 50 0 5 6 7 8 9 10 5 6 7 8 9 10 5 6 7 8 9 10 di-yuan norway dfn-bwin Resource Limit (#VM) DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 45

In the paper… • Convex/Linear Problem Formulation • Facility Location Problem & queuing model • P-median facility location + convex cost function • P-median facility location + piecewise linear cost function • Piecewise Linear Function: Minimize maximal difference • Convexity Proof • Evaluation • Pareto optimal solutions • Compare linear/convex problem • Approx. Ratio • Runtime DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 46

In conclusion… … adjust your latency-sensitive service: • Faster! • Adapt to demand fluctuations swiftly • Accurate! • With queuing delay – no surprises at runtime DCC 2014 Response Time-Optimized Distributed Cloud Resource Allocation 47