Dynamic request allocation and scheduling for context aware - PowerPoint PPT Presentation

Dynamic request allocation and scheduling for context aware applications subject to a percentile response time SLA in a distributed cloud Keerthana Boloor ∗ , Rada Chirkova ⋆ ⋄ , Tiia Salo ⋄ and Yannis Viniotis ∗⋄ ∗ Department of Electrical and Computer Engineering ⋆ Department of Computer Science North Carolina State University ⋄ IBM Software Group Research Triangle Park Cloudcom 2010, Indianapolis, Indiana, USA 1 / 17

Agenda Agenda Problem description Dynamic request allocation and scheduling scheme Comparison with static allocation and FIFO/Weighted Round Robin scheduling scheme Conclusion Cloudcom 2010, Indianapolis, Indiana, USA 2 / 17

Problem description Problem description More web applications are designed to be context aware. Most context aware applications are built on SOA principles. Cloud computing systems - the most preferred platform for deployment. Service Level Agreements (SLA) - terms of service and pricing model. What is this presentation about? Cloudcom 2010, Indianapolis, Indiana, USA 3 / 17

Problem description Geographically distributed cloud computing system Geographically distributed cloud computing system Data center hosting K context-aware applications Data center hosting Data center hosting K context-aware K context-aware Clients applications applications Data center hosting K context-aware applications Cloudcom 2010, Indianapolis, Indiana, USA 4 / 17

Problem description Context aware applications SOA based context aware application 2. Client request allocated to and scheduled at end-server 3. Load required 4. Load required service-endpoint Gateway contextdata Contextaware SOA Contextdata applications stores End servers DATA CENTER Internet 1. Client request with Updates to contexts at context-id contextdata stores Cloudcom 2010, Indianapolis, Indiana, USA 5 / 17

Problem description Model of an end-server An end-server serving multiple user classes Class 1 Server ‘j’ at data Class 2 center ‘i’ Class K Each context aware application services multiple classes of users Each user class is guaranteed different quality of service based on economic considerations SLA specifies different service levels and service charges for the different user classes Cloudcom 2010, Indianapolis, Indiana, USA 6 / 17

Problem description Percentile Service Level Agreements Percentile Service Level Agreements Profit P 0 100 X Conformance(%) X % - the fraction of requests of a particular user class which need to have a response time less than r seconds $ P - The profit charged by the cloud, if the percentile of requests that have response time less than r seconds is greater than or equal to X % Cloudcom 2010, Indianapolis, Indiana, USA 7 / 17

Problem description Problem statement Problem statement Allocate and schedule service requests locally at the end-servers so as to globally: � max profit j (1) 1 ≤ j ≤ K where profit j is the profit charged for conformance of the requests from users of class j . Cloudcom 2010, Indianapolis, Indiana, USA 8 / 17

Problem description Problem statement Problem statement Allocate and schedule service requests locally at the end-servers so as to globally: � max profit j (1) 1 ≤ j ≤ K where profit j is the profit charged for conformance of the requests from users of class j . This problem is NP-hard!! Cloudcom 2010, Indianapolis, Indiana, USA 8 / 17

Solution Management scheme description Heuristic-based data-oriented request management scheme Periodic allocation and adaptation at each datacenter. Allocation Allocation Allocation Allocation Allocation phase phase phase phase phase subinterval Adaptation Adaptation Adaptation Adaptation Adaptation phase phase phase phase phase Observation interval (T) Cloudcom 2010, Indianapolis, Indiana, USA 9 / 17

Solution Management scheme description Heuristic-based data-oriented request management scheme Periodic allocation and adaptation at each datacenter. Allocation Allocation Allocation Allocation Allocation phase phase phase phase phase subinterval Adaptation Adaptation Adaptation Adaptation Adaptation phase phase phase phase phase Observation interval (T) Adaptation phase Datacenters exchange conformance levels. Allocation phase Rank-based request allocation and gi-FIFO scheduling. Aim at increasing global profit. Cloudcom 2010, Indianapolis, Indiana, USA 9 / 17

Solution Rank-based allocation and gi-FIFO scheduling Rank-based allocation and gi-FIFO scheduling Profit-score calculation Profit: p k Profit−score assigned to each arriving request of class 1 ($) Class 1 SLA − Profit of 2000$ on conformance of 75% Required global conformance: c k 2000 Current global conformance: cc k 1500 If cc k < c k 1000 Profit-score = p k / ( c k − cc k ) 500 Else 0 Profit-score = 0 0 10 20 30 40 50 60 70 80 90 100 Current conformance of class 1 (%) Cloudcom 2010, Indianapolis, Indiana, USA 10 / 17

Solution Rank-based allocation and gi-FIFO scheduling Rank-based request allocation 1 Query hash-based lookup table ([context-id,machine-id] or [service-id,machine-id]) 2 Rank-based compatibility test 1 The arriving request is assigned a rank based on its profit-score and deadline. 2 Does the arriving request meet its deadline? - Machine compatible!!! 3 Compatible machine not found? - Choose least loaded closest to context DB Cloudcom 2010, Indianapolis, Indiana, USA 11 / 17

Solution Rank-based allocation and gi-FIFO scheduling gi-FIFO scheduling Choose the request of user class with the highest current profit-score Choose one with maximum waiting time but which results in a response time less than or equal to r If no such request exists, choose the request with higher waiting time resulting in a response time greater than r gi-FIFO has been proven to be the most suitable for percentile SLAs for a single server serving multiple classes. Cloudcom 2010, Indianapolis, Indiana, USA 12 / 17

Evaluation Evaluation Dynamic scheme vs static schemes 11000 10000 9000 Dynamic rank based allocation with gi−FIFO scheduling 8000 Static allocation with WRR scheduling 7000 Profit incurred ($) Static allocation with FIFO scheduling 6000 5000 4000 3000 2000 1000 0 5 10 15 20 25 30 35 40 45 50 Request rate Cloudcom 2010, Indianapolis, Indiana, USA 13 / 17

Evaluation Dynamic rank based allocation vs static allocation scheme 11000 Static allocation with gi−FIFO scheduling 10000 Dynamic rank based allocation with gi−FIFO scheduling 9000 8000 Profit incurred ($) 7000 6000 5000 4000 3000 2000 1000 0 0 50 100 150 Request rate Cloudcom 2010, Indianapolis, Indiana, USA 14 / 17

Evaluation Variation in subinterval length Variation in context update interval 18000 18000 16000 16000 Uniform distribution of classes, stringent SLA 14000 Uniform distribution of classes, relaxed SLA 14000 Non−uniform distribution of classes, stringent SLA 12000 Non−uniform distribution of classes, relaxed SLA 12000 Profit obtained ($) Profit obtained($) 10000 10000 Low contextdata load times High contextdata load times 8000 8000 Medium Contextdata load times 6000 6000 4000 4000 2000 2000 0 0 0 50 100 150 200 250 300 350 400 450 500 0 20 40 60 80 100 120 140 160 180 200 Subinterval period Contextdata update interval Cloudcom 2010, Indianapolis, Indiana, USA 15 / 17

Conclusion Conclusion Identified the need for dynamic request scheduling and allocation for context aware applications in a distributed cloud. Proposed a novel rank-based request allocation and gi-FIFO scheduling scheme for managing percentile SLAs with an aim to maximize profit obtained by the cloud. Cloudcom 2010, Indianapolis, Indiana, USA 16 / 17

Questions?? Cloudcom 2010, Indianapolis, Indiana, USA 17 / 17