QoS management in Grid environments Antonella Di Stefano Giovanni - PowerPoint PPT Presentation

Consorzio COMETA - Progetto PI2S2 FESR QoS management in Grid environments Antonella Di Stefano Giovanni Morana Daniele Zito Consorzio Cometa Grid Open Days all’Università di Palermo Palermo, 6-7.12.2007 www.consorzio-cometa.it

Outline • Focus • General Design • QoS management • Advance Reservation  definition and JMS  features  activities • QoS on a Grid middleware • SLA  definition  example • JAM • Composition • Activities Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 2

Focus • Modelling QoS in a gLite platform • Designing suitable protocol to manage different kinds of constraints on suppplying QoS guarantees • Handling issues related to advance reservation Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 3

General Design SLA Management gLite GLOBUS CE CE CE JMS JMS JMS AR Management W W W W W W W W W W W N N N N N N N N N N N The highlighted features are strictly related: managing QoS at collective layer requires low level mechanisms and policies to monitor and guarantee the assignment of a resource to a job (or user) for a specific time slot. Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 4

QoS Management • SLA Definition • Negotiation:  roles  partecipants  protocols • QoS Level:  providing  monitoring • Service Composition Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 5

AR Management • Introduction of scheduling policies  in accordance with the requested QoS • Implementing advance reservation of a resource • Policies and mechanisms to improve the overall resource utilization Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 6

from JMS to Advance Reservation Scheduling Resource Policies Request of Monitor resources Available Job Scheduler Resources Job and their User requests Server Job Dispatcher Resource Allocation and job execution User Job CE AR policies Available AR info Resources AR Request of AR Manager Resource User Job and their Available requests Monitor Resources Server Job Scheduler Request of resources Job Dispatcher Scheduling policies Resource Allocation and job execution CE User Job Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 7

Advance Reservation: features • AR related to a job • A reservation can be modified • Once job is terminated, the resources must become free • Advance reservation of resources fractions (strorage,cpu,etc..) • ACL and priority • Both defined and undefined AR Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 8

Activities (1/2) • Introduction of advance reservation • Introduction of deadline guarantees • Job priorization • Check of cluster load • Scheduling policies • round robin • by_queue • sort_by:  shortest_job_first  longest_job_first  smallest_memory_first  largest_memory_first  high_priority_first  low_priority_first  large_walltime_first  cmp_job_walltime_asc  multi_sort: sort on multiple keys Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 9

Activities (2/2) • Resource bounding • Load balancing through adaptive algorithms • Automatic tuning of host threesholds • Exploit features of multiserver solutions  scalability  fault tolerance • AR profiles Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 10

QoS on a Grid middleware • Handling QoS on Grid systems is very difficult because:  They contain heterogeneous resources  these resources are managed by multi-level manager Matchmaking Flusso IP standard RB parametrico delle risorse Selezione del CE CE SE WN adatto Algoritmi per la gestione delle CE WN WN Selezione del WN WN code nei router WN adatto WN WN Flusso IP “QoS” Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 11

SLA: definition • A Service Level Agreement(SLA) sets the rules and the conditions for proper service fruition. • Every SLA implementation should contain: – technical specifications:  service name  service description, in terms of input/output parameters and availability  participants: provider, consumer and any third party entities to ensure a "trusted" service.  service access mode, i.e. involved protocols and exchanged messages. – Non functional parameters:  service cost  QoS section, i.e. QoS level specific related to the provided service, number of involved parameters and the values/range of single parameters  exceptions/benefits, i.e. corrective actions to be undertaken when the agreements are violated, penalties related to a fault or potential price cost rise in case of advance target reaching. Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 12

SLA example Service name: JamProfiler Service description: • input • output • parameters: • first • second QoS section: • etc responseTime = 375 sec • availability service cost = 2$ Service access mode: • gridFtp Exception section: • https : port 1098 if responseTime < 350 sec • messages: ... then cost = 2.5$ QoS section if 400 < responseTime <= 500 sec Exception section then cost = 1.8$ if 500 < responseTime <= 600 sec then cost = 1$ if responseTime > 600 sec then cost=0$ Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 13

JAM I want reserve A with QoS x GT4 check MDS4 JAM ask to reserve drive WS-GRAM allocate/ reserve JMS: OPENPBS/LSF WN WN WN WN WN Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 14

Infrastructure for Service Composition • Provides users with an overall QoS. • Provides the necessary runtime coordination between (simpler) services and adaptation in case unexpected conditions, such as faults, delays, etc., arise. screenshot Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 15

Activities • Monitoring  tool for the analys of Globus service container through a monitoring of communication between service user and service provider o type and number of requested resources o number and value of specified parameter o time(s) related to Axis engine o total execution time  collects information on the performance variability (in relation with number of available resources and users)  these information could be used for: o performance analys o negotiation strategy • Automatic deployment  tool for the automatic deployment of Globus services since user defines the Java interface Palermo, Grid Open Days all’Università di Palermo, 6-7.12.2007 16