Introduction to gLite Middleware Malik Ehsanullah - PowerPoint PPT Presentation

Introduction to gLite Middleware Malik Ehsanullah (ehsan@barc.gov.in) BARC – Mumbai 1

Introduction • The Grid relies on advanced software, called middleware, which interfaces between resources and the applications • gLite 3.1 based on Scientific Linux 4 • gLite 3.2 based on Scientific Linux 5 2

What gLite does? • What gLite does not do – Somehow my application just run faster – My application can run as long as it needs – Users can access to any resource – Users can rely of a huge amount of software, libraries • What gLite can do – Provides sharing of resources (CPUs, Storage, Sensors …) – Allows the creation of virtual organizations (People, Groups) – Applications will run faster only if properly developed and best if thought for Grid environments (Trivial parallelization, MPIs) – Provides access to computational/storage/other resources accordingly to defined: Policies and Access rights 3

gLite Grid gLite Grid System aims to: Integrate Virtualize Manage RESOURCE s and SERVICE s across different Vo s The gLite middleware is the set of software packages able to do this www.glite.org 4

gLite evolution gLite - Lightweight Middleware for Grid Computing www.glite.org LCG-1 LCG-2 gLite-1 gLite-2 gLite-3 GTK2 Web services based GTK4 5

Grid Scenario • Usage scenario University of Catania VO – Many users of different organizations geographically ditributed (Virtual Oranizations VOs) requesting high Garr-B computational and storage capacities, collaborating each other Italian institute ? VO of Particle ? VO – Many computational Italian CNR Physics resources (computing power and storage) belongs to different institutions but transparently accessible 6

Overview • The user joins to a VO INTERNET 7

Overview • The user joins to a VO • Each VO shares grid resources to other VOs accordingly to several policies. INTERNET 8

Overview • The user joins to a VO • Each VO shares grid resources to other Vos accordingly to several policies. • The Grid middleware allow to use and share: – “Computing Elements (CE)” – “Storage Element (SE)” INTERNET 9

Overview • The users join to a VO • Each VO shares grid resources to other Vos accordingly to several policies. • The Grid middleware allow to use and share: – “Computing Elements (CE)” – “Storage Element (SE)” INTERNET • Plus Additional services to empower the capabilities of the Grid 10

Overview • The users join to a VO • Each VO shares grid resources to other Vos accordingly to several policies. • The Grid middleware allow to use and share: – “Computing Elements (CE)” – “Storage Element (SE)” INTERNET • Plus Additional servicees to empower the capabilities of the Grid • Result: COLLABORATION 11

gLite Main components services • UI: User Interface • WMS: Workload management system • LB: Logging and bookkeeping service • VOMS: Virtual Organization Management service • BDII: Information system • CE: Computing element (LCG/gLite) • WN: Worker nodes • SE: Storage element • LFC: File catalog • MyProxy: User Credential Storage 12

Components • The User Interface (UI) is the user entry point normally considered as the user workstation. It is normally considered as a WMS component. • The Workload Management System (WMS) is a set of services having the responsibility to find the best available computing element where to submit user’s job in a transparent fashion • The Logging and bookkeeping service (LB), keep track of user job execution in terms of statuses: Ready, Scheduled, Waiting, Running, Done • The Computing element (CE) is the computational resource, the entry point to a cluster or PCs handled by a job queue management system; in particular: TORQUE, PBS, LSF, CONDOR • The Worker Nodes are the machines where jobs are really executed and managed by the CE’ queue management system 13

Components • The Information System and Monitoring maintain data related to available grid resources and their health status. • The Virtual Organization Management service (VOMS) , is the way gLite improves the management of authentication and authorization to the Grid resources. The VOMS allows to their own members to define different access rights to VO’ resources • The Storage element (SE) and the File catalogue (LFC), allow to manage Grid files and offer a mechanism to locate them easily for users and jobs. 14

Job life cycle 15

Job Workflow in gLite LFC Catalog Input “sandbox” UI DataSets info JDL Output “sandbox” Resource Broker Expanded JDL Job Submit Event Author. &Authen. Job Query Publish Storage RSL Element Job Submission Job Status Service Logging & Computing Job Status Book-keeping Element 16

Job Workflow in gLite LFC Catalog Input “sandbox” UI DataSets info JDL Output “sandbox” Resource Broker Expanded JDL Job Submit Event Job Query Publish Storage RSL Element Job Submission Job Status Service Job Status 17

gLite services • gLite services can be grouped in 5 main high level set of services Grid Access Security Information system & Monitoring Job Workload Management System Data Management 18

gLite – Grid access Two possibilities: API s or CLI Built on top of them there exist GridPortals and GUIs 19

gLite Security • User authentication is based on X.509 (PKI) ‏  Authorized Certification Authorities (CA) can generate user and service certificates who identify univocally people or Grid services in the whole Grid  Each Grid service may support or not certificates coming from different CAs  To reduce the vulnerabilities the identification of users in to the grid is done through the use of proxy certificates. Proxies are signed ‘copies’ of the original user certificate, having a limited lifetime. • The use of Proxy certificates allows the following:  Delegation: Any grid service can operates on the user behalf making signed copies of the original proxy. (Single Sign On)  Add additional info (Add VO specific information provided by VOMS) ‏  Store a long term proxy on a secure server (MyProxy)  Renewal (A Proxy close to the expiration time can be automatically renewed) ‏ 20

gLite Security: AutH/AhtZ • Authentication – The user receive a certificate from a CA (PKI third party) – He connects to the UI via SSH – He Creates the proxy (single sign on) – All grid services will use this proxy to identify the user. • Authorization – The user has to subscribe to a VO (VOMS) – The VO establishes the user rights – In any Grid service it will be verified if the user belongs to the VO and assigns the proper access rights to the user – A special configuration file named the “gridmapfile”, maintains the correspondency between grid users and resource users (unix pool accounts) 21

VOMS • Virtual Organization Membership Service – Manages many Virtual Organizations (VOs). – Multiple user roles can be defined inside each VO • Extends the X509 schema • Extensions are Digitally Signed – Service maintenance provided by a web front-end – Support MyProxy (stored proxies) – Allow the access rights by VO or by Role – Each Grid site associates to each VO member or role • Allows to implement fine grained security policies to grid resources 22

VOMS Authentication Request VOMS AC C=IT/O=INFN VOMS AC /L=CNAF Auth /CN=Pinco Palla DB /CN=proxy 23

Joining a Virtual Organisation • Users (and machines) are identified by certificates. Obtaining certificate: Annually • Steps CA – User obtains certificate from Certification Authority Joining – User registers at the VO VO manager VO: • usually via a web form Once – VO manager authorizes the user • VO DB updated – User information is replicated onto VO Membership VO Membership VO resources within 24 hours Service Service Replicating VOMS VOMS DB VOMS database once a day database Grid sites User’s identity in the Grid = Subject of certificate: /C=IN/O=DAE/OU=BARC/CN=mvineet 24

MyProxy • MyProxy – Stores a long term proxy certificates to allow the automatic proxy renewal mechanism – Allow to execute jobs requesting a computation time larger that the normal proxy lifetime (normally 12 hrs) • The WMS is the responsible for the proxy renewal • Users should not use long lived proxy directly – Allow the user to access grid resources without carrying out the public and private keys. • Proxy Delegation 25

Information System and Monitoring B erkeley D atabase I nformation I ndex (BDII) The information hierarchically stored via tree modeling (The LDAP implementation of GLUE ) GRIS Stores information at resource level Site BDII Stores information at site level BDII Stores information at VO level BDII (gilda) VO Level Other GIIS GIIS GIIS Site Level (gilda) INFN sez. CT Merida (gilda) Globus MDS Resource Level GRISes GRISes GRISes 26

Workload Management WMS set of middleware components responsible of distribution and management of jobs across Grid resources. Two core components of WMS WM : accepts and satisfy requests for job management. (Matchmaking) is the process of assigning the best available resource. Logging & Bookeeping : keeps track of job execution in term of events: (Submitted, Running, Done,...) 27