From EGEE Operations Portal From EGEE Operations Portal towards EGI - PowerPoint PPT Presentation

Enabling Grids for E sciencE Enabling Grids for E-sciencE From EGEE Operations Portal From EGEE Operations Portal towards EGI Operations Portal Hélène Cordier CNRS-IN2P3 IN2P3-CC CNRS-IN2P3, IN2P3-CC ISGC March 9 th -12 th 2010 Taipei, Taiwan www eu egee org www.eu-egee.org COD21 22 Sept 2009 http://cic.gridops.org/ ISGC, March 10th 2010 EGEE-III INFSO-RI-222667

Content Enabling Grids for E-sciencE EGEE Operations portal customers : EGEE actors (site, federation, operators) • and main features of the Operations Portal EGEE Operations Model EGEE Operations Model • Situation at the end of EGEE-III • EGI Operations portal Backend Reorganization: Symfony • Enhancement of standardized access mechanism: Lavoisier Enhancement of standardized access mechanism: Lavoisier • Current and future work • 2/12 EGEE-III INFSO-RI-222667

EGEE Operations Portal features Enabling Grids for E-sciencE RESSOURCE USER : end-user or PROVIDER community Site info VO info Integration Tools Integration Tools Monitoring tools CIC Portal User Support Communication tools OPERATOR OPERATOR OPERATIONS CENTER 3/12 EGEE-III INFSO-RI-222667

EGEE Operations portal services Enabling Grids for E-sciencE Service Availability Monitoring Gstat Production Production Ressource Ressource provider provider Workflows Workflows & & Operations Tools Operations Tools C Communication Communication C i i ti ti VO VO ops ops info info Overview Overview VO ID VO ID Card Card Operator Operator Operator Operator VO life cycle VO life cycle Management Management 4/12 EGEE-III INFSO-RI-222667

EGEE Daily Operations Model Enabling Grids for E-sciencE – Overview of the ressource providers activity: O f SAM based Operations Dashboard created December 2004 – EGEE-I : Implementation centrally driven by 4 federations � Procedures build-up – EGEE-II : Operations driven by 11 federations EGEE II O ti d i b 11 f d ti � Procedures upgrade/ training material � Build-up of QoS: metrics/failover � Build up of QoS: metrics/failover – EGEE-III : decentralization of operations � New model definition – Continuous improvement through quarterly meetings – De facto tight interaction with Sam, Samap, gstat, gocdb, dashboard g , p, g , g , – Training sessions and workhops on procedures and tools � Decentralized operations dashboard based on Nagios relying on � D t li d ti d hb d b d N i l i standard-based Web technology 5/12 EGEE-III INFSO-RI-222667

Implementation of Symfony Framework Enabling Grids for E-sciencE Incidental – Multi developpers environment – Optimisation of development O ti i ti f d l t – Streamlining of code for future customisation of workflows Structural Needs St t l N d – Optimize maintenance time – Integrate my sql for future customisation – Anticipate multi-linguistic feature – Provide packaging for EGI operational entities/ NGI or group of NGIs Advantages: Optimisation of conception, uniform structure for developments, maintenance, pattern automation, multi-OS platform and multi DB compatible. Drawbacks: Heavy initial reorganization workload Drawbacks: Heavy initial reorganization workload 6/12 EGEE-III INFSO-RI-222667

Enhancement of Lavoisier ref. CHEP 2006 Enabling Grids for E-sciencE Data Sources Data Sources Data Views Manager Data Views Manager Service Service Service Data Data Data Data flat flat flat View View flat flat flat flat file flat file file file file file file file adapter adapter triggers: triggers: startup startup notified notified getView getView getView getView Data Data View View SQL SQL RDBMS RDBMS RDBMS adapter adapter triggers: triggers: about to expire about to expire getDataView g g getDataView Data Data WS WS WS View View WS WS adapter adapter Data Data View View XSLT XSLT adapter adapter triggers: triggers: refreshed refreshed processXSL processXSL processXSL config config config config Advantages: Standardisation of data access , Uniformisation of information management Reliability Ease of maintenance management, Reliability, Ease of maintenance Drawbacks: Heavy-weight for single info source 7/12 EGEE-III INFSO-RI-222667

� Dashboard status at EGI start Enabling Grids for E-sciencE Customized views hosted centrally Beta-package for local hosting available Web portal module User Available in production : end of March 2010 p Lavoisier module Database External module data Internal sources data REGIONAL SCENARIOS REGIONAL SCENARIOS (1) Full distributed model regional helpdesk, regional helpdesk, dashboard + DB (2) Partially distributed ( ) y model : dashboard + DB (3)Helpdesk distributed 8/12 EGEE-III INFSO-RI-222667

Current work Enabling Grids for E-sciencE – Data access standardization via common re-engineering of front-end and back-end of the site and operational info d t b databases together with GOCDB team t th ith GOCDB t � Ergonomic global info management � All kind of decentralization scenarios � All kind of decentralization scenarios – Re-design of existing key features of the portal to provide Re design of existing key features of the portal to provide information via standard xml format � VO operational information, life-cycle; EGEE Broadcast – Customization with Operations portal v2.0 : https://operations-portal.in2p3.fr/ Dashboard, Broadcast, Downtimes subscription, VO life- cycle resource provider tools cycle, resource provider tools � Integration of other DCI (GISELA) 9/12 EGEE-III INFSO-RI-222667

Future work Enabling Grids for E-sciencE – Operations Portal within JRA1 activity in EGI-Inspire � OTAG � OTAG – Operations Tools Advisory Group monthly Operations Tools Advisory Group monthly otag@mailman.egi.eu 3rd meeting in Upsalla at the UF 5 3rd meeting in Upsalla at the UF 5 https://www.egi.eu/indico/categoryDisplay.py?categId=4 10/12 EGEE-III INFSO-RI-222667

Need to know more Enabling Grids for E-sciencE Try the dashboard beta-package: https://forge.in2p3.fr/documents/show/76 Open a GGUS ticket: dedicated CIC Portal SU / OTAG SU Contact us: cic-information@in2p3.fr / feedback form C t t i i f ti @i 2 3 f / f db k f 11/12 EGEE-III INFSO-RI-222667

Backup Backup slides slides COD activity Transition to EGI Enabling Grids for E-sciencE OE5 (PL, NL) - OE13 (FI) project-eu-egee-sa1-c-cod- followup@cern.ch p@ - POLE1 : CE, NE Oversight of daily operations Best practices/training - Nagios acceptance criteria/timeline - Model/metrics Model/metrics OTAG: - POLE2 : NE otag@mailman.egi.eu - Ops Manuels, Best practices p , p GOCDB, Operations Portal, GOCDB O ti P t l - Training guides, How-to Dashboard, … - Feedback on NGI readiness - POLE3 : FR POLE3 FR - Failover Procedure for recommendations - Procedure on NGI operations set-up Procedure on NGI operations set up - Forge Forum/Wiki/Documents 12/12 EGEE-III INFSO-RI-222667

Implications of L2 regionalization ( regionalized instance deployed nationally) Enabling Grids for E-sciencE A regionalized tool requires • – interoperation between the national and the central instance interoperation between the national and the central instance (interoperation with non-EGI grids needs to be addressed in any case) – a scalable and reliable transport mechanism, which becomes a critical component p – has a software development/maintenance cost – committment/resources to test and then run it nationally in a reliable manner manner A regionalized tool allows for • – better scalability of the overall model – easier customization easier c stomi ation – use of native language (where relevant e.g. Helpdesk) – persistent storage of national data (e.g. accounting URs) – directly responsible of the service availability – one technical reference solution for all, instead of independent development efforts [ courtesyTF : https://www.egi.eu/indico/contributionDisplay.py?contribId=12&sessionId=3&confId=1 ] Tool Regionalization | Transition meeting, Amsterdam 03 March 2010 13 EGEE-III INFSO-RI-222667