German e-Science Initiative 2003 Agenda 2006: The German e-Science - PowerPoint PPT Presentation

GridKa Karlsruhe, September 11 – 15, 2006 Building the German D-Grid *) Wolfgang Gentzsch D-Grid, RENCI, Duke, OGF, PCAST *) funded by the German Ministry for Education and Research 1 September, 2006 Wolfgang Gentzsch, D-Grid

German e-Science Initiative � 2003 ‘Agenda 2006‘: “ The German e-Science Initiative is a strategic task ! ” � Build Sustainable Grid Infrastructure in Germany � Combine existing grid activities and infrastructure � Middleware components driven by Community Grids � Develop e-science services for research community � Science Service Grid � Important: Sustainability � Continuing production grid after the end of the funding period � Integration of new grid communities (2. generation) � Business models for grid services 2 September, 2006 Wolfgang Gentzsch, D-Grid

History of D-Grid Initiative • The 90s: Several Government funded projects for the area of “distributed computing”, like e.g. Unicore,… • 2000++: Many German research centers are part of European grid projects, like DataGrid, GridLab, EuroGrid, CrossGrid, EGEE, GridCoord, CoreGrid, UniGrids, NextGrid,… • 01/2003: Scientists started D-Grid working groups => White Paper for Govt • 03/2004: German Govt (BMBF) announced e-Science Initiative • 08/2004: Call for Grid proposals • 01/2005: “Pre-Project”: Finalists gather in 4-month D-Grid workshops => final review • 09/2005: Start of the D-Grid projects, 20 Mio Euro funding • 06/2006: Deadline D-Grid-2 Call, 20 Mio Euro funding • 01/2007: Start D-Grid-2 3 September, 2006 Wolfgang Gentzsch, D-Grid

D-Grid Pre-Project Workshops February – May 2006 To minimize overlapping of tasks and parallel activities: − Monitoring (von Voigt, Gentzsch) − Portale, User Interfaces (Wehrens, Harms) − Scheduling, Workflow (Schwiegelshohn, Gentzsch) − Storage Management, Archivierung (Ludwig, Reuter) − Metadaten, Datenmanagement (Kasemann, Reuter) − Lizenzmanagement, Accounting, Virtuelle Organisationen (Mickel, Harms) − Security, Privacy (Sax, Gentzsch) => Final Review => 6 Projects 4 September, 2006 Wolfgang Gentzsch, D-Grid

Study on Major Grid Projects • UK-e-Science: Tony Hey, Steven Newhouse, Carole Goble, Malcolm Atkinson, John Darlington, Trevor Cooper Chadwick, Monica Schraefel, Luc Moreau, Paul Watson, Aaron Turner • TeraGrid: Charlie Catlett, Dane Skow • ChinaGrid: Hai Jin • Naregi: Kazushige Saga, Satoshi Matsuoka, Kenichi Miura • EGEE: Bob Jones, Dieter Kranzlmueller • RENCI: Dan Reed and Alan Blatecky 5 September, 2006 Wolfgang Gentzsch, D-Grid

D-Grid Structure Community Grids Grids Community Application Application � � � � � Grid specific Grid specific Developments Development CG CG Middle- Middle- ware ware Information and Knowledge Management Generic Grid Middleware and Grid Services Courtesy Dr. Krahl PT/ BMBF Integration Project Integration Project 6 September, 2006 Wolfgang Gentzsch, D-Grid

D-Grid Structure D-Grid Knowledge Management Im Wissensnetz ONTOVERSE WIKINGER Astro-Grid HEP-Grid MediGrid Textgrid WIN-EM C3-Grid IN-Grid . . . Generic Grid Middleware and Grid Services Integration Project 7 September, 2006 Wolfgang Gentzsch, D-Grid

D-Grid Structure D-Grid Knowledge Management D-Grid-2 Im Wissensnetz Projects ONTOVERSE WIKINGER Astro-Grid HEP-Grid MediGrid Textgrid WIN-EM C3-Grid IN-Grid . . . Generic Grid Middleware and Grid Services Integration Project 8 September, 2006 Wolfgang Gentzsch, D-Grid

German Core Grid Visions • establishment and operation of the Core D-Grid infrastructure by the D-Grid integration project (DGI) • evaluation of interoperability and performance of resources The Core D-Grid comprises: • acquisition, implementation and provision of middleware (Globus, gLite, UNICORE) • attachment of storage robots to the storage element of the D- Grid software (dCache, SRM/SRB) • work for the admission of users to the resources (AA) • operation of a virtual Grid Operating Center (GOC) 9 September, 2006 Wolfgang Gentzsch, D-Grid

D-Grid Middleware User Application Development GAT API GridSphere and Plug-In User Access Nutzer Scheduling Workflow Management High-level Monitoring Grid Services UNICORE Data management LCG/gLite Accounting Billing User/VO-Mngt Basic Grid Services Globus 4.0.1 Security Distributed Network Distributed Resources Compute Infrastructur Data Archive Data/ in D-Grid Resources Software 10 September, 2006 Wolfgang Gentzsch, D-Grid

DGI Infrastructure Project WP 1: D-Grid basic software components, large storage, data interfaces, virtual organizations, management WP 2: Develop, operate and support robust core grid infrastructure, resource description, monitoring, accounting, and billing WP 3: Network (transport protocols, VPN) Security (AAI, CAs, Firewalls) WP 4: Business platform and sustainability, project management, communication and coordination � Scalable, extensible, generic grid platform for future � Longterm, sustainable grid operation, SLAs based September, 2006 Wolfgang Gentzsch, D-Grid

Monitoring, Accounting, Billing Reliable monitoring systems will be developed, error tolerant and scalable, for resource and job information Accounting will answer conceptual, technical and legal questions. • Determination, definition, delivery, allocation of calculation units • customized presentation of resulting fees A comprehensive billing solution for the mutual provision and usage of grid resources will be established and a transparent and comprehensible billing framework for grid-computing is going to be developed. 12 September, 2006 Wolfgang Gentzsch, D-Grid

GOC, German Core Grid sites Site Resource Amount FZJ/ZAM IBM Supercomputer with 8,5 TFlops 32 CPUs STK data robot system with 2,8 PByte 300 TByte FZK/IWR 8 nodes Opteron 2x2.2 GHz 100% 8 processors of a system NEC SX-5 50% 1 p630 with 4 processors 50% RRZN 1 SX-6i to do tests 50% 2 nodes Opteron 2x2.2 GHz to do tests 50% LRZ SGI high performance system with 20 TFlop/s 5% Intel IA32 and IA 64 Cluster, 5% PC² IBM p690, SunFire 80 5% TUD FZJ MPI/RZG IBM supercomputer with 4,5 TFlops, 32 CPUs PC cluster with 2 TFlops 400 TByte Data robot system with 8 PByte RWTH PC² Cluster of 400 Xeon 64 Bit processors, high 10% FHG/ performance visualization and FPGAs ITWM RWTH/RZ 2 SunFire 6900 with 24 UltraSPARC IV each 100% Uni-KA TU- SGI O2K(56 proc)/O3K(192 proc.) : 10% Dresden/ZIH T3E (64 proc): 20% FZK PC cluster with 30 processors, 20% RZG end off 2005: new system with 1000 proc. 2% Uni-H/RRZN PC-Cluster mit 64 CPUs assoc. LRZ Uni-KA PC-Pool assoc. FHG/ITWM assoc. 13 September, 2006 Wolfgang Gentzsch, D-Grid

Requirements for Core Grid Resources 5.3 Mio Euro Investment in D-Grid in October 2006 • Each resource with full middleware stack (GT4, gLite, UNICORE). • Storage systems accessible through DGI tools (dCache, SRB, OGSA-DAI) • All systems should run the existing middleware stack, to reduce migration overhead • Resources should be available for ALL communities (DGrid1 and DGrid2) • Sustainability at least until the end of resource life cycle • Different resources from different grid communities in one location have to be consolidated 14 September, 2006 Wolfgang Gentzsch, D-Grid

Core D-Grid Infrastructure • 8 DGI Centers to build the resource and service backbone for all communities • These centers are responsible for a reliable and sustainable grid operation • These backbone resources are available for all communities exclusively for grid activities • In addition, each community is building a community resource infrastructure with similar quality and services • FZK provides supervision, coordination, monitoring as part of this 3-tier architecture 15 September, 2006 Wolfgang Gentzsch, D-Grid

Installation of a Support Infrastructure Communities DGI D-Grid User Support from both FZK and FZJ common DGUS trouble ticket system or Operation User compatible Community A mngt. support Community A systems Consulting Center Community A Consulting Center Community Consulting Center Consulting Center Deployment-Team Dienste- Dienste- Anbieter Entwickler J Dienste- Anbieter Entwickler J Service Anbieter Developer provider 16 T. Fieseler, FZJ/ZAM, 10.07.2006 Joint EGEE / D-Grid Meeting 16

D-Grid User Support 17 17 Joint EGEE / D-Grid Meeting T. Fieseler, FZJ/ZAM, 10.07.2006

DGI Services, Available Dec 2006 • Sustainable grid operation environment with set of core D-Grid middleware services for all grid communities • Central registration and information management for all (including new) resources • Packaged middleware components for gLite, Globus and Unicore and for data management systems SRB, dCache and OGSA-DAI • D-Grid support infrastructure will support new communities / VOs with installation and integration of new grid resources into D-Grid Help-Desk, Monitoring System and central Information Portal 18 September, 2006 Wolfgang Gentzsch, D-Grid