Enabling Grids for E-sciencE Experiences integrating new applications in EGEE Roberto Barbera University of Catania and INFN First EGEE User Forum CERN, 01-03.03.2006 www.eu-egee.org INFSO-RI-508833
Outline Enabling Grids for E-sciencE • The mission • The present results • The future First EGEE User Forum, CERN, 01-03.03.2006 2 INFSO-RI-508833
Goals of EGEE Applications Sector Enabling Grids for E-sciencE • Drive the evolution of the grid technology through specific, challenging applications. – pilot applications (HEP and Biomed) committed to use the large distributed infrastructure EGEE to achieve their scientific goals. • Demonstrate that EGEE provides a viable computing infrastructure for research to several scientific communities. – EGEE hosts a number of scientifically diverse applications with the help of teams of engineers funded by EGEE and the scientific communities concerned. First EGEE User Forum, CERN, 01-03.03.2006 3 INFSO-RI-508833
Enabling Grids for E-sciencE Pilot Applications: HEP First EGEE User Forum, CERN, 01-03.03.2006 4 INFSO-RI-508833
How to prepare for LHC: LCG Service Challenges Enabling Grids for E-sciencE • LHC starts in 2007 • Ramp-up with series of service challenges to ensure key services & infrastructure in place for the Apr05 – SC2 Complete Apr05 – SC2 Complete experiments computing systems June05 - Technical Design Report June05 - Technical Design Report • Extremely aggressive timescale • Emphasis on providing a service Jul05 – SC3 Throughput Test Jul05 – SC3 Throughput Test • Data movement • Data handling Sep05 - SC3 Service Phase Sep05 - SC3 Service Phase Dec05 – Tier-1 Network operational Dec05 – Tier-1 Network operational Apr06 – SC4 Throughput Test Apr06 – SC4 Throughput Test May06 –SC4 Service Phase starts May06 –SC4 Service Phase starts Sep06 – Initial LHC Service in stable operation Sep06 – Initial LHC Service in stable operation Apr07 – LHC Service commissioned Apr07 – LHC Service commissioned 2005 2005 2005 2005 2006 2006 2006 2006 2007 2007 2007 2007 2008 2008 2008 2008 SC2 SC2 SC2 SC3 SC3 SC3 First physics First physics First physics cosmics cosmics cosmics First beams First beams First beams SC4 SC4 SC4 Full physics run Full physics run Full physics run LHC Service Operation LHC Service Operation LHC Service Operation preparation preparation preparation setup setup setup service service service First EGEE User Forum, CERN, 01-03.03.2006 5 INFSO-RI-508833
HEP success stories Enabling Grids for E-sciencE DIRAC.Barcelona.es 0.214% DIRAC.Bologna-T2.it 0.696% • DIRAC.CERN.ch 0.571% DIRAC.Cambridge.uk 0.001% LHCb Fundamental activity in CPU used: 6,389,638 h DIRAC.CracowAgu.pl 0.001% DIRAC.IF-UFRJ.br 0.175% DIRAC.LHCBONLINE.ch 0.779% DIRAC.Lyon.fr 2.552% DIRAC.PNPI.ru 0.000% DIRAC.Santiago.es 0.148% preparation of LHC start up Data Output: 77 TB DIRAC.ScotGrid.uk 3.068% DIRAC.Zurich-spz.ch 0.003% DIRAC.Zurich.ch 0.756% LCG.ACAD.bg 0.106% LCG.BHAM-HEP.uk 0.705% LCG.Barcelona.es 0.281% – Physics LCG.Bari.it 1.357% LCG.Bologna.it 0.032% LCG.CERN.ch 10.960% LCG.CESGA.es 0.528% LCG.CGG.fr 0.676% LCG.CNAF-GRIDIT.it 0.012% LCG.CNAF.it 13.196% LCG.CNB.es 0.385% – Computing systems LCG.CPPM.fr 0.242% LCG.CSCS.ch 0.282% LCG.CY01.cy 0.103% LCG.Cagliari.it 0.515% LCG.Cambridge.uk 0.010% LCG.Catania.it 0.551% • LCG.Durham.uk 0.476% LCG.Edinburgh.uk 0.031% Examples: LCG.FZK.de 1.708% LCG.Ferrara.it 0.073% LCG.Firenze.it 1.047% LCG.GR-01.gr 0.349% LCG.GR-02.gr 0.226% LCG.GR-03.gr 0.171% – LHCb: ~700 CPU/years in 2005 LCG.GR-04.gr 0.056% LCG.GRNET.gr 1.170% LCG.HPC2N.se 0.001% LCG.ICI.ro 0.088% LCG.IFCA.es 0.022% LCG.IHEP.su 1.245% on the EGEE infrastructure LCG.IN2P3.fr 4.143% LCG.INTA.es 0.076% LCG.IPP.bg 0.033% LCG.ITEP.ru 0.792% LCG.Imperial.uk 0.891% LCG.Iowa.us 0.287% LCG.JINR.ru 0.472% LCG.KFKI.hu 1.436% – ATLAS: over 10,000 jobs per LCG.Lancashire.uk 6.796% LCG.Legnaro.it 1.569% LCG.Manchester.uk 0.285% LCG.Milano.it 0.770% LCG.Montreal.ca 0.069% LCG.NIKHEF.nl 5.140% day LCG.NSC.se 0.465% LCG.Napoli.it 0.175% LCG.Oxford.uk 1.214% LCG.PIC.es 2.366% LCG.PNPI.ru 0.278% LCG.Padova.it 2.041% � Comprehensive analysis: see LCG.Pisa.it 0.121% LCG.QMUL.uk 6.407% LCG.RAL-HEP.uk 0.938% LCG.RAL.uk 9.518% LCG.RHUL.uk 2.168% LCG.SARA.nl 0.675% S.Campana et al., “Analysis of LCG.Sheffield.uk 0.094% LCG.Torino.it 1.455% LCG.Toronto.ca 0.343% LCG.Triumf.ca 0.105% LCG.UCL-CCC.uk 1.455% LCG.USC.es 1.853% the ATLAS Rome Production t o yea s a ead experience on the EGEE ATLAS Computing Grid“, e-Science 2005, Melbourne, Australia – A lot of activity in all involved applications (including as usual a lot of activity within non-LHC experiments like BaBar, CDF and D0) First EGEE User Forum, CERN, 01-03.03.2006 6 INFSO-RI-508833 20-6-2005, P. Jenni LCG POB: ATLAS on the LCG/EGEE Grid 4
10,000 jobs/day ! Enabling Grids for E-sciencE From Accounting data: � ~3 million jobs in 2005 so far � Sustained daily rates (per month Jan – Nov 2005): [2185, 2796, 7617, 10312, 11151, 9247, 9218, 11445, 10079, 11124, 9491] � ~8.2 M kSI2K.cpu.hours � >1000 cpu years � Real usage is higher as accounting data was not published from all sites until recently First EGEE User Forum, CERN, 01-03.03.2006 7 INFSO-RI-508833
Enabling Grids for E-sciencE Pilot Applications: Biomed First EGEE User Forum, CERN, 01-03.03.2006 8 INFSO-RI-508833
Medical image processing Enabling Grids for E-sciencE • GATE: Radiotherapy planning – CNRS – Monte Carlo simulation – Parallel execution on different seeds • Pharmacokinetics: contrast agent diffusion study – UPV – Medical images registration – Distribution of registration pairs First EGEE User Forum, CERN, 01-03.03.2006 INFSO-RI-508833
Bioinformatics Enabling Grids for E-sciencE • GPS@: bioinformatics portal – http://gpsa.ibcp.fr/ web portal – Existing (but overloaded NPSA portal) – Tens of bioinformatics legacy code – Thousands of potential users – Large input databases • Electron-microscopic image reconstruction – Image filtering and noise reduction – 3D structure analysis First EGEE User Forum, CERN, 01-03.03.2006 INFSO-RI-508833
First biomedical data challenge: World-wide In Silico Docking On Malaria (WISDOM) Enabling Grids for E-sciencE • Significant biological parameters Domain distribution of Flexx run jobs – two different molecular docking com; 1072 bg; 597 applications (Autodock and FlexX) cy; 383 de; 715 – about one million virtual ligands uk; 8106 es; 5122 selected tw; 827 ru; 218 – target proteins from the parasite ro; 337 pl; 1877 fr; 7580 responsible for malaria nl; 3356 gr; 2004 it; 3687 il; 263 • Significant numbers – Total of about 46 million ligands docked in 6 weeks – 1TB of data produced New data challenge in the fall of 2006 – Up to 1000 computers in 15 New malaria targets countries used simultaneously for a total of about 80 CPU years Focus on other neglected diseases Enlarged collaboration • Significant results (possibly including related projects) – Best hits to be re-ranked using Molecular Dynamics First EGEE User Forum, CERN, 01-03.03.2006 11 INFSO-RI-508833
Enabling Grids for E-sciencE Generic Applications First EGEE User Forum, CERN, 01-03.03.2006 12 INFSO-RI-508833
The EGEE Virtuous Cycle Enabling Grids for E-sciencE NA2, NA3, N4 JRA1 NA3, NA4, SA1 SA1 First EGEE User Forum, CERN, 01-03.03.2006 13 INFSO-RI-508833
The birth of a new VO in EGEE Enabling Grids for E-sciencE New community Deployment & configuration Gen. Apps. Quest. & Prop. EGAAP Resource Recommended allocation VO candidate Ask for proposal change MoU NA3/NA4/SA1 (OAG) VO requirements First EGEE User Forum, CERN, 01-03.03.2006 14 INFSO-RI-508833
The MoU’s Enabling Grids for E-sciencE First EGEE User Forum, CERN, 01-03.03.2006 15 INFSO-RI-508833
The status of Generic Applications deployment Enabling Grids for E-sciencE • Applications accepted before Pisa conference – Earth Science Research (Earth Observation, Hydrology, Climate) – Geophysics (Industry) – Computational Chemistry – Astro(particle)-physics (MAGIC and Planck collaborations) – Finance (EGRID) • Applications approved at last EGEE conference in Pisa (October 2005) – Fusion (ITER) – Archaeology – EC Projects (EELA, EUMEDGRID, EUCHINAGRID, BIOINFOGRID) First EGEE User Forum, CERN, 01-03.03.2006 16 INFSO-RI-508833
Computational Chemistry Enabling Grids for E-sciencE Programs SimGate GEMS MPI libraries Apache server EGEE Grid Client side HTTP Server side First EGEE User Forum, CERN, 01-03.03.2006 17 INFSO-RI-508833
Computational Chemistry Enabling Grids for E-sciencE : s u n e y V d o b y n a m r o s f m T e C t Q s y s First EGEE User Forum, CERN, 01-03.03.2006 18 INFSO-RI-508833
Earth Science Research Enabling Grids for E-sciencE Earthquakes’ epicenter determination Ozone maps Climate First EGEE User Forum, CERN, 01-03.03.2006 19 INFSO-RI-508833
FUSION: reactor confinement optimization Enabling Grids for E-sciencE First EGEE User Forum, CERN, 01-03.03.2006 20 INFSO-RI-508833
FUSION: reactor confinement optimization Enabling Grids for E-sciencE First EGEE User Forum, CERN, 01-03.03.2006 21 INFSO-RI-508833
Recommend
More recommend
