ENES PRACE Board of Directors March 20th, 2013 Brussels Outline 1. - PowerPoint PPT Presentation

ENES – PRACE Board of Directors March 20th, 2013 Brussels Outline 1. ENES and European Climate models 2. ENES infrastructure strategy 2012-2022 & HPC needs 3. Collaboration with PRACE and ENES

ENES European Network for Earth System modelling http://enes.org A network of European groups in IS-ENES climate/Earth system modeling Infrastructure for ENES Launched in 2001 (MOU) § European projects Ca 50 groups from academic, 2009-2013; 2013-2017 public and industrial world Infrastructure Main focus : Models & their environment discuss strategy Model data (ESGF) to accelerate progress in Interface with HPC ecosystem climate/Earth system modelling and understanding Users : Climate modelling community Several EU projects Impact studies Collaboration with PRACE

State of the art in climate modelling CMIP5 experiments 27 modelling groups 7 in Europe 58 models 1 Canada 5 China / 1 Korea 1 Russia 4 Japan 6 USA 2 Australia

Modelling the Earth’s climate system IPCC (1990) IPCC (1995) IPCC (2001) IPCC (2007) ESMs Basic physical laws Based on Navier-Stokes Conservation of: energy, mass (air, water, carbon) & Parameterisations clouds, radiation, subgrid-scale processes each ESM > 1000 man years: strong legacy

Infrastructure Strategy for the European Earth System Modelling Community 2012-2022 Drivers : Science & Society From understanding to development of “Climate Services” Key science questions Q1. How predictable is climate on a range of timescales ? Q2. What is the sensitivity of climate and how can we reduce uncertainties ? Q3. What is needed to provide reliable predictions of regional climate changes ? Q4. Can we model and understand glacial-interglacial cycles ? Q5. Can we attribute observed signals to understand processes ? Writing team: J. Mitchell, R. Budich, S. Joussaume, B. Lawrence & J. Marotzke 52 contributors from BE, CZ, DE, DK, FI, FR, IT, NO, SE, SP, UK

Q1. How predictable is climate at different time scales ? Multi-model decadal predictions HPC 5 European models, 10 yr simulations at every 5 years Data assimilation With ocean initial conditions Large ensemble runs Resolution Surface air temperature Observations Multi model COMBINE EU Project, courtesy of A. Bellucci (CMCC) 2015 1960

Q2. What is the sensitivity of climate and can we reduce uncertainties ? Feedbacks (eg. clouds, carbon cycle), nonlinear behaviours Temperature change to 2 x CO 2 HPC Ensemble experiments Uncertainty to cloud feedbacks (eg. process studies) Multi-model mean Inter-model CMIP3 (AR4) Clouds 2 to 4.5 °C mean 3°C Dufresne & Bony J. Climate, 2008

Q3. What is needed to provide reliable predictions/projections of regional climate changes ? Summer precipitation 2005 Simulations HPC: global climate model HADGEM3 Spatial Resolution Resolutions 135km à 12km Ensemble runs PRACE UPSCALE project (internal variability, parameterisations) 135 km 60 km 40 km Observations 12 km 25 km 17 km Courtesy of PL Vidale (NCAS) & M. Roberts (MO/HC)

Q4. Can we model and understand glacial-interglacial cycles and better constrain model sensitivity using the past ? Last Glacial Maximum (21 000 years BP) HPC Simulations (PMIP2) Duration (1000 to 100 000 simulated years) Complexity Observed past 600 000 years N 2 0 CO 2 CH 4 T IPCC (2007) Braconnot et al. (Clim Past 2007) ice In thousand of years Source : EPICA community members, Nature 2004

Q5. Can we attribute observed signals to understand processes ? Globally ? Extreme events ? HPC Large ensemble runs For extreme events: Spatial resolution Fast availability of computing power IPCC (2007) observations Simulations Natural & anthropogenic forcings Simulations Natural forcings

Infrastructure Strategy Roadmap Needs for HPC x 3 ≈ x 27 And data storage x 100 ≈ x 10 6 EO, � � n Data Assim. � x 5-10 o i t Complexity � u l Computing � o s Resources � e From Jim Kinter, R the World Modelling Summit, 2008 ensemble: x 10 duration: x 10-100

Infrastructure Strategy Roadmap Challenge: towards 1 km scale global climate models NASA « CORDEX » « CMIP5 »

Infrastructure strategy for ENES for the next 10 years Recommandations: 1) Access to world-class HPC for climate at least «tailored » for climate up to « dedicated » 2) Develop the next generation of climate models 3) Set up data infrastructure (global and regional models) for large range of users from impact community 4) Improve physical network (e.g. link national archives) 5) Strengthen European expertise and networking Input to IS-ENES2 ENES Towards an European Climate Infrastructure Initiative : a sustainable virtual laboratory

ENES & PRACE Collaboration in PRACE IPs: § PRACE 1IP, WP7: § SARA (John Donners): collaboration on EC-EARTH high-resolution Benchmarks: very limited interaction with scientific community PRACE 2IP WP8 : § ENES priorities: coupler, I/O, dynamical cores, fault tolerance only partially followed & limited interactions PRACE 3IP: none yet § Projects on Tier0 machines: § UPSCALE & PULSATION HIRESCLIM & SPRUCE Involvment in PRACE organisation: § Members of SSC: Sylvie Joussaume, Jose Baldasano/Antonio Navarra Member of PRACE User Forum: Pier Luigi Vidale

Feedbacks from PRACE projects Projects on Tier0 machines: § - UPSCALE Pier Luigi Vidale (UK) Hermit - PULSATION Sébastien Masson (FR) Curie - HIRESCLIM Colin Jones (SE) MareNostrum3 - SPRUCE Eric Maisonnave (FR) Curie Preparatory access phase: too short (several components, tests, workflow) (0.5-1 yr) § Even if code ready on similar machine: time needed to adapt on Tier 0 § (e.g. different environment, availability of tools, test experiments) “CPU hours must be used regularly during the one year long project”: impossible Need for IO, data storage and some data analysis easily accessible from Tier0 and § petascale data storage in the network neighbourhood Trained man power is key § Very difficult and limited with 1-year access § Need some long-term planning of machines and use §

World-class HPC for climate « Tailored for climate » Key requirements ( ENES Strategy and EOI for programme access ) § Data intensive science : high- performance data IO and storage e.g. Upscale : 500 TB § Need multi-year access time to scientifically develop and validate the model configuration used (not just porting) § Recognise specificities : § Need of both capability and capacity : several runs in parallel: need to be recognised as massively parallel § Several codes coupled § Environment: support multi-executables, mix MPI/openPM § Workflow: queue system match, long initialisations, large number of files …

Scalability issue Joint Weather and Climate Research Programme A partnership in climate research Scalability tests at resolution 25-30 km for the atmosphere HadGEM3 – CRAY XE6 Hector ARPEGE + simplified ocean ≈ 12 K cores EC-Earth AOGCM [E-W processors] x [N-S processors] x [openMP threads] IPSL Atmosphere P.L. Vidale (NCAS) M. Roberts (MO/HC) CPUs/1000 G. Riley et al., IS-ENES

Scalability issue Revisit dynamical cores And numerics Parallel I/O Parallel coupling Issue: develop Computing /climate collaboration

Contribution of PRACE to international experiments ? Next international experiments : CMIP6/CORDEX6, still under definition § Most experiments to be performed at national level § Some « high-end » experiments to be performed on PRACE ? § High international visibility, coordinated European experiments Additional requirements : § Commitment known in advance § Prepare simulations in advance and then no change of machine Typically : validation phase (2 years) & production phase (2 years) § Strict deadlines for submission of results

Conclusions « The mission of PRACE is to enable high impact scientific discovery and engineering research and development across all disciplines to enhance European competitiveness for the benefit of society. PRACE seeks to realize this mission by offering world class computing and data management resources and services through a peer review process. » From climate community: q Increasing demand from society: more reliable predictions of climate change for adaptation q Strong needs for HPC: Scalability: Increased resolution, Number of experiments, complexity Duration of experiments remains a problem q HPC facilities tailored to our needs : IO, Data storage, physical network Stability of computing environment: development/evaluation/ production runs Strong expectations that PRACE can serve our needs