Regional Climate Modelling using Grid Infrastructures S. K. Dash - PowerPoint PPT Presentation

Regional Climate Modelling using Grid Infrastructures S. K. Dash and Deepika Vaddi Acknowledgements: Stefano Cozzini, Filippo Giorgi, Abdus Salam ICTP Centre for Atmospheric Sciences Indian Institute of Technology Delhi Hauz Khas, New Delhi-110016

Points to Discuss • Importance of Indian summer monsoon & need of regional models • RegCM3 simulations at IIT Delhi • RegCM implemented on Garuda GRID • RegCM4 simulations • Efforts for integrating RegCM4 on ICTP computer using EU GRID

Importance of Monsoon in India • Indian summer monsoon during June September is very important for Indian agriculture and water sources • Monsoon is scientifically challenging • Monsoon features are difficult to be simulated by GCMs primarily because of large temporal and spatial variations • There is a need for high resolution Regional Models

Regional aspects of Indian surface features

Large spatial variations in monsoon rainfall in five homogeneous zones of India. The numbers inside the zones indicate mean monsoon rainfall (mm), standard deviation (mm) and coefficient of variation (%) from top to bottom respectively Dash et al., 2002, Mausam, 53(2), 133-144

1) ICTP RegCM3 simulations at IIT Delhi showing the impact of variations in Tibetan snow on summer monsoon rainfall 2) Two experiments (i) with 10cm of snow And (ii) no snow in the extreme case of global warming

Model domain used in RegCM3 and the five homogeneous zones of India such as North West India (NWI), West Central India (WCI), Central Northeast India (CNI), North East India (NEI) and South Peninsular India (SPI) (Parthasarathy et al., 1995) o N, 80 o E Central Lat and Lon- 20 101 X 115 Points along XY direction o E to 105 o E and 5 o S to 45 o N Domain covers 55 with Grid distance- 55 Km Dash et al., 2006, Theor. Appl. Climatol , special issue, 1-12.

Region over which 10cm of snow has been introduced uniformly in the snow experiment

120 60 60 a c b AI no-snow snow no-snow snow no-snow snow NWI NEI 50 50 100 80 40 40 60 30 30 40 20 20 20 10 10 0 0 0 1993 1994 1995 1996 average 1993 1994 1995 1996 average 1993 1994 1995 1996 average 60 60 60 f d e no-snow snow no-snow snow no-snow snow WCI CNI 50 SPI 50 50 40 40 40 30 30 30 20 20 20 10 10 10 0 0 0 1993 1994 1995 1996 average 1993 1994 1995 1996 average 1993 1994 1995 1996 average Comparison of JJAS mean rainfall (cm) over All India and its five homogeneous zones as simulated by RegCM3 in no-snow and snow experiments Decrease of rainfall 30% for AI 20% for WCI 25% for CNI 23% for NWI 15% for SPI

Simulation of Monsoons 1982-2009 Using ICTP RegCM3 Initial Conditions: 25 th April to 3 rd May up to 30 th September, 9-member Horizontal grid distance: 55 Km Domain chosen: 51 O E to 109 O E and 3 O S to 43 O N Data used: USGS Global 30 Arc-Sec. elevation datasets at 30 ’ resolution to create terrain USGS Global GLCC dataset at 30 ’ resolution to create vegetation or landuse file Weekly analysis OISST available from NOAA for integration NCEP Reanalysis (NNRP1) are used for setting the initial and boundary conditions

• Variations in rainfall, temperature and winds at time scales of intra-seasonal, seasonal and inter-annual examined in detail and compared with observations. • Special emphasis given on contrasting monsoon years such as 1982-1983, 1987- 1988, 2002-2003 etc to validate RegCM.

Clusters on Garuda ( being used ) Garuda Head Node (gridfs) GG-BLR Begabung GG-HYD (Linux Xeon Cluster (Linux Xeon machine Head Node at Bangalore) (IITD) Hyderabad)

Resources used Cluster OS ARCH Memory Job Conf #procs Name Manag Procs er Ajaymeru Linux2. x86_6 32 GB - 128 128 ( CAS, IITD ) 6.18- 4 128. gg-blr.tfg Linux2. x86_6 16 GB PBS 240 208 ( Garuda ) 6.18- 4 53. begabung Linux2. x86_6 4 GB PBS 36 17 ( Garuda 6.9- 4 cluster, 42.0 IITD )

Time Evaluation Seasonal Runs #processors Execution Execution Gain on the grid Time(gg-blr) Time(Ajaymeru) 12 processors 1:35:40 2:35:42 1:00:02 24 processors 1: 10:49 2:02:54 0:52:05 30 processors 0:54:12 1:49:23 0:55:11 40 processors 0:46:36 1:20:26 0:33:50 60 processors 0:41:02 1:05:47 0:24:45

Global Data Pre-Processing Model Execution Post Processing & Visualisation GLCC dataset RegCM Terrain GISST, OISST, OIWK CDO GTOPO ATM SST (landuse) RAD NCO RegcmMPI Dataset SRF Global re- ICBC Grads analysis data EIN15, ERA40, NNRP1, NNRP2 Represents the different components of RegCM modelling system .

HPC local resources resources 1. Pre- Processing Data for Pre- Storage Grid processing 3. Upload Input data 5.Download input data 7. Upload output data 2. Move Input data to UI 6. RegCM execution 8. Download Final output GRID Computing Element User Interface 4. JDL Job submission Schematic diagram of interaction of model simulation with data management within the Grid Infrastructure

Clusters on Garuda used for Demo Garuda Head Node(GHN) Gridfs Cluster Head Cluster Head Cluster Head Node(CHN) Node(CHN) Node(CHN) Hyderabad Bangalore IIT Delhi gg-hyd (CDAC) gg-blr Begabung connecting Compute nodes Compute Garuda(via nodes NKN)

Flow chart for integrating the model Login to the Cluster Head Node Begabung Login to the Garuda Head Node Gridfs Login to the Cluster Head Node to be used for execution gg-blr/gg-hyd Transfer model and data to the clusters Compile the model -> create binaries Go back to Gridfs Submit the job using Job template Completion of the job (DONE status) Output on CHN (gg-blr/gg-hyd) Transfer output data back to Begabung

Data transfer specifications  Global data(CHN begabung to gg-blr/gg-hyd) EIN15, 1 year data, 15GB begabung to gg-blr/gg-hyd- 120 min@625kbps begabung to gridfs -120 min @625kbps gridfs to gg-blr/gg-hyd- 20 min @11.2mbps SURFACE data, 2GB – 10 min SST data, 368MB – 2-3 min  Output data(gg-blr/gg-hyd to begabung) @ 450kbps 1 month, SRF, 967MB - 20 min @ 450kbps SAV, 219MB- 5 min @ 380kbps ATM, 1.9GB- 45 min @ 780 kbps RAD, 1.4GB- 30 min @780 kbps

Grid Resources Used Name CPU type # Cores Network Argo Intel Nehalem 2.4 8 Infiniband DDR GHz Garuda Grid Intel Harpertown 8 Infiniband India 3.16 GHz @20gbps full gg-blr duplex EU: Briareo/Ce- AMD 2.4GHz 4 Infiniband 01 @10gbps EU:Legnaro X5650 @ 2.67GHz 12 - EU:Legnaro - AMD Opteron(tm) 24 Processor 6174

Climate of JJAS precipitation (cm) in RegCM3 and observations CRU RegCM3 IMD APHRODITE RegCM3 CMAP GPCP

Percentage differences in JJAS precipitation in RegCM3 and Observations RegCM3-IMD RegCM3-CRU RegCM3-APHRODITE RegCM3-CMAP RegCM3-GPCP

July June August September JJAS Correlation Coefficients between RegCM3 and IMD observed ensemble mean monsoon rainfall for the period 1982-2009 spanning 28 years. The contours are obtained with 9 point smoothing to the gridded result.

Domain details South Asia CORDEX Domain Model: RegCM4.1.1 Grid points: Y direction-160 Central Longitude= 70 o E Grid Points: X direction-224 Central Latitude= 16 o N Horizontal Resolution: 50Km Simulation Period: 6 Years 01 Jan 1998 to 31 Dec 2003 • CORDEX domain experiments have been conducted using ARGO, HPC cluster of ICTP for Indian summer monsoon configuration • One year climate run on 32 processors on ICTP cluster takes about 7 hrs CPU time

Experiments conducted Exp-0 default settings (Table 1) Exp-1 modified Grell over land Exp-2 modified Grell over land + with irrigated crop and ocean + with irrigated crop and ocean + removed irrigated crop Exp-3 modified Grell over land Exp-4 modified Grell over land Exp-5 modified Grell over land and ocean + removed irrigated and ocean + removed Irrigated and ocean + removed irrigated crop + dtauc15 crop + dtauc25 crop + rsmincrop Exp-8 modified Grell 2 nd time Exp-6 modified Grell over land Exp-7 modified Grell over land and 2 nd time over o cean + and ocean + removed irrigated over o cean + removed irrigated crop + rsminforest_fcmax removed irrigated crop crop Exp-10 modified Grell 2 nd time Exp-9 modified Grell over land Exp-11 modified Grell over and ocean + Emanuel over land + over o cean + Emanuel over land ocean + Emanuel over land + removed irrigated crop + removed irrigated crop with irrigated crop Exp-13 modified Grell 2 nd time Exp-14 modified Grell 2 nd time Exp-12 modified Grell over over o cean + Emanuel over land over o cean + Emanuel over ocean + Emanuel over land + with irrigated crop + modified + removed irrigated crop + land + removed irrigated crop + Zeng modified Zeng modified Zeng + rsmincrop Exp-15 modified Grell over ocean + Emanuel over land + removed irrigated crop + modified Zeng + rsmincrop

Bias between RegCM4.1.1 and CMAP observed JJAS accumulated rainfall (cm) climatology Bias between RegCM4.1.1 and IMD observed JJAS accumulated rainfall (cm) climatology

Bias between RegCM4.1.1 simulated and CRU observed climatology of JJAS mean surface temperature ( o C) from 1998-2003

The 1990 – 2008 mean annual cycle of precipitation(mm/day) in Indian land and in its five homogeneous regions. RegCM4.1.1 simulated precipitation climate (1990-2008) is compared with IMD (1990-2008), APHRO (1990-2007) GPCP (1997-2008) and TRMM (1998-2008)