Ensemble sensitivity and sampling error correction evaluated using a - PowerPoint PPT Presentation

Ensemble sensitivity and sampling error correction evaluated using a convective-scale 1000 member ensemble Tobias Necker (LMU) M. Weissmann (LMU, DWD), S. Geiss (LMU), T. Miyoshi (RIKEN), J. Ruiz (CIMA), G.-Y. Lien (TCBW) and J. Anderson (NCAR) 1 ISDA 2019

Outline 1000 Motivation Ø Ø 1000 member ensemble simulation Sampling error (correction algorithms) Ø Ø Potential impact of observed quantities Ø Conclusions 2 ISDA 2019

Motivation - 1000 member ensemble 1000 1000 Realistic correlations Correlation of 3-h precipitation forecast inside the box to initial 850 hPa humidity 3 ISDA 2019

Observing strategies Motivation - 1000 member ensemble Realistic correlations Goal : 1000 members Potential impact of observations 40 members Sub- sampling Correlation of 3-h precipitation forecast inside the box to initial 850 hPa humidity Goal : Quantifiaction of sampling errors 4 ISDA 2019

Convective-scale 1000 member ensemble Japanese "SCALE-RM" model l LETKF (15km; conventional observations) l Spin up : 1 week l Downscaling to from 15km (CY) to 3km (FC) l Period of 5 days/10 FCs in Mai/June 2016 for convective-scale forecasts l Global GFS ensemble BC using NCEP 20 l Forecast domain: 3 km grid spacing, member analysis ensemble combined with 350x250 grid points with 30 levels 1000 random perturbations Domains: 0 3 6 9 12 15 t global GFS Ens. BC GFS Ens. BC GFS Ens. BC 3-h cycling CY 15 km forecast Down scaling FC 3 km forecast 14-h forecasts 5 ISDA 2019

Sampling error correction S: sensitvity Ensemble sensitivity analysis / sample correlation: J : response function x : state variable r : sample correlation (-1,1) r sec : sampling error corrected correlation σ : sample standard deviation SEC: sampling error correction Sampling Error Correction (SEC) : Designed to replace/reduce need of localization • Offline Monte-Carlo technique -> look-up table • r sec table depends on ensemble size, • sample correlation and assumed prior (normal) correlation distribution -> Samling error corrected sensitivity: Anderson (2012): Localization and Sampling Error Correction in EnKF DA 6 ISDA 2019

ECMWF analysis, 29.05.2016 – 0 UTC Example of temporal correlation ESA setup: l 3-h lead time domain l Response function J: Precipitation coarse grained over 40x40 grid points l Spatio-temporal correlations Correlation of precipitation forecast 1000 member ensemble mean precipitation to 2-m temperature field 1000 J Streamlines of 500 hPa wind 7 ISDA 2019

Qualitative analysis Sampling errors: l 40 & 200 member 1000 200 ensemble are sub-samples of 1000 member ensemble l SEC systemtically reduces sampling errors l Confidence test (T95) discards correlations 40+SEC 40+T95 40 Correlation of 3-h precipitation forecast to initial 2-m temperature, 1 forecast 8 ISDA 2019

Quantitative analysis Sampling error correction (SEC): l Improves frequency distribution significantly l Reduces error for small correlation values l For 2-m T: no improvements for large correlation values (Note: small sample size!) l 1000 member PDF: better prior assumption could improve SEC performance Frequency distribution Mean absolute error (MAE) Correlation of 3-h precipitation forecast to initial 2-m temperature, 10 forecasts 9 ISDA 2019

Sampling error as function of ensemble size Sampling errors: l Doubling the ensemble size from 40 to 80 member decreases sampling error by 30% l SEC significantly reduces sampling errors for all investigated ensemble sizes l 40 member + SEC performs better then 80 member Correlation of 3-h precipitation forecast to initial 2-m temperature, 10 forecasts 10 ISDA 2019

Sampling error correction for different variables Spatio-temporal correlation using 40 member ensemble: SEC reduces RMSE by about 20 - 30% • SEC significantly corrects magnitude BIAS caused by spurious correlations • Correlation of 3-h precipitation forecast to various model fields, 10 forecasts using 40 member 11 ISDA 2019

Spatial correlations for DA Spatial correlations as function of horizontal distance: l 40 member: SEC reduces sampling error up to 30% l For e.g. highly positivly correlated variables, no improvements due to insufficient prior à Different or adaptive prior r(x,J) needed as in CER algorithm (Anderson, 2016) Correlation Cross-Correlation T 2m to T 2m T 500hPa to QV 500hPa Absolute mean Absolute error Absolute mean Absolute error 12 ISDA 2019

Correlation as function of ensemble size Accumulated squared correlation (ASC) as proxy for potential impact : l Saturation below 1000 member for all variables l Overestimation of potential impact for smaller ensembles due to sampling errors 500 hPa zonal wind 2-m temperature Precipitation 1000 member 13 ISDA 2019

Sampling error and potential impact Confidence test (T95) and SEC : l SEC performes better then confidence test l 200 + SEC close to 1000 member (dotted line) l 1000 + T95 prove of confidence 500 hPa zonal wind 2-m temperature Precipitation 1000 member 14 ISDA 2019

Conclusions Ø Temporal and spatial correlations have been evaluated for a convective-scale 1000 member ensemble simulation over Europe Ø Sampling error correction (ESA, spatio-temporal correlations): - Significantly reduces sampling errors - Simple prior assumption is suitable Ø Sampling error correction (DA, spatial correlations) : - Promissing especially for convective-scale and vertical application - Adaptiv prior required for better performance (as done by Anderson 2016) Ø Confidence test (T95) ) -> suitable for qualitative analysis (ESA) Sampling error correction (SEC) -> qualitative & quantitative improvements Ø Accumulated squared correlation (ASC) used as proxy for potential impact ( Required ensemble size: 200 member + SEC close to 1000 member ) 15 ISDA 2019

References Torn, R. D., 2010 : Ensemble-Based Sensitivity Analysis Applied to African Easterly Waves. Weather and Forecasting. Anderson, J. L. 2012: Localization and Sampling Error Correction in Ensemble Kalman Filter Data Assimilation. Mon. Wea. Rev. Anderson, J. L., 2016: Reducing Correlation Sampling Error in Ensemble Kalman Filter Data Assimilation. Mon. Wea. Rev. Necker, T. et al 2018: The importance of appropriate verification metrics for the assessment of observation impact in a convection-permitting modelling system. Q. J. R. Meteorol. Soc. 16 ISDA 2019