Using Spatial Gradient Information to Extract Small Scale - PowerPoint PPT Presentation

7 th International Symposium on Data Assimilation: 21 January 2019 Using Spatial Gradient Information to Extract Small Scale Information from Observations With Correlated Errors Joël Bédard and Mark Buehner Data Assimilation and Satellite Meteorology Research Section

2/15 Motivation Even though satellite brightness temperature is the observation type with the biggest impact on atmospheric analyses and ensuing forecasts, they are still underused (e.g. ECCC does horizontal thinning at 150km) • Brightness temperature errors can have significant spatial correlations, due to: – Correlated instrument noise and representativeness error – Correlation from background state used in the observation operator • Assimilation algorithms assume uncorrelated observation errors (especially spatial correlations), because: – Inverting the full covariance matrix is prohibitively expensive – Difficult to estimate the real error correlations • It is common practice to spatially thin obs. to reduce error correlation between remaining obs., resulting in a loss of small-scale information • Goal: Develop a practical assimilation approach to extract small-scale information from observations with spatially correlated errors, while still assuming spatially uncorrelated obs. in the assimilation

3/15 Experimental framework Assimilation experiments are performed in a simplified context to assess the impact of observations with spatially correlated errors on analyses • Unidimensional periodic domain with 100 grid points • Single dimensionless variable • Observations are available at every grid point (no interpolation, H = I ) • Background and observation error variances are set to 1.0 • Tests performed with several correlation functions (GAUS, SOAR, FOAR) and correlation length scales are defined w.r.t. the e-folding scale: – Background (prior) error correlation length scale ( L x ): 3 grid points – Obs. error correlation length scale ( L y ): 0.0, 1.5, 3.0, and 4.5 grid points • The true analysis error covariances ( A ) computed as: A = ( I – K ) B ( I – K ) T + KRK T where K = B ( R est + B ) -1 – B and R are the true background and obs. error covariance matrices – R est is the estimated obs. error covariance matrix used (typically diagonal) • All assimilation tests using diagonal observation error covariance matrix employ optimally inflated variances 3 16 January, 2019

4/15 Correlated observation error • Positively correlated errors have more energy at the large scales • The spectrum is similar to red noise 4 16 January, 2019

5/15 Correlated observation error • Simply neglecting the error correlations can degrade the resulting analysis by overfitting large scales and underfitting the small scales 5 16 January, 2019

6/15 Correlated observation error • Inflation allows correctly fitting the scales at which background error has the most energy (large scales), but further neglects small scales 6 16 January, 2019

7/15 Correlated observation error • Thinning reduces spatial error correlation between the remaining obs. (spectrum more flat), but information on the small scales is lost 7 16 January, 2019

8/15 Impact of thinning on analysis error • With more thinning, the impact of neglecting correlations is reduced, at the expense of losing skill for an increasing number of wavenumbers at the small scales • Similar results (but less significant) are obtained for the different correlation functions (GAUS, SOAR, FOAR) and observation error correlation length scales 8 16 January, 2019

9/15 Thinning impact with inflated diagonal R • Optimal inflation ≈ 1 when correlation ≈ 0.2 (Liu and Rabier, 2002) • With optimally inflated diagonal R, the analysis is degraded slightly with thinning (it is best to keep most or all the obs.) • The analysis error is sensitive to the obs. error correlation function (GAUS, SOAR, FOAR) and length scale ( L y ) used (not shown) 9 16 January, 2019

10/15 Difference-observations • Thinning and inflation both can prevent overfitting large scales, but at the expense of loosing information at the small scales • Similar to other studies that use spatial transformations to reduce error correlations (Ruggiereo et al. , 2016), spatial differences of neighboring observations (difference-obs.) can be used to increase the weight on small scales (Inspired from Anderson's presentation at the Australia data assimilation workshop in 2016) Original observations … Y 1 Y 2 Y 3 Y 4 Y n-1 Y n … Y 3 – Y 2 Y 4 – Y 3 Y n – Y n-1 Y 2 - Y 1 Difference observations • Combining thinned obs. (as they are currently used in operational NWP) with spatial difference-obs. (both with their own tuned diagonal R) may allow introducing complementary information at small scales 10 16 January, 2019

11/15 Difference-observations • Spatial differential operator = multiplying spectrum by wavenumber • More weight is given to "intermediate" scales and the information at wavenumber 0 is lost 11 16 January, 2019

12/15 Difference-observations • The link between neighboring difference-obs. introduces negative correlations not accounted for in the assimilation (not shown) • By computing differences, the information at wavenumber 0 is lost • Using difference-obs. with diagonal R improves "intermediate" scales Difference-obs Original obs GAUS L y > L x 12 16 January, 2019

13/15 Combining original and difference-obs • The analysis benefits from both the large scales of the original observations and the small scales from difference-obs. • Ignoring difference-obs. correlations still limits the skill at fine scales when observation error correlation length scale is large ( L y > L x ) 13 16 January, 2019

14/15 Combining original and difference-obs • The analysis error is reduced and it is now less sensitive to thinning • Optimal inflation has to be computed for both thinned and diff-obs. • The analysis error is also less sensitive to the observation error correlation function (GAUS, SOAR, FOAR) and length scale ( L y ) used (not shown) 14 16 January, 2019

15/15 Take Home Messages The necessity of using a diagonal R matrix sacrifices small scales • Both inflation and thinning can be used to give correct weight to large scales, but underweights the small scales (when obs. error is correlated) • The analysis error is sensitive to the observation error correlation function (GAUS, SOAR, FOAR) and length scale ( L y ) used • With optimally inflated diagonal R, the analysis is degraded with thinning and it is best to keep all the obs. Supplement current methods with complementary information • Difference observations: lose large scale information, but improves analysis at smaller ("intermediate") scales while still using a diagonal R matrix Combining thinned observations with difference-obs. • Two parameters (inflation factors) need to be tuned experimentally • Reduces overall analysis error as compared with thinned obs. alone • Provides good fit to observations at both large and finer scales • Analysis error is less sensitive to the amount of thinning and also to the structure (Gaussian, SOAR, FOAR) and length scale of obs. error correlations

7 th International Symposium on Data Assimilation: 21 January 2019 Additional Material Joël Bédard and Mark Buehner Data Assimilation and Satellite Meteorology Research Section

A1 Combining original and difference-obs • The link between neighboring difference-obs. introduces negative correlations not accounted for in the assimilation (not shown) 17 16 January, 2019

A2 Thinning impact on combined obs. • With optimally inflated diagonal R, the analysis is not significantly degraded with thinning of the original observations • Optimal inflation has to be computed for both thinned and diff-obs. • The analysis error is less sensitive to the obs. error correlation function (GAUS, SOAR, FOAR) and length scale ( L y ) used (not shown) 18 16 January, 2019