Estimation of error covariance matrices in data assimilation Pierre - PowerPoint PPT Presentation

Estimation of error covariance matrices in data assimilation Pierre Tandeo Associate professor at IMT-Atlantique, Brest, France 20 th of February 2018

Before starting My organization: ◮ New engineering school: Telecom Bretagne + Mines Nantes ◮ Top 10 best school ◮ Topics: numeric, energy, environment Works in collaboration with:

State-space model Nonlinear formulation with additive and Gaussian noises: � x ( t ) = M ( x ( t − d t ) + η ( t ) (1) y ( t ) = Hx ( t ) + ǫ ( t ) (2) with: ◮ M the dynamical model (physical or statistical) ◮ H the transformation matrix from state x to observations y ◮ η ( t ) ∼ N ( 0 , Q ( t )) the model error ◮ ǫ ( t ) ∼ N ( 0 , R ( t )) the observation error ⇒ Estimate Q and R is a key point in data assimilation

State of the art ◮ 4 main families of methods to jointly estimate Q and R ◮ More than 50 papers in data assimilation from the 90’s ◮ No comparison between all these methods ◮ Poor links with signal processing and statistical communities ⇒ Review on estimation of Q and R is the goal of this presentation

Importance of errors Simple univariate, linear and Gaussian state-space model: � x ( k ) = 0 . 95 x ( k − 1) + η ( k ) y ( k ) = x ( k ) + ǫ ( k ) with η ( k ) ∼ N (0 , Q t = 1) and ǫ ( k ) ∼ N (0 , R t = 1)

Importance of errors ◮ Bad Q / R ratio: ◮ Q too low (top) ◮ R too low (bottom) ◮ Impact on state reconstruction (RMSE)

Importance of errors ◮ Good Q / R ratio: ◮ Q and R too low (top) ◮ Q and R too high (bottom) ◮ Impact on uncertainty quantification (% in CI)

Outline Filtering/smoothing methods Data Assimilation algorithms Kalman equations Innovation-based methods Innovations in the observation space Lag-innovation statistics Likelihood-based methods Bayesian approaches Maximization of the total likelihood Applications Meteorology Spatial oceanography Ecology Conclusions and perspectives Conclusions Perspectives

Data Assimilation algorithms ◮ Variational approaches: ◮ filtering/smoothing covariances not estimated ◮ adjoint model needed ◮ Particle filter/smoother: ◮ optimal in theory but... ◮ ... not relevant for high-dimensional problems ◮ Kalman-based methods: ◮ robust and widely used ◮ additive Gaussian assumptions as in Eq. (1-2)

Kalman equations Filtering step: � − 1 K ( k ) = P f ( k ) H ⊤ � HP f ( k ) H ⊤ + R ( k ) x a ( k ) = x f ( k ) + K ( k ) � y ( k ) − Hx f ( k ) �

Innovation-based methods Innovation statistics (mean and covariance): d ( k ) = y ( k ) − Hx f ( k ) (3) Σ ( k ) = HP f ( k ) H ⊤ + R ( k ) (4) Historical references: ◮ [Daley, 1992] ◮ [Dee, 1995] Issues: ◮ difficult to estimate Q and R ... ◮ ... using only the current innovation ◮ pointed out by [Blanchet et al., 1997]

Innovations in the observation space Main references: ◮ [Desroziers et al., 2005] (use of various innovations among d o − f ( k ) = y ( k ) − Hx f ( k ) and d o − a ( k ) = y ( k ) − Hx a ( k )) ◮ [Li et al., 2009] and [Miyoshi, 2011] (estimation of covariance inflation for P f and covariance R ) Solve the following system: � d o − f ( k ) d o − f ( k ) ⊤ � = HP f ( k ) H ⊤ + R ( k ) = Σ ( k )  E (5)  � d o − a ( k ) d o − f ( k ) ⊤ � = R ( k ) (6) E 

Lag-innovation statistics Main references: ◮ [Mehra, 1970] (signal processing community) ◮ [Berry and Sauer, 2013] (only lag-1 innovation) ◮ [Harlim et al., 2014] (various lag-L innovations) ◮ [Zhen and Harlim, 2015] (comparison lag-1 VS lag-L) Solve the following system (example of lag-0 VS lag-1):  � d ( k ) d ( k ) ⊤ � = HP f ( k ) H ⊤ + R ( k ) = Σ ( k ) (7) E     � d ( k ) d ( k − 1) ⊤ � = HF ( k ) P f ( k − 1) H ⊤ E    − HF ( k ) K ( k − 1) Σ ( k − 1) (8) 

Bayesian approaches Main references: ◮ classic in the statistical community ◮ [Stroud and Bengtsson, 2007] (scalar parameters for Q and R ) ◮ [Ueno and Nakamura, 2016] (parameterization of R ) ◮ [Stroud et al., 2017] (spatial parameterization of R ) Write the joint distribution of x , Q and R : p ( x ( k ) , Q , R | y (0 : k )) = p ( x ( k ) | Q , R , y (0 : k )) p ( Q , R | y (0 : k )) (9) with p ( Q , R | y (0 : k )) ∝ p ( y ( k ) | Q , R , y (0 : k − 1)) p ( Q , R | y (0 : k − 1)) (10)

Maximization of the total likelihood Main references: ◮ [Shumway and Stoffer, 1982] (linear and Gaussian case) ◮ [Ueno et al., 2010] (using grid-based algorithm) ◮ [Ueno and Nakamura, 2014] (using EM, only for R ) ◮ [Dreano et al., 2017] (using EM, for both Q and R ) Maximize the likelihood function: L ( Q , R ) = K K � � p ( x (0)) p ( x ( k ) | x ( k − 1) , Q ) p ( y ( k ) | x ( k ) , R ) (11) k =1 k =0 ⇒ estimate iteratively Q and R using the Expectation Maximization (EM) algorithm

Meteorology (1D example) 1 ◮ In situ wind speed data (Brest, France) ◮ Statistical AR(1) model for the temporal dynamics Timeseries of hourly­recorded wind intensity (January 2012) in Brest (France) 16 14 12 10 8 6 4 2 0 05/1 10/1 15/1 20/1 25/1 30/1 time( t ) Log­likelihood Q­estimates R­estimates 0.85 −1200 1.2 0.80 1.0 −1400 0.75 0.8 −1600 0.70 0.6 −1800 0.4 0.65 0.2 0.60 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 iteration iterations­( r ) iterations­( r ) 1from T.T.T. Chau, PhD candidate at Univ. Rennes II, France

Spatial oceanography (2D example) 2 ◮ Interpolation of daily SST (Sea Surface Temperature) ◮ 40 years of noisy satellite images (AVHRR-Pathfinder) 2from Autret and Tandeo 2017, ”Atlantic European North West Shelf Seas - High Resolution L4 Sea Surface Temperature Reprocessed”, http://marine.copernicus.eu

Spatial oceanography (2D example) ◮ Statistical AR(1) model for the spatio-temporal dynamics ◮ Estimate Q (statistical model error) and R (satellite error)

Ecology (1D multivariate ODE example) 3 ◮ Biogeochemical model with 80 variables (various ocean depths) ◮ Satellite observations of Chl-a (8 days, 4 km) ◮ Monitoring blooms in the Red Sea 3from Dreano 2017, ”Data and Dynamics Driven Approaches for Modelling and Forecasting the Red Sea Chlorophyll”, PhD dissertation

Conlusions ◮ Q and R are crucial in data assimilation: ◮ for prediction/filtering/smoothing modes ◮ for uncertainty quantification ◮ Review paper in preparation: ◮ exhaustive bibliography in the data assimilation community ◮ comparison between the 4 main methods ◮ Free Python code ( https://github.com/ptandeo/CEDA ):

Perspectives ◮ Combine different approaches: ◮ first offline for calibration of Q and R ... ◮ ... and then online for adaptive estimation of Q ( k ) and R ( k ) ◮ Possible extensions: ◮ deal with the initial condition (background x b and B ) ◮ parameterize Q and R for high dimensional problems ◮ Ongoing works: ◮ using conditional particle filter/smoother ◮ using fully data-driven approaches (analog data assimilation)

Thank you! Any questions?

Berry, T. and Sauer, T. (2013). Adaptive ensemble Kalman filtering of non-linear systems. Tellus, Series A: Dynamic Meteorology and Oceanography, 65(20331):1–16. Blanchet, I., Frankignoul, C., and Cane, M. A. (1997). A comparison of adaptive kalman filters for a tropical pacific ocean model. Monthly Weather Review, 125(1):40–58. Daley, R. (1992). Estimating Model-Error Covariances for Application to Atmospheric Data Assimilation. Monthly Weather Review, 120(8):1735–1746. Dee, D. P. (1995). On-line Estimation of Error Covariance Parameters for Atmospheric Data Assimilation. Monthly Weather Review, 123(4):1128–1145. Desroziers, G., Berre, L., Chapnik, B., and Poli, P. (2005). Diagnosis of observation, background and analysis-error statistics in observation space. Quarterly Journal of the Royal Meteorological Society, 131(613):3385–3396. Dreano, D., Tandeo, P., Pulido, M., Chonavel, T., AIt-El-Fquih, B., and Hoteit, I. (2017). Estimating model error covariances in nonlinear state-space models using Kalman smoothing and the expectation-maximisation algorithm. Quarterly Journal of the Royal Meteorological Society, 143(705):1877–1885. Harlim, J., Mahdi, A., and Majda, A. J. (2014). An ensemble Kalman filter for statistical estimation of physics constrained nonlinear regression models. Journal of Computational Physics, 257:782–812. Li, H., Kalnay, E., and Miyoshi, T. (2009). Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filte. Quarterly Journal of the Royal Meteorological Society, 135(2):523–533. Mehra, R. K. (1970). On the Identification of Variances and Adaptive Kalman Filtering. IEEE Transactions on Automatic Control, 15(2):175–184.