Introduction to Constrained Estimation Graham C. Goodwin September - PowerPoint PPT Presentation

Introduction to Constrained Estimation Graham C. Goodwin September 2004 Centre for Complex Dynamic Systems and Control

2.1 Background Constraints are also often present in estimation problems. A classical example of a constrained estimation problem is the case in which binary data (say ± 1) are transmitted through a communication channel where it suffers dispersion causing the data to overlay itself. In the field of communications, this is commonly referred to as intersymbol interference [ISI]. The associated estimation problem is: Given the output of the channel, provide an estimate of the transmitted signal. Centre for Complex Dynamic Systems and Control

To illustrate some of the ideas involved in the above problem, let us assume, for simplicity, that the intersymbol interference produced by the channel can be modelled via a finite impulse response [FIR] model of the form: m � y k = g ℓ u k − ℓ + n k , (1) ℓ = 0 where y k , u k , n k denote the channel output, input and noise. Centre for Complex Dynamic Systems and Control

Heuristically, one might expect that one should “invert” the channel so as to recover the input sequence { u k } from a given sequence of output data { y k } . Such an inverse can be readily found by utilising feedback ideas. Centre for Complex Dynamic Systems and Control

Expand the channel transfer function as: G ( z ) = g 0 + . . . + g m z − m = g 0 + ˜ G ( z ) , then we can form an inverse by the feedback circuit shown in Figure 1. Centre for Complex Dynamic Systems and Control

Figure 1 y k u k ˜ + 1 g 0 − ˜ G ( z ) Figure: Feedback inverse circuit. Centre for Complex Dynamic Systems and Control

To verify that the circuit of Figure 1 does, indeed, produce an inverse, we see that the transfer function from y k to ˜ u k is 1 1 1 g 0 T ( z ) = = = G ( z ) . ˜ g 0 + ˜ G ( z ) G ( z ) 1 + g 0 Centre for Complex Dynamic Systems and Control

Running Example Consider the channel model y k = u k − 1 . 7 u k − 1 + 0 . 72 u k − 2 + n k , where u k is a random binary signal and n k is an independent identically distributed [i.i.d.] noise having a Gaussian distribution of variance σ 2 . Centre for Complex Dynamic Systems and Control

Figure 2 4 3 2 u k 1 u k , ˜ 0 −1 −2 −3 −4 0 5 10 15 20 25 k Figure: Data u k (circle-solid line) and estimate ˜ u k (triangle-solid line) using the feedback inverse circuit of Figure 1. Noise variance: σ 2 = 0. Centre for Complex Dynamic Systems and Control

Next, we simulate the inversion estimator when the received signal is affected by noise n k of variance σ 2 = 0 . 1. Centre for Complex Dynamic Systems and Control

Figure 3 4 3 2 u k 1 u k , ˜ 0 −1 −2 −3 −4 0 5 10 15 20 25 k Figure: Data u k (circle-solid line) and estimate ˜ u k (triangle-solid line) using the feedback inverse circuit of Figure 1. Noise variance: σ 2 = 0 . 1. Centre for Complex Dynamic Systems and Control

An improvement seems to be to simply take the nearest value from the set { + 1 , − 1 } corresponding to ˜ u k . This leads to the circuit shown in Figure 4, where  + 1 if ˜ u k ≥ 0 ,   sign(˜ u k ) �   − 1 if ˜ u k < 0 .   Centre for Complex Dynamic Systems and Control

Figure 4 ˜ ˆ y k u k u k 1 + sign g 0 − ˜ G ( z ) Figure: Constrained feedback inverse circuit. Centre for Complex Dynamic Systems and Control

Figure 5 4 3 2 u k 1 u k , ˆ 0 −1 −2 −3 −4 0 5 10 15 20 25 k Figure: Data u k (circle-solid line) and estimate ˆ u k (triangle-solid line) using the constrained feedback inverse circuit of Figure 4. Noise variance: σ 2 = 0 . 1. Centre for Complex Dynamic Systems and Control

Our belief is that ˆ u k should be a better estimate of the input than u k since we have forced the constraint ˆ ˜ u k ∈ { + 1 , − 1 } . This suggests that we could try feeding back ˆ u k instead of ˜ u k , as shown in Figure 6. This is called a Decision Feedback Equaliser (DFE) in the Communications Literature. Centre for Complex Dynamic Systems and Control

Figure 6 y k u k ˆ + 1 sign g 0 − ˜ G ( z ) Figure: Constrained estimation with decision feedback, or “decision f eed- back equaliser [DFE].” Centre for Complex Dynamic Systems and Control

Figure 7 4 3 2 u k 1 u k , ˆ 0 −1 −2 −3 −4 0 5 10 15 20 25 k Figure: Data u k (circle-solid line) and estimate ˆ u k (triangle-solid line) using the DFE of Figure 6. Noise variance: σ 2 = 0 . 1. Centre for Complex Dynamic Systems and Control

We see that this circuit has led to perfect recovery of the transmitted data! One might wonder if the DFE circuit would always perform so well. We next investigate the performance of the DFE of Figure 6 when the noise variance is increased by a factor of 2; that is, σ 2 = 0 . 2. Centre for Complex Dynamic Systems and Control

Figure 8 4 3 2 u k 1 u k , ˆ 0 −1 −2 −3 −4 0 5 10 15 20 25 k Figure: Data u k (circle-solid line) and estimate ˆ u k (triangle-solid line) using the DFE of Figure 6. Noise variance: σ 2 = 0 . 2. Centre for Complex Dynamic Systems and Control

We see that the circuit now performs badly in the case of increased measurement noise. We can gain some insight as to from where further improvements might come by expressing the result shown in Figure 6 as the solution to an optimisation problem. Specifically, assume that we are given (estimates of) past values of the input, { ˆ u k − 1 , . . . , ˆ u k − m , . . . } , and that we model the output ˆ y k as y k = g 0 u ′ ˆ k + g 1 ˆ u k − 1 + . . . + g m ˆ u k − m . Centre for Complex Dynamic Systems and Control

We can now ask what value of u ′ k causes ˆ y k to be, at time k , as close as possible to the observed output y k . We measure how close ˆ y k is to y k by the following one-step objective function: y k ] 2 . y k , u ′ V 1 (ˆ k ) = [ y k − ˆ We also require that u ′ k ∈ { + 1 , − 1 } . Centre for Complex Dynamic Systems and Control

The solution to this constrained optimisation problem is readily seen to be: � 1 � u k = sign ˆ [ y k − g 1 ˆ u k − 1 − . . . − g m ˆ u k − m ] (2) . g 0 The above is the DFE. Centre for Complex Dynamic Systems and Control

Generalise to the following tw o-stage objective function: y k ] 2 + [ y k + 1 − ˆ y k + 1 ] 2 , V 2 (ˆ y k , ˆ y k + 1 , u ′ k , u ′ k + 1 ) = [ y k − ˆ (3) where y k = g 0 u ′ ˆ k + g 1 ˆ u k − 1 + . . . + g m ˆ u k − m , (4) y k + 1 = g 0 u ′ ˆ k + 1 + g 1 u ′ k + g 2 ˆ u k − 1 + . . . + g m ˆ u k − m + 1 , (5) and where the past estimates { ˆ u k − 1 , ˆ u k − 2 , . . . } are again assumed fixed and known. Centre for Complex Dynamic Systems and Control

The solution to the above problem can be readily computed by simple evaluation of V 2 for all possible constrained inputs; that is, for { u ′ k , u ′ k + 1 } ∈ � {− 1 , − 1 } , {− 1 , 1 } , { 1 , 1 } , { 1 , − 1 } � . (6) Centre for Complex Dynamic Systems and Control

We could then fix the estimate of u k (denoted ˆ u k ) as the first element of the solution to this optimisation problem. We might then proceed to measure y k + 2 and re-estimate u k + 1 , plus obtain a fresh estimate of u k + 2 by minimising: y k + 1 ] 2 + [ y k + 2 − ˆ y k + 2 ] 2 , V 2 (ˆ y k + 1 , ˆ y k + 2 , u ′ k + 1 , u ′ k + 2 ) = [ y k + 1 − ˆ where y k + 1 = g 0 u ′ ˆ k + 1 + g 1 ˆ u k + . . . + g m ˆ u k − m + 1 , y k + 2 = g 0 u ′ ˆ k + 2 + g 1 u ′ k + 1 + g 2 ˆ u k + . . . + g m ˆ u k − m + 2 , Centre for Complex Dynamic Systems and Control

By the above procedure, we are already generating constrained estimates via a moving horizon estimator [MHE] subject to the constraint u ′ k ∈ { + 1 , − 1 } . Centre for Complex Dynamic Systems and Control

The corresponding simulation results, for noise variance σ 2 = 0 . 2, are shown in Figure 9. Centre for Complex Dynamic Systems and Control

Figure 9 4 3 2 u k 1 u k , ˆ 0 −1 −2 −3 −4 0 5 10 15 20 25 k Figure: Data u k (circle-solid line) and estimate ˆ u k (triangle-solid line) using the moving horizon two-step estimator. Noise variance: σ 2 = 0 . 2. Centre for Complex Dynamic Systems and Control

Connections Between Constrained Control and Estimation The brief introduction to constrained control and estimation given above will have, no doubt, left the reader with the impression that these two problems are, at least, very similar . Indeed, both have been cast as finite horizon constrained optimisation problems. We will see later that these problems lead to the same underlying question, the only difference being a rather minor issue associated with the boundary conditions. Centre for Complex Dynamic Systems and Control

Actually, we will show that a strong connection between constrained control and estimation problems is revealed when looked upon via a Lagrangian duality perspective. This will be the topic of the Lecture 2 of Friday. Centre for Complex Dynamic Systems and Control