Closed-Loop Applicability of the Sign-Perturbed Sums Method aji 1 - PowerPoint PPT Presentation

Closed-Loop Applicability of the Sign-Perturbed Sums Method aji 1 Erik Weyer 2 Bal´ azs Csan´ ad Cs´ 1 Institute for Computer Science and Control (SZTAKI), Hungarian Academy of Sciences, Hungary 2 Department of Electrical and Electronic Engineering (EEE), University of Melbourne, Australia 54th IEEE CDC, Osaka, Japan, December 15-18, 2015

Overview I. Introduction II. Sign-Perturbed Sums for Open-Loop Systems III. Sign-Perturbed Sums for Closed-Loop Systems – Direct Identification – Indirect Identification – Joint Input-Output Identification IV. Experimental Results V. Summary and Conclusion Cs´ aji & Weyer Closed-Loop SPS | 2

Closed-Loop General Linear System N t H ( z -1 ) R t U t Y t L ( z -1 ) G ( z -1 ) + + – F ( z -1 ) t : (discrete) time, Y t : output, U t : input, N t : noise, R t : reference, F , G , H , L (causal) rational transfer functions, z − 1 : backward shift. Cs´ aji & Weyer Closed-Loop SPS | 3

Closed-Loop General Linear System Dynamical System: General Linear Y t � G ( z − 1 ; θ ∗ ) U t + H ( z − 1 ; θ ∗ ) N t t : (discrete) time, Y t : output, U t : input, N t : noise, R t : reference, G , H : transfer functions, z − 1 : backward shift, θ ∗ : true parameter. Controller: Closed-Loop with Reference Signal U t � L ( z − 1 ; η ∗ ) R t − F ( z − 1 ; η ∗ ) Y t L , F : transfer functions parametrized independently of G , H . Cs´ aji & Weyer Closed-Loop SPS | 4

Main Assumptions (A1) The “true” systems generating { Y t } and { U t } are in the model classes; G and H have known orders. (A2) Transfer function H ( z − 1 ; θ ) has a stable inverse, and G (0; θ ) = 0 and H (0; θ ) = 1, for all θ ∈ Θ. (A3) The noise sequence { N t } is independent, and each N t has a symmetric probability distribution about zero. (A4) The initialization is known, Y t = N t = R t = 0, t ≤ 0. (A5) The subsystems from { N t } and { R t } to { Y t } are asymptotically stable and have no unstable hidden modes. (A6) Reference signal { R t } is independent of the noise { N t } . Cs´ aji & Weyer Closed-Loop SPS | 5

Review: SPS for Open-Loop Systems General Linear Systems Y t � G ( z − 1 ; θ ∗ ) U t + H ( z − 1 ; θ ∗ ) N t – Sign-Perturbed Sums (SPS) is a finite sample system identifi- cation method which can build confidence regions. – SPS is distribution-free, it can work for any symmetric noise. – The confidence set has exact confidence probability (user-chosen). – The SPS sets are build around the prediction error estimate. – SPS is strongly consistent (for lin. reg.). – The sets of SPS are star convex (for lin. reg.). – Efficient ellipsoidal outer approximations exists (for lin. reg.). Cs´ aji & Weyer Closed-Loop SPS | 6

Open-Loop Prediction Error Estimate Prediction Error or Residual (for parameter θ ) � � ε t ( θ ) � H − 1 ( z − 1 ; θ ) Y t − G ( z − 1 ; θ ) U t � Note that � ε t ( θ ∗ ) = N t , hence, it is accurate for θ = θ ∗ . Prediction Error Estimate (for model class Θ) n � ˆ ε 2 θ PEM � arg min V ( θ | Z ) = arg min t ( θ ) � θ ∈ Θ θ ∈ Θ t =1 where Z is the available data: finite realizations of { Y t } and { U t } . In general, there is no closed-form solution for PEM. Cs´ aji & Weyer Closed-Loop SPS | 7

Open-Loop Prediction Error Equation The PEM estimate can be found, e.g., by using the equation PEM Equation � n θ V (ˆ ψ t (ˆ ε t (ˆ ∇ θ PEM | Z ) = θ PEM ) � θ PEM ) = 0 t =1 where ψ t ( θ ) is the negative gradient of the prediction error, ψ t ( θ ) � −∇ ε t ( θ ) . θ � These gradients can be directly calculated in terms of the defining polynomials of the rational transfer functions G and H . Cs´ aji & Weyer Closed-Loop SPS | 8

Perturbed Samples: Open-Loop Case Perturbed Output Trajectories ¯ Y t ( θ, α i ) � G ( z − 1 ; θ ) U t + H ( z − 1 ; θ ) ( α i , t � ε t ( θ )) where { α i , t } are random signs: α i , t = ± 1 with probability 1 2 each. Recall that ψ t ( θ ) is a linear filtered version of { Y t } and { U t } , ψ t ( θ ) = W 0 ( z − 1 ; θ ) Y t + W 1 ( z − 1 ; θ ) U t , where W 0 and W 1 are vector-valued, and ψ t ( θ ) ∈ R d . Perturbed (Negative) Gradients ψ t ( θ, α i ) � W 0 ( z − 1 ; θ ) ¯ ¯ Y t ( θ, α i ) + W 1 ( z − 1 ; θ ) U t Cs´ aji & Weyer Closed-Loop SPS | 9

Sign-Perturbed Sums: Open-Loop Case Reference and m − 1 Sign-Perturbed Sums � n − 1 S 0 ( θ ) � Ψ n ( θ ) 2 t =1 ψ t ( θ ) � ε t ( θ ) � n − 1 S i ( θ ) � ¯ ¯ 2 Ψ n ( θ, α i ) ψ t ( θ, α i ) α i , t � ε t ( θ ) t =1 where Ψ n and ¯ Ψ n are (sign-perturbed) covariances estimates � n Ψ n ( θ ) � 1 t =1 ψ t ( θ ) ψ T t ( θ ) n � n Ψ n ( θ, α i ) � 1 ¯ ψ t ( θ, α i ) ¯ ¯ ψ T t ( θ, α i ) n t =1 Cs´ aji & Weyer Closed-Loop SPS | 10

Non-Asymptotic Confidence Regions: Open-Loop Case R ( θ ) is the rank of � S 0 ( θ ) � 2 among {� S i ( θ ) � 2 } (with tie-breaking). SPS Confidence Regions for General Linear Systems � � θ ∈ R d : R ( θ ) ≤ m − q � Θ n � where m > q > 0 are user-chosen (integer) parameters. We have S 0 (ˆ θ PEM ) = 0, thus, ˆ θ PEM ∈ � Θ n , if it is non-empty. Exact Confidence of SPS for General Linear Systems � � = 1 − q θ ∗ ∈ � P Θ n m Cs´ aji & Weyer Closed-Loop SPS | 11

Closed-Loop Prediction Error Methods (PEMs) – Direct Identification (Simply neglect the controller, treat the system as the inputs were independent, i.e., if the system operated in open-loop). – Indirect Identification (If the controller is known, treat the reference signal as the input, leading to a reformulated open-loop system). – Joint Input-Output Identification (Identify both the system and the controller as if the observa- tions would come from a system with vector-valued outputs). Cs´ aji & Weyer Closed-Loop SPS | 12

Direct Identification Direct Identification (PEM) – Goal: to estimate θ ∗ , i.e., to identify H and G . – Assumption: controller is informative. – Idea: feedback is neglected. – Method: SISO Open-Loop PEM (original system). Simply neglecting the feedback does not work for SPS, as { ¯ Y t ( θ ∗ , α 1 ) } , . . . , { ¯ Y t ( θ ∗ , α m − 1 ) } { Y t } and does not have the same distribution (essential for exact confidence). The alternative outputs should be built using alternative inputs. Cs´ aji & Weyer Closed-Loop SPS | 13

Closed-Loop SPS for Direct PEM Assume that the controller can be simulated (black box). Then, the alternative output trajectories can be redefined as Direct SPS: Perturbed Output Trajectories Y t ( θ, α i ) � G ( z − 1 ; θ ) ¯ � U t ( θ, α i ) + H ( z − 1 ; θ ) ( α i , t � ε t ( θ )) using alternative feedbacks given the alternative outputs Direct SPS: Alternative Feedbacks U t ( θ, α i ) � L ( z − 1 ; η ∗ ) R t − F ( z − 1 ; η ∗ ) � ¯ Y t ( θ, α i ) The exact confidence probability of Direct SPS is then guaranteed. Cs´ aji & Weyer Closed-Loop SPS | 14

Indirect Identification Indirect Identification (PEM) – Goal: to estimate θ ∗ , i.e., to identify H and G . – Assumptions: controller is known, { R t } is measurable. – Idea: restate as an open-loop system, treat { R t } as inputs. – Method: SISO Open-Loop PEM (reformulated system). An alternative open-loop system can be formulated as Y t = G 0 ( z − 1 ; κ ∗ ) R t + H 0 ( z − 1 ; κ ∗ ) N t where the parametrization, κ , can be different and G 0 ( z − 1 ; κ ∗ ) � (1 + GF ) − 1 GL H 0 ( z − 1 ; κ ∗ ) � (1 + GF ) − 1 H Cs´ aji & Weyer Closed-Loop SPS | 15

Closed-Loop SPS for Indirect PEM Then, open-loop SPS can applied by treating { R t } as the input. In order to test θ , the alternative κ should be first computed from (1 + G ( θ ) F ) − 1 G ( θ ) L = G 0 ( κ ) (1 + G ( θ ) F ) − 1 H ( θ ) = H 0 ( κ ) If an (exact or approximate) solution is given by κ = g ( θ ), then Indirect SPS Confidence Regions � Θ id � { θ ∈ Θ : R ( g ( θ )) ≤ m − q } n which results in exact confidence under the additional assumption (A7) Parameter transformation g satisfies g ( θ ∗ ) = κ ∗ . Cs´ aji & Weyer Closed-Loop SPS | 16

Joint Input-Output Identification Joint Input-Output Identification (PEM) – Goal: to estimate ( θ ∗ , η ∗ ), the controller is also identified. – Assumption: no reference signal (for simplicity). – Idea: reformulate as an autonomous vector-valued system. – Method: MIMO Open-Loop PEM (vector-valued system). [ Y t , U t ] T is treated as output of a vector-valued autonomous system � Y t � � � ( I + GF ) − 1 H N t = � H ( z − 1 , κ ∗ ) N t , Z t � = − F ( I + GF ) − 1 H U t driven by symmetric and independent noise terms { N t } . Thus, a vector-valued variant of SPS is needed (future research). Cs´ aji & Weyer Closed-Loop SPS | 17