Adaptive Filters Algorithms (Part 1) Gerhard Schmidt - PowerPoint PPT Presentation

Adaptive Filters – Algorithms (Part 1) Gerhard Schmidt Christian-Albrechts-Universität zu Kiel Faculty of Engineering Electrical Engineering and Information Technology Digital Signal Processing and System Theory Slide 1

Contents of the Lecture Today: Exercises:  Topics for the Talks Adaptive Algorithms:  Introductory Remarks  Recursive Least Squares (RLS) Algorithm  Least Mean Square Algorithm (LMS Algorithm) – Part 1  Least Mean Square Algorithm (LMS Algorithm) – Part 2  Affine Projection Algorithm (AP Algorithm) Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 2 Slide 2

Adaptive Filters – Talks Possible Topics Suggestions:  Hearing aids  GSM (source) coding  Localization and tracking  Active noise control (anti-noise)  Noise suppression  Bandwidth extension  Audio upmix of stereo signals  Adaptive beamforming  MPEG audio coding  Non-linear echo cancellation  Adaptation of neural networks  Feedback suppression  … Your own topic suggestions are welcome … Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 3 Slide 3

Adaptive Filters – Algorithms Contents Exercises:  Topics for the Talks Adaptive Algorithms:  Introductory Remarks  Recursive Least Squares (RLS) Algorithm  Least Mean Square Algorithm (LMS Algorithm) – Part 1  Least Mean Square Algorithm (LMS Algorithm) – Part 2  Affine Projection Algorithm (AP Algorithm) Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 4 Slide 4

Introductory Remarks Motivation Why adaptive filters?  Signal properties are not known in advance or are time variant.  System properties are not known in advance or time variant. Examples:  Speech signals  Mobile telephone channels Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 5 Slide 5

Introductory Remarks Literature Books:  E. Hänsler, G. Schmidt: Acoustic Echo and Noise Control , Wiley, 2004  S. Haykin: Adaptive Filter Theory , Prentice Hall, 2002  A. Sayed: Fundamentals of Adaptive Filtering , Wiley, 2004  E. Hänsler: Statistische Signale: Grundlagen und Anwendungen , Springer, 2001 (in German) Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 6 Slide 6

Introductory Remarks Two Hook-Ups of Adaptive Filters Adaptive filter for channel equalization: Transmission Adaptive channel filter Adaptive filter for system identification: System Adaptive filter Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 7 Slide 7

Introductory Remarks Application Examples – Part 1 Adaptive filter for cancellation of hybrid echoes: Adaptive Hybrid filter Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 8 Slide 8

Introductory Remarks Application Examples – Part 2 Adaptive filter for noise reduction with reference signal: Noisy signal Signal source Transmission path 1 Adaptive Noise filter source Transmission path 2 Reference signal Signal model Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 9 Slide 9

Introductory Remarks Application Examples – Part 3 Antenna array: Adaptive Adaptive Adaptive filter 1 filter 2 filter N Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 10 Slide 10

Introductory Remarks Application Examples – Part 4 Adaptive equalization without reference signal Adaptive filter Decision circuit Assumptions: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 11 Slide 11

Introductory Remarks Generic Setup Desired output signal Adaptive filter Adaptive algorithm Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 12 Slide 12

Introductory Remarks Structure of an Adaptive FIR Filter Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 13 Slide 13

Introductory Remarks Error Measures – Part 1 No local noise + Mean square (signal) error: System distance: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 14 Slide 14

Introductory Remarks Mean Square Error and System Distance Relation of the normalized mean square (signal) error power and the system distance: Let be white noise: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 15 Slide 15

Introductory Remarks Adaptation Local noise Basic principle: + New = old + correction + Properties:  „Correction“ depends on the input signal and the error signal .  Procedures differ by the functions and : Step size Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 16 Slide 16

Introductory Remarks Error Measures Local noise + + Three error measures control the adaptation:  Coefficient error  A priori error  A posteriori error: Old data New filter Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 17 Slide 17

Adaptive Filters - Algorithms Contents Exercises:  Topics for the Talks Adaptive Algorithms:  Introductory Remarks  Recursive Least Squares (RLS) Algorithm  Least Mean Squares Algorithm (LMS Algorithm) – Part 1  Least Mean Squares Algorithm (LMS Algorithm) – Part 2  Affine projection Algorithm (AP Algorithm) Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 18 Slide 18

Recursive Least Squares (RLS) Algorithm Algorithmic Properties Attributes of the RLS algorithm:  No a priori knowledge of signal statistics is required.  Optimization criterion is the (weighted) sum of squared errors. Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 19 Slide 19

Recursive Least Squares (RLS) Algorithm Error Criterion Filter Signal Filter filter Forgetting factor Alternative: Adaptation adaptive algorithm algorithm error: Filter at time n Signal at time l error inserted: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 20 Slide 20

Recursive Least Squares (RLS) Algorithm Derivation – Part 1 Cost function: Differentiate with respect to the complex filter coefficients and setting the result to zero: Definitions: … Estimate for the auto correlation matrix … Estimate for the cross correlation vector Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 21 Slide 21

Recursive Least Squares (RLS) Algorithm Derivation – Part 2 From Simon Haykin, „Adaptive Filter Theory“, Prentice Hall, 2002: or: The Matrix Cookbook [ http://matrixcookbook.com ] Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 22 Slide 22

Recursive Least Squares (RLS) Algorithm Derivation – Part 3 Filter filter Adaptation adaptive algorithm algorithm Inserting the results leads to: „Wiener solution“ … assuming that the auto correlation matrix is invertible Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 23 Slide 23

Recursive Least Squares (RLS) Algorithm Recursion – Part 1 Recursion of the auto correlation matrix over time: Recursion of the cross correlation vector over time: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 24 Slide 24

Recursive Least Squares (RLS) Algorithm Recursion – Part 2 Recursion for the auto correlation matrix: Matrix Inversion Lemma: Inserting the Lemma in the recursion: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 25 Slide 25

Recursive Least Squares (RLS) Algorithm Recursion – Part 3 Recursion for the auto correlation matrix: Definition of a gain vector: Inserting this definition leads to: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 26 Slide 26

Recursive Least Squares (RLS) Algorithm Recursion – Part 4 Definition of a gain factor: Multiplication by the denominator on the right hand side leads to: Rewriting leads to: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 27 Slide 27

Recursive Least Squares (RLS) Algorithm Recursion – Part 5 Recursion of the filter coefficient vector: Step from n to n+1: Reducing the right hand side: Inserting the recursion of the cross correlation vector leads to: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 28 Slide 28

Recursive Least Squares (RLS) Algorithm Recursion – Part 6 What we have so far: If we insert the recursive computation of the inverse auto correlation matrix we obtain: Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 29 Slide 29

Recursive Least Squares (RLS) Algorithm Recursion – Part 7 What we have so far: Inserting according to results in Gain factor Error: old filter with new data Digital Signal Processing and System Theory| Adaptive Filters | Algorithms – Part 1 Slide 30 Slide 30