Tracking Perform ance of the MMax Conjugate Gradient Algorithm Bei - PowerPoint PPT Presentation

Tracking Perform ance of the MMax Conjugate Gradient Algorithm Bei Xie and Tam al Bose Wireless@VT Bradley Dept. of Electrical and Computer Engineering Virginia Tech

Outline  Motivation  Background  Conjugate Gradient (CG) Algorithm  Partial Update Methods  Partial Update CG Algorithm  Tracking Performance Analysis  Simulations  Summary

Motivation  How to reduce the computational complexity of an adaptive filter? Solutions: Using partial update (PU) methods.  What are the tracking performance by using partial update methods to CG?

Conjugate Gradient Algorithm  Solve system with form w R b Equivalent to find a w to minimize the cost function   1   T T J n n n n w ( ) w ( ) Rw ( ) w ( ) b 2 The gradient of the cost function is:      J n n w ( ) ( ) w R b w The residual vector is defined as:       n J n n w g ( ) w ( ) b R ( ) w

A line search method for minimizing the cost function has the form:     n n n w ( ) w ( 1 ) p ( ) p (n) is the direction vector CG chooses the direction is conjugately orthogonal to the previous directions  ,  T n m n m ( Rp ) ( ) 0 p

      J n n Now we find to minimize w ( 1 ) p ( )       J n n n w ( 1 ) ( ) p ( )          T T n n n n n R w p p p ( ) ( 1 ) ( ) ( ) ( ) b 0      T T n n n n R p ( ) b w ( 1 ) p ( ) g ( 1 )    n ( ) T T n n n n R R ( ) ( ) ( ) ( ) p p p p The residual vector is also equal to          n n n n n g ( ) b R w ( ) b R w ( 1 ) ( ) p ( )     n n n g ( 1 ) ( ) Rp ( ) The residual vector is orthogonal to the previous direction vectors,  ,  T n m n m ( ) ( ) 0 g p

CG chooses the direction in the form of    n n β n n p ( 1 ) g ( ) ( ) p ( ) Use Polak-Ribière (PR) method,   T   n n n ( ) ( 1 ) ( ) g g g  β n ( )   T n n g ( 1 ) g ( 1 ) PR method is chosen because it is a non-reset method and performs better for non-constant matrix R CG with PR method usually converges faster than Fletcher- Reeves (FR) method

CG Algorithm in adaptive filter system A basic adaptive filter system model is   T d n n v n * w ( ) ( ) ( ) x d(n) is the desired signal x (n)=[x(n),x(n-1),…,x(n-N+1)] T is the input data vector of an unknown system * ] T is the impulse response w * =[w 1 * ,w 2 * ,…,w N vector of the unknown system w * is constant for time-invariant system w * changes for time-varying sytem v(n) is a white noise

 To estimate the R and b in w R b The exponentially decaying data window is used n         n i T T n i i n n n R ( ) x ( ) x ( ) R ( 1 ) x ( ) x ( )  i 0 n         n i n d i i n d n n b ( ) ( ) x ( ) b ( 1 ) ( ) x ( )  i 0 λ is the forgetting factor

The CG algorithm in an adaptive filter system is summarized as: Initial conditions: w (0)= 0 , R (0)= 0 , p (1)= g (0)     T n n n n ( ) ( 1 ) ( ) ( ) R R x x  T n n p ( ) g ( 1 )          η is used to guarantee n ( ) , 0 . 5 T n n n R p ( ) ( ) p ( ) convergence     n n n ( ) ( 1 ) ( ) w w p             T n n n n n n n n n d n n n w g ( ) b ( ) R ( ) ( ) g ( 1 ) ( ) R ( ) p ( ) x ( ) ( ) x ( ) w ( 1 )   T   n n n g ( ) g ( 1 ) g ( )  β n ( )   T n n g ( 1 ) g ( 1 )    n n β n n ( 1 ) ( ) ( ) ( ) p g p

Partial Update (PU) Methods  Update part of the weights to save the computational complexity  Each update step, update M<N coefficients  Basic PU methods include periodic, sequential, stochastic, and MMax methods The periodic method: update the  weights at every S th iteration and copy the weights at the other iterations, where   N  M S    

The sequential method: choose the  subset of the weights in a round-robin fashion. The stochastic method: is a randomized  version of the sequential method. Usually a uniformly distributed random process will be applied. The MMax method: the elements of the  weight w are updated according to the position of the M largest elements of the input vector x (n).

Partial Update CG Algorithm The partial update CG algorithm in an adaptive filter system is summarized as:     T ˆ ˆ n n n n ˆ ˆ ( ) ( 1 ) ( ) ( ) R R x x  T n n p ( ) g ( 1 )          n ( ) , 0 . 5 T ˆ n n n p ( ) R ( ) p ( )     n n n ( ) ( 1 ) ( ) w w p           ˆ T n n n n n n d n n n ˆ g ( ) g ( 1 ) ( ) R ( ) p ( ) x ( ) ( ) x ( ) w ( 1 )   T   n n n g ( ) g ( 1 ) g ( )  β n ( )   T n n g ( 1 ) g ( 1 )    n n β n n p ( 1 ) g ( ) ( ) p ( )

 n n n ˆ ( ) ( ) ( ) x I x M   i n  ( ) 0 0 1   i n   0 ( )    n 2 ( ) I   M    0   i n    0 0 ( ) N N      i n M i n ( ) , ( ) 0 , 1 k k  k 1 The number of multiplications of CG is 3N 2 +10N+3 per sample The number of multiplications of PU CG is 2N 2 +M 2 +9N+M+3 per sample

The MMax method: the elements of the input x are chosen according to the position of the M largest elements of the input vector x (n)     if n n M 1 x ( ) max x ( ) ,    k l N l i n  1 ( ) k otherwise  0 SORTLINE method is used for comparison Mmax CG needs 2 + 2 log 2 N comparisons

Tracking Performance Analysis of PU CG Desired signal becomes:  x  T d n n n v n * ( ) ( ) w ( ) ( ) Time-varying system w * (n) uses a first-order Markov model      n n n * * w ( ) w ( 1 ) ( )  is very close to unity  ( n is process noise )

Assumptions: • The coefficient error w (n)- w *(n) is small and independent of the input signal x (n) at steady state • White noise v(n) is independent of the input signal x (n) and is independent of process noise η (n) • Input signal x (n) is independent of both v(n) and η (n)

At steady state, the MSE of PU CG with correlated input is     2 2   T E e n E d n n n ( ) ( ) x ( ) w ( )        2 1 ~ ~            T ˆ tr 2 2 1 R R R R R      v v     2   1 1      T E n n R x ( ) x ( )   ~ ~     T E n n n ˆ R x ( ) x ( ) ( 1 ) R ( ) ~ ~     T n n n n ˆ R ( ) R ( 1 ) x ( ) x ( )    ˆ T E n n ˆ ˆ R x ( ) x ( )

At steady state, the MSE of PU CG with white input is     N ( 1 )  2        E e n 2 2 2 2 4 ( ) ~ v v x x x   ˆ 1  2   tr 2 ( R )  x   2 1   tr 2 ( R ) x   ˆ tr 2 ( R ) x ˆ ~   tr 2 ( R ) ~ x   Variance of noise   E v n 2 2 ( ) v

For MMax method and white input,       2 2 2 2 , ~ x x x x ˆ κ <1, κ is close to 1      N 2 ( 1 ) 2       E e n tr 2 2 2 ( ) ( R )    v v x      2 1 1 For white process noise η (n)      N 2 ( 1 ) 2        E e n 2 2 2 2 ( )    v v x      2 1 1

Simulations System identification model The initial impulse response of unknown system is 16-order (N=16) FIR filter w*(n)=[0.01,0.02,-0.04,-0.08,0.15,-0.3,0.45, 0.6, 0.6,0.45,-0.3,0.15,-0.08,-0.04,0.02,0.01] T

The variance of the input noise σ v 2 =0.0001 Parameter λ =0.9 and η =0.6 White input, variance is 1  in Markov model is 0.9998

Tracking Performance of MMax CG 30 30 MMax CG, M=8,   =0.0001 MMax CG, M=4,   =0.0001 20 20 MMax CG, M=8,   =0.001 MMax CG, M=4,   =0.001 MMax CG, M=8,   =0.01 MMax CG, M=4,   =0.01 10 10 0 0 MSE(dB) MSE(dB) -10 -10 -20 -20 -30 -30 -40 -40 -50 -50 0 500 1000 1500 2000 2500 3000 3500 4000 0 500 1000 1500 2000 2500 3000 3500 4000 samples samples Comparison of MSE of Comparison of MSE of MMax CG for varying MMax CG for varying process noise η , M=4 process noise η , M=8