Parametric Signal Modeling and Linear Prediction Theory 5. Lattice - PowerPoint PPT Presentation

5 Lattice Predictor Appendix: Detailed Derivations Parametric Signal Modeling and Linear Prediction Theory 5. Lattice Predictor Electrical & Computer Engineering University of Maryland, College Park Acknowledgment: ENEE630 slides were based on class notes developed by Profs. K.J. Ray Liu and Min Wu. The LaTeX slides were made by Prof. Min Wu and Mr. Wei-Hong Chuang. Contact: minwu@umd.edu. Updated: November 19, 2012. ENEE630 Lecture Part-2 1 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Introduction Recall: a forward or backward prediction-error filter can each be realized using a separate tapped-delay-line structure. Lattice structure discussed in this section provides a powerful way to combine the FLP and BLP operations into a single structure. ENEE630 Lecture Part-2 2 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Order Update for Prediction Errors (Readings: Haykin § 3.8) Review: � � � � u m [ n ] u [ n ] signal vector u m +1 [ n ] = = 1 u [ n − m ] u m [ n − 1] 2 Levinson-Durbin recursion: � a m − 1 � � � 0 a m = + Γ m (forward) a B ∗ 0 m − 1 � a m − 1 � � � 0 a B ∗ m = + Γ ∗ (backward) a B ∗ m 0 m − 1 ENEE630 Lecture Part-2 3 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Recursive Relations for f m [ n ] and b m [ n ] f m [ n ] = a H m u m +1 [ n ]; b m [ n ] = a BT m u m +1 [ n ] 1 FLP: � � � � � � � u m [ n ] � u [ n ] f m [ n ] = a H + Γ ∗ 0 a BT m − 1 0 u [ n − m ] m m − 1 u m [ n − 1] (Details) f m [ n ] = f m − 1 [ n ] + Γ ∗ m b m − 1 [ n − 1] 2 BLP: � � � � � � � � u [ n ] u m [ n ] 0 a BT a ∗ m − 1 0 b m [ n ] = + Γ m m − 1 u m [ n − 1] u [ n − m ] (Details) b m [ n ] = b m − 1 [ n − 1] + Γ m f m − 1 [ n ] ENEE630 Lecture Part-2 4 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Basic Lattice Structure � f m [ n ] � � � � � 1 Γ ∗ f m − 1 [ n ] m = , m = 1 , 2 , . . . , M b m [ n ] Γ m 1 b m − 1 [ n − 1] Signal Flow Graph (SFG) ENEE630 Lecture Part-2 5 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Modular Structure Recall f 0 [ n ] = b 0 [ n ] = u [ n ], thus To increase the order, we simply add more stages and reuse the earlier computations. Using a tapped delay line implementation, we need M separate filters to generate b 1 [ n ] , b 2 [ n ] , . . . , b M [ n ]. In contrast, a single lattice structure can generate b 1 [ n ] , . . . , b M [ n ] as well as f 1 [ n ] , . . . , f M [ n ] at the same time. ENEE630 Lecture Part-2 6 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Correlation Properties Given from a zero-mean w.s.s. Predict process: ⇒ (FLP) { u [ n − 1] , . . . , u [ n − M ] } u [ n ] ⇒ (BLP) { u [ n ] , u [ n − 1] , . . . , u [ n − M + 1] } u [ n − M ] 1 . Principle of Orthogonality i.e., conceptually E [ f m [ n ] u ∗ [ n − k ]] = 0 (1 ≤ k ≤ m ) f m [ n ] ⊥ u m [ n − 1] E [ b m [ n ] u ∗ [ n − k ]] = 0 (0 ≤ k ≤ m − 1) b m [ n ] ⊥ u m [ n ] 2 . E [ f m [ n ] u ∗ [ n ]] = E [ b m [ n ] u ∗ [ n − m ]] = P m Proof : (Details) ENEE630 Lecture Part-2 7 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Correlation Properties 3 . Correlations of error signals across orders: � i = m P m (BLP) E [ b m [ n ] b ∗ i [ n ]] = 0 i < m i.e. , b m [ n ] ⊥ b i [ n ] (FLP) E [ f m [ n ] f ∗ i [ n ]] = P m for i ≤ m Proof : (Details) (can obtain the case i > m by conjugation) Remark : The generation of { b 0 [ n ] , b 1 [ n ] , . . . , } is like a Gram-Schmidt orthogonalization process on { u [ n ] , u [ n − 1] , . . . , } . As a result, { b i [ n ] } i =0 , 1 ,... is a new, uncorrelated representation of { u [ n ] } containing exactly the same information . ENEE630 Lecture Part-2 8 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Correlation Properties 4 . Correlations of error signals across orders and time: E [ f m [ n ] f ∗ i [ n − ℓ ]] = E [ f m [ n + ℓ ] f ∗ i [ n ]] = 0 (1 ≤ ℓ ≤ m − i , i < m ) E [ b m [ n ] b ∗ i [ n − ℓ ]] = E [ b m [ n + ℓ ] b ∗ i [ n ]] = 0 (0 ≤ ℓ ≤ m − i − 1 , i < m ) Proof : (Details) 5 . Correlations of error signals across orders and time: � P m i = m E [ f m [ n + m ] f ∗ i [ n + i ]] = 0 i < m E [ b m [ n + m ] b ∗ i [ n + i ]] = P m i ≤ m Proof : (Details) ENEE630 Lecture Part-2 9 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Correlation Properties 6 . Cross-correlations of FLP and BLP error signals: � Γ ∗ i P m i ≤ m E [ f m [ n ] b ∗ i [ n ]] = 0 i > m Proof : (Details) ENEE630 Lecture Part-2 10 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Joint Process Estimator: Motivation (Readings: Haykin § 3.10; Hayes § 7.2.4, § 9.2.8) In (general) Wiener filtering theory, we use { x [ n ] } process to estimate a desired response { d [ n ] } . Solving the normal equation may require inverting the correlation matrix R x . We now use the lattice structure to obtain a backward prediction error process { b i [ n ] } as an equivalent, uncorrelated representation of { u [ n ] } that contains exactly the same information. We then apply an optimal filter on { b i [ n ] } to estimate { d [ n ] } . ENEE630 Lecture Part-2 11 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Joint Process Estimator: Structure ˆ d [ n | S n ] = k H b M +1 [ n ], where k = [ k 0 , k 1 , . . . , k M ] T ENEE630 Lecture Part-2 12 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Joint Process Estimator: Result To determine the optimal weight to minimize MSE of estimation: 1 Denote D as the ( M + 1) × ( M + 1) correlation matrix of b [ n ] � � b [ n ] b H [ n ] D = E = diag ✿✿✿ ( P 0 , P 1 , . . . , P M ) ∵ { b k [ n ] } M k =0 are uncorrelated 2 Let s be the crosscorrelation vector s � [ s 0 , . . . , s M . . . ] T = E [ b [ n ] d ∗ [ n ]] 3 The normal equation for the optimal weight vector is: Dk opt = s ⇒ k opt = D − 1 s = diag( P − 1 0 , P − 1 1 , . . . , P − 1 M ) s i.e., k i = P − 1 s i , i = 0 , . . . , M i ENEE630 Lecture Part-2 13 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Joint Process Estimator: Summary The name “joint process estimation” refers to the system’s structure that performs two optimal estimation jointly: One is a lattice predictor (characterized by Γ 1 , . . . , Γ M ) transforming a sequence of correlated samples u [ n ], u [ n − 1] , . . . , u [ n − M ] into a sequence of uncorrelated samples b 0 [ n ] , b 1 [ n ] , . . . , b M [ n ]. The other is called a multiple regression filter (characterized by k ), which uses b 0 [ n ] , . . . , b M [ n ] to produce an estimate of d [ n ]. ENEE630 Lecture Part-2 14 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering Inverse Filtering The lattice predictor discussed just now can be viewed as an analyzer, i.e., to represent an (approximately) AR process { u [ n ] } using { Γ m } . With some reconfiguration, we can obtain an inverse filter or a synthesizer, i.e., we can reproduce an AR process by applying white noise { v [ n ] } as the input to the filter. ENEE630 Lecture Part-2 15 / 29

5.1 Basic Lattice Structure 5 Lattice Predictor 5.2 Correlation Properties Appendix: Detailed Derivations 5.3 Joint Process Estimator 5.4 Inverse Filtering A 2-stage Inverse Filtering u [ n ] = v [ n ] − Γ ∗ 1 u [ n − 1] − Γ ∗ 2 (Γ 1 u [ n − 1] + u [ n − 2]) = v [ n ] − (Γ ∗ 1 + Γ 1 Γ ∗ u [ n − 1] − Γ ∗ 2 ) u [ n − 2] 2 � �� � ���� a ∗ a ∗ 2 , 1 2 , 2 ∴ u [ n ] + a ∗ 2 , 1 u [ n − 1] + a ∗ 2 , 2 u [ n − 2] = v [ n ] ⇒ { u [ n ] } is an AR( 2 ) process. ENEE630 Lecture Part-2 16 / 29