EI331 Signals and Systems Lecture 11 Bo Jiang John Hopcroft Center - PowerPoint PPT Presentation

EI331 Signals and Systems Lecture 11 Bo Jiang John Hopcroft Center for Computer Science Shanghai Jiao Tong University April 2, 2019

Contents 1. Mean-square Convergence of Fourier Series 2. Pointwise Convergence of Fourier Series 3. Fourier Series of Periodic Impulse Train 4. Filtering 1/34

Best Mean-square Approximation Given orthonormal system { e 1 , . . . , e n } , best mean-square approximation of x by element in span { e 1 , . . . , e n } , i.e. by � n element of form y = a k e k is k = 1 � n y = � x , e k � e k , k = 1 with minimum mean-square error � n � x − y � 2 = � x � 2 − |� x , e k �| 2 . k = 1 2/34

Geometry of Best Mean-square Approximation Orthogonal projection onto subspace span { e 1 , . . . , e n } Pythagorean theorem � n � x � 2 = |� x , e k �| 2 + � x − y � 2 x k = 1 x − y � x , e 2 � e 2 e 2 y e 1 O � x , e 1 � e 1 span { e 1 , e 2 } 3/34

Convergence in Normed Vector Space In normed vector space ( V , � · � ) , sequence { x n } converges to x ∈ V if n →∞ � x n − x � = 0 lim We say x is the limit of { x n } and write x = lim n →∞ x n , x n → x , as n → ∞ . or Sequence { x n } ⊂ V is Cauchy sequence iff � x n − x m � → 0 , as n , m → ∞ . Theorem. Every convergent sequence is Cauchy. Proof. If x n → x , triangle inequality yields � x n − x m � ≤ � x n − x � + � x m − x � → 0 , as n , m → ∞ . 4/34

Banach Space Normed vector space ( V , � · � ) is complete if every Cauchy sequence converges. Complete normed vector space is called Banach space. Examples of Banach spaces • ( R n , � · � p ) , ( C n , � · � p ) • DT signal space ℓ p = { x ∈ C Z : � x � p < ∞} with ℓ p norm • CT signal space L p = { x ∈ C R : � x � p < ∞} with L p norm • Space L 2 ( T ) of CT signals with period T and finite � average power, � x � 2 = 1 T | x ( t ) | 2 dt < ∞ T • Space C ( T ) of continuous T -periodic CT signals with L ∞ norm � x � ∞ = sup | x ( t ) | = sup | x ( t ) | . t ∈ [ 0 , T ] t ∈ R x n → x in L ∞ norm means uniform convergence 5/34

Banach Space Examples of incomplete spaces • ( Q , | · | ) . Its completion is ( R , | · | ) • Space C ( T ) of continuous T -periodic CT signals with L 2 norm. Its completion is L 2 ( T ) ◮ periodic odd function with period T = 1  0 ≤ t ≤ 1  nt ,  n n ≤ 1 1 2 − 1 x n = 1 ,  n  1 2 − 1 n ≤ t ≤ 1 1 − nt , 2 ◮ x n ∈ C ( 1 ) ◮ x n converges to periodic square wave in L 2 norm ◮ but periodic square function not in C ( 1 ) Same vector space with different norms becomes different normed vector spaces, e.g ( C ( T ) , � · � ∞ ) vs. ( C ( T ) , � · � 2 ) 6/34

Incompleteness of C ( T ) with L 2 Norm x n ( t ) 1 x n t 1 1 n − 1 x ( t ) 1 x t 1 − 1 �� � � 1 1 2 + 1 | x n ( t ) − x ( t ) | 2 dt ≤ 4 n n � x n − x � 2 2 = + n → 0 − 1 1 2 − 1 n n x ∈ L 2 ( 1 ) but x / ∈ C ( 1 ) . Also � x n − x � ∞ = 1 , ∀ n 7/34

L 2 Convergence vs. Pointwise Convergence 1 n = 0 , k = 0 For n ≥ 0 and 0 ≤ k < 2 n , t 1 � 1 2 n ≤ t < k + 1 k n = 1 , k = 0 1 , 2 n x 2 n + k = t 0 , otherwise 1 1 n = 1 , k = 1 t 1 • x 2 n + k → 0 in L 2 norm 1 n = 2 , k = 0 1 t 1 � x 2 n + k − 0 � 2 = 2 n / 2 → 0 1 n = 2 , k = 1 t 1 • not convergent at any t 0 ∈ [ 0 , 1 ) 1 n = 2 , k = 2 ◮ ∀ n , ∃ k 1 s.t x 2 n + k 1 ( t 0 ) = 1 t ◮ ∀ n , ∃ k 0 s.t x 2 n + k 0 ( t 0 ) = 0 1 1 n = 2 , k = 3 t 1 8/34

Hilbert Space Inner product space ( V , �· , ·� ) is also normed vector space � with induced norm � x � = � x , x � . ( V , �· , ·� ) is complete if ( V , � · � ) with induced norm is complete. Complete inner produce space is called Hilbert space. Examples of Hilbert spaces • R n with � x , y � = � n k = 1 x k y k ; C n with � x , y � = � n k = 1 x k ¯ y k • Space ℓ 2 with � x , y � = � ∞ k = −∞ x k ¯ y k � • Space L 2 with � x , y � = R x ( t ) y ( t ) dt � • Space L 2 ( T ) with � x , y � = 1 T x ( t ) y ( t ) dt T Example of incomplete space � • Space C ( T ) with � x , y � = 1 T x ( t ) y ( t ) dt T L 2 ( T ) is completion of C ( T ) . 9/34

Complete Orthonormal Sequence Orthonormal sequence { e k : k ∈ N } in Hilbert space H is complete if every x ∈ H has expansion � ∞ x = � x , e k � e k k = 1 i.e. � n n →∞ � x − lim � x , e k � e k � = 0 k = 1 Complete orthonormal sequence also called orthonormal basis of H . Example. { δ k : k ∈ Z } is orthonormal basis of ℓ 2 10/34

Parseval’s Identity If { e k : k ∈ N } orthonormal basis of H , � ∞ � x � 2 = |� x , e k �| 2 k = 1 Proof. By Pythagorean theorem � n � n � x � 2 = |� x , e k �| 2 + � x − � x , e k � e k � 2 k = 1 k = 1 Let n → ∞ and last term goes to zero by definition of orthonormal basis. 11/34

Parseval’s Identity H is isomorphic to ℓ 2 , x ↔ {� x , e k � : k ∈ N } and � ∞ � x , y � = � x , e k �� y , e k � k = 1 Proof. � � � x , y � = � � x , e k � e k , � y , e m � e m � k m � � = � x , e k � � y , e m �� e k , e m � k m � � = � x , e k � � y , e m � δ [ k − m ] k m � = � x , e k �� y , e k � k NB. When x = y , we recover � x � 2 = � ∞ k = 1 |� x , e k �| 2 12/34

Mean-square Convergence of Fourier Series T t : k ∈ Z } is orthonormal basis of L 2 ( T ) . Theorem. { e jk 2 π For any x ∈ L 2 ( T ) , Fourier series converges in mean-square, i.e. in L 2 norm N →∞ � x − S N ( x ) � = 0 lim NB. Convergence in mean-square does not imply pointwise convergence Parseval’s identity � � ∞ x [ k ] | 2 = � x � 2 = 1 | x ( t ) | 2 dt | ˆ T T k = −∞ Interpretation: Energy conservation x [ k ] | 2 is average power of k -th harmonic component • | ˆ • � x � 2 is average power of x 13/34

Fourier Series of L 2 Signals Correspondence between periodic functions and doubly infinite sequences; time domain vs. frequency domain FS FS ← − − → ˆ or x ( t ) ← − − → ˆ x [ k ] x x Fourier series is isomorphism between Hilbert space of L 2 signals and Hilbert space of ℓ 2 Fourier coefficients FS : L 2 ( T ) → ℓ 2 x �→ ˆ x Will see discrete-time Fourier transform goes from ˆ x to x Parseval’s theorem � x , y � L 2 ( T ) = � ˆ x , ˆ y � ℓ 2 14/34

Contents 1. Mean-square Convergence of Fourier Series 2. Pointwise Convergence of Fourier Series 3. Fourier Series of Periodic Impulse Train 4. Filtering 15/34

Integral Representation of Partial Sum WLOG, focus on T = 2 π and ω 0 = 1 N � x [ k ] e jkt S N ( x )( t ) = ˆ k = − N � 1 � � � N x ( τ ) e − jk τ d τ e jkt = 2 π 2 π k = − N � � N = 1 e jk ( t − τ ) d τ x ( τ ) 2 π 2 π k = − N � = 1 x ( τ ) D N ( t − τ ) d τ = ( x ∗ D N 2 π )( t ) 2 π 2 π where D N is Dirichlet kernel � N e jkt = sin(( N + 1 2 ) t ) D N ( t ) = sin( t / 2 ) k = − N 16/34

Dirichlet Kernel Plot of D N 2 π for various N t t t N = 4 N = 8 N = 16 If D N 2 π → δ (more precisely, periodic impulse train on slide 22), then would have S N ( x ) = x ∗ D N 2 π → x ∗ δ = x at least for continuous x Unfortunately, exists continuous x whose Fourier partial sum S N ( x ) , as N → ∞ , fails to converge at all at some t , let alone converges to x ( t ) at such t 17/34

Pointwise Convergence Theorem. If x is differentiable, then lim N →∞ S N ( x )( t ) = x ( t ) , ∀ t . Proof. Fix t . Define  x ( t − τ ) − x ( t )  , τ � = 0 F ( τ ) = sin( τ/ 2 )  − 2 x ′ ( t ) , τ = 0 � π 1 which is continuous on [ − π, π ] . Note − π D N ( τ ) d τ = 1 . 2 π � π � π S N ( x )( t ) − x ( t ) = 1 x ( t − τ ) D N ( τ ) d τ − 1 x ( t ) D N ( τ ) d τ 2 π 2 π − π − π � π = 1 [ x ( t − τ ) − x ( t )] D N ( τ ) d τ 2 π − π � π � � = 1 ( N + 1 F ( τ ) sin 2 ) τ d τ 2 π − π 18/34

Pointwise Convergence Theorem. If x is differentiable, then lim N →∞ S N ( x )( t ) = x ( t ) . Proof (cont’d). Define y ( τ ) = e − j τ 2 F ( τ ) . � π � � S N ( x )( t ) − x ( t ) = 1 ( N + 1 F ( τ ) sin 2 ) τ d τ 2 π − π � π = − Im 1 y ( τ ) e − jN τ d τ 2 π − π = − Im ˆ y [ N ] Being continuous on [ − π, π ] , y ( τ ) ∈ L 2 ( 2 π ) . Bessel’s inequality implies ˆ y [ N ] → 0 , so S N ( x )( t ) − x ( t ) = − Im ˆ y [ N ] → 0 , as N → ∞ 19/34

Dirichlet Test Theorem. If x satisfies following Dirichlet conditions 1. x is bounded, i.e. � x � ∞ < ∞ , 2. x is piecewise monotone, i.e. ∃ finite partition of a period s.t. x is monotone on each segment, then N →∞ S N ( x )( t ) = x ( t + ) + x ( t − ) lim 2 Example. x ( t ) 1 t 1 − 1  t ∈ ( n , n + 1  1 , 2 ) ,  N →∞ S N ( x )( t ) = lim t ∈ ( n − 1 , n ∈ Z − 1 , 2 , n ) ,   t = n 0 , 2 , 20/34

Contents 1. Mean-square Convergence of Fourier Series 2. Pointwise Convergence of Fourier Series 3. Fourier Series of Periodic Impulse Train 4. Filtering 21/34