Systems Prof. Seungchul Lee Industrial AI Lab. Most slides from - PowerPoint PPT Presentation

Systems Prof. Seungchul Lee Industrial AI Lab. Most slides from (edX) Discrete Time Signals and Systems by Prof. Richard G. Baraniuk 1

Systems • A discrete-time system 𝐼 is a transformation (a rule or formula) that maps a discrete- time input signal 𝑦 into a discrete-time output signal 𝑧 2

Example: Systems 3

Linear Systems • A system 𝐼 is linear if it satisfies the following two properties: – Scaling: – Additivity: 4

Linear Systems and Matrix Multiplication • Matrix multiplication (aka linear combination) is a fundamental signal processing system • Matrix multiplications are linear systems where ℎ 𝑜,𝑛 = 𝐼 𝑜,𝑛 represents the row- 𝑜 , column- 𝑛 entry of the matrix 𝐼 • All linear systems can be expressed as matrix multiplications 5

Matrix Multiplication and Linear Systems in Pictures • System output as a linear combination of columns • System output as a sequence of inner products 6

Time-Invariant Systems • For infinite-length signals – A system 𝐼 processing infinite-length signals is time-invariant (shift-invariant) if a time shift of the input signal creates a corresponding time shift in the output signal • For finite-length signals – A system 𝐼 processing infinite-length signals is time-invariant (shift-invariant) if a circular time shift of the input signal creates a corresponding circular time shift in the output signal 7

Linear Time-Invariant (LTI) Systems • A system 𝐼 is linear time-invariant (LTI) if it is both linear and time-invariant • We will only consider Linear Time-Invariant (LTI) systems. 8

Matrix Multiplication and LTI Systems (Infinite -Length Signals) • When the linear system is also shift invariant, 𝐼 has a special structure • Linear system for infinite-length signals can be expressed as • Enforcing time invariance implies that for all 𝑟 ∈ ℤ • Change of variables: 𝑜 ′ = 𝑜 − 𝑟 and 𝑛 ′ = 𝑛 − 𝑟 9

Matrix Multiplication and LTI Systems (Infinite -Length Signals) • We see that for an LTI system • Entries on the matrix diagonals are the same - Toeplitz matrix 10

Matrix Multiplication and LTI Systems (Infinite -Length Signals) • All of the entries in a Toeplitz matrix can be expressed in terms of the entries of the – 0-th column – Time-reversed 0-th row • Row- 𝑜 , column- 𝑛 entry of the matrix 11

Matrix Multiplication and LTI Systems (Infinite -Length Signals) • Note the diagonals ! 12

Matrix Multiplication and LTI Systems (Finite-Length Signals) • Linear system for signals of length 𝑂 can be expressed as • Enforcing time invariance implies that for all 𝑟 ∈ ℤ • Change of variables: 𝑜 ′ = 𝑜 − 𝑟 and 𝑛 ′ = 𝑛 − 𝑟 13

Matrix Multiplication and LTI Systems (Finite-Length Signals) • We see that for an LTI system • Entries on the matrix diagonals are the same + circular wraparound - Circulent matrix 14

Matrix Multiplication and LTI Systems (Finite-Length Signals) • All of the entries in a circulent matrix can be expressed in terms of the entries of the – 0-th column – Time-reversed 0-th row • Row- 𝑜 , column- 𝑛 entry of the matrix 15

Matrix Multiplication and LTI Systems (Finite-Length Signals) • Note the diagonals and circulent shifts ! 16

Impulse Response 17

Impulse Response • We will Illustrate it with an infinite-length signal • The 0-th column of the matrix 𝐼 has a special interpretation • Compute the output when the input is a delta function (impulse) • This suggests that we call ℎ the impulse response of the system • Output of system to delta function (impulse) is ℎ . So, We call ℎ the impulse response of the system 18

Impulse Response • From ℎ , we can build matrix 𝐼 – Columns/rows of 𝐼 are the shifted versions of the 0-th column/row – ℎ contains all the information of the LTI system • LTI systems are Toeplitz matrices (infinite-length signals) – Entries on the matrix diagonals are the same • Let the input 𝜀[𝑜] and compute 𝑧[𝑜] • The impulse response characterizes an LTI system (that is, carries all of the information contained in matrix 𝐼 ) 19

Examples (Infinite-Length Signals) • Impulse response of the scaling system 20

Examples (Infinite-Length Signals) • Impulse response of the shift system 21

Examples (Infinite-Length Signals) • Impulse response of the moving average system 22

Examples (Infinite-Length Signals) • Impulse response of the recursive average system 23

Examples (Finite-Length Signals) • Entries on the matrix diagonals are the same + circular wraparound 24

Examples (Finite-Length Signals) • Impulse response of the shift system 25

Examples (Finite-Length Signals) • Impulse response of the moving average system 26

Examples (Finite-Length Signals) • Impulse response of the recursive average system 27

Time Response to Arbitrary Input: Convolution 28

Convolution • Convolution is defined as the integral of the product of the two signals after one is reversed and shifted • Output 𝑧[𝑜] came out by convolution of input 𝑦[𝑜] and system ℎ[𝑜] – Time reverse the impulse response ℎ and shift it 𝑜 time steps to the right (delay) – Compute the inner product between the shifted impulse response and the input vector 𝑦 29

Convolution For Infinite-Length Signals • Toeplitz Matrices • It is an inner product of ℎ vectors and 𝑦 30

Convolution For Infinite-Length Signals • Convolution is product of matrix 𝐼 and 𝑦 31

Convolution using Toeplitz Matrix • LTI systems are Toeplitz matrices (infinite-length signals) – Entries on the matrix diagonals are the same 32

Superposition (Linear) and Time-Invariant • Think about convolution in time – Break input into additive parts and sum the responses to the parts Source: Prof. Denny Freeman at MIT 33

Superposition (Linear) and Time-Invariant • Think about convolution in time – You are standing at time 𝑜 Source: Prof. Denny Freeman at MIT 34

Convolution • If a system is linear and time-invariant (LTI) then its output is the sum of weighted and shifted unit- sample responses. Source: Prof. Denny Freeman at MIT 35

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input Source: Dr. Francois Fleuret at EPFL 36

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output Source: Dr. Francois Fleuret at EPFL 37

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output 7 Source: Dr. Francois Fleuret at EPFL 38

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output 7 9 Source: Dr. Francois Fleuret at EPFL 39

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output 7 9 12 Source: Dr. Francois Fleuret at EPFL 40

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output 7 9 12 2 Source: Dr. Francois Fleuret at EPFL 41

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output 7 9 12 2 -1 Source: Dr. Francois Fleuret at EPFL 42

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input 1 3 0 -1 h[-n] Output 7 9 12 2 -1 0 Source: Dr. Francois Fleuret at EPFL 43

1D Convolution 1 3 2 3 0 -1 1 2 2 1 Input w 1 3 0 -1 h[-n] Output 7 9 12 2 -1 0 6 Source: Dr. Francois Fleuret at EPFL 44

Structure of Convolution Source: Prof. Denny Freeman at MIT 45

Structure of Convolution Source: Prof. Denny Freeman at MIT 46

Structure of Convolution Source: Prof. Denny Freeman at MIT 47

Structure of Convolution Source: Prof. Denny Freeman at MIT 48

Structure of Convolution Source: Prof. Denny Freeman at MIT 49

Structure of Convolution Source: Prof. Denny Freeman at MIT 50

Structure of Convolution Source: Prof. Denny Freeman at MIT 51

Structure of Convolution Source: Prof. Denny Freeman at MIT 52

Structure of Convolution Source: Prof. Denny Freeman at MIT 53

Structure of Convolution Source: Prof. Denny Freeman at MIT 54

Structure of Convolution Source: Prof. Denny Freeman at MIT 55

Structure of Convolution Source: Prof. Denny Freeman at MIT 56

Structure of Convolution Source: Prof. Denny Freeman at MIT 57

Structure of Convolution Source: Prof. Denny Freeman at MIT 58

Graphical Illustration Source: Applied Digital Signal Processing, Theory and Practice 59

Discrete-Time Convolution: Summary • Representing an LTI system by a single signal • Unit-impulse response ℎ[𝑜] is a complete description of an LTI system • Given ℎ[𝑜] , one can compute the response to any arbitrary input signal 𝑦[𝑜] 60

Convolution: Commutative • Convolution is commutative → Signal = System 61

Convolution in MATLAB • For finite-length signals 62

Convolution in MATLAB • For finite-length signals 63

Convolution Function • You have to include conv_m function file in the path 64

Convolution in MATLAB 65