CTP431- Music and Audio Computing Audio Signal Processing (Part #1) - PowerPoint PPT Presentation

CTP431- Music and Audio Computing Audio Signal Processing (Part #1) Graduate School of Culture Technology KAIST Juhan Nam 1

Types of Audio Signal Processing § Amplitude – Gain, fade in/out, automation curve, compressor § Timbre – EQ, distortion, modulation, chorus, flanger § Spatial effect – Delay, reverberation § Pitch – Pitch shifting (e.g. auto-tune) § Duration – Time stretching § Playback Rate Conversion (resampling) – pitch-shifting /time-stretching / timbre change 2

Filter Output Input Filter 𝑦 𝑜 𝑧 𝑜 § A system that takes signals and modifies them in some way § In particular, we are interested in digital filters that takes discrete number sequences as input and output 3

Basic Operations in Digital Filters Output Input Filter 𝑦 𝑜 𝑧 𝑜 § Multiplication: 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 § Delaying: 𝑧 𝑜 = 𝑦 𝑜 − 1 § Summation: 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 1 4

Linear Time-Invariant (LTI) Digital Filters Output Input Filter 𝑦 𝑜 𝑧 𝑜 § Linearity – Homogeneity: if 𝑦 𝑜 → 𝑧 𝑜 , then a ' 𝑦 𝑜 → a ' 𝑧 𝑜 – Superposition: if 𝑦 - 𝑜 → 𝑧 - 𝑜 and 𝑦 . 𝑜 → 𝑧 . (n) , then 𝑦 - 𝑜 + 𝑦 . 𝑜 → 𝑧 - 𝑜 + 𝑧 . 𝑜 § Time-Invariance – If 𝑦 𝑜 → 𝑧 𝑜 , then 𝑦 𝑜 − 𝑂 → 𝑧 𝑜 − 𝑂 for any 𝑂 – This means that the system does not change its behavior over time 5

LTI Digital Filters § A LTI digital filters performs a combination of the three operations – 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑐 - ' 𝑦 𝑜 − 1 + 𝑐 . ' 𝑦 𝑜 − 2 + ⋯ + 𝑐 2 ' 𝑦 𝑜 − 𝑁 § This is a general form of Finite Impulse Response (FIR) filter 6

The Simplest Lowpass Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 𝑦(𝑜 − 1) § Signal flow graph + 𝑦 𝑜 𝑧 𝑜 𝑨 7- “Delay Operator” 7

The Simplest Lowpass Filter: Sine-Wave Analysis § Measure the amplitude and phase changes given a sinusoidal signal input 8

The Simplest Lowpass Filter: Frequency Response § Plot the amplitude and phase change over different frequency – The frequency sweeps from 0 to the Nyquist rate 9

The Simplest Lowpass Filter: Frequency Response § Mathematical approach – Use complex sinusoid as input: 𝑦 𝑜 = 𝑓 9:; – Then, the output is: 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 1 = 𝑓 9:; + 𝑓 9:(;7-) = 1 + 𝑓 79: ' 𝑓 9:; = 1 + 𝑓 79: ' 𝑦(𝑜) = 𝑓 9 > ? + 𝑓 79 > ? 𝑓 79 > . )𝑓 79 > : – Frequency response: 𝐼 𝜕 = 1 + 𝑓 79: ? = 2cos ( ? : – Amplitude response: 𝐼(𝜕) = 2 cos . : – Phase response: ∠𝐼 𝜕 = − . 10

The Simplest Highpass Filter § Difference equation: 𝑧 𝑜 = 𝑦 𝑜 − 𝑦(𝑜 − 1) Frequency response § 11

Impulse Response Output Input Filter 𝑧 𝑜 = ℎ(𝑜) 𝑦 𝑜 = 𝜀 𝑜 ℎ 𝑜 § The filter output when the input is a unit impulse – 𝑦 𝑜 = 𝜀 𝑜 = 1, 0, 0, 0, … → 𝑧 𝑜 = ℎ(𝑜) § Characterizes the digital system as a sequence of numbers – A system is represented just like audio samples! 12

Examples: Impulse Response § The simplest lowpass filter – h 𝑜 = 1, 1 § The simplest highpass filter – h 𝑜 = 1, −1 § Moving-average filter (order=5) - - - - - – h 𝑜 = J , J , J , J , J § General FIR Filter – h 𝑜 = 𝑐 & , 𝑐 - , 𝑐 . , … , 𝑐 2 à A finite length of impulse response 13

Convolution § The output of LTI digital filters is represented by convolution operation between 𝑦 𝑜 and ℎ 𝑜 2 𝑧 𝑜 = 𝑦 𝑜 ∗ ℎ 𝑜 = P 𝑦(𝑗) ' ℎ(𝑜 − 𝑗) MN& § Deriving convolution – The input can be represented as a time-ordered set of weighted impulses • 𝑦 𝑜 = 𝑦 & , 𝑦 - , 𝑦 . , … , 𝑦 2 = 𝑦 & ' 𝜀 𝑜 + 𝑦 - ' 𝜀 𝑜 − 1 + 𝑦 . ' 𝜀 𝑜 − 2 + ⋯ + 𝑦 2 ' 𝜀 𝑜 − 𝑁 – By the linearity and time-invariance 2 • 𝑧 𝑜 = 𝑦 & ' ℎ 𝑜 + 𝑦 - ' ℎ 𝑜 − 1 + 𝑦 . ' ℎ 𝑜 − 2 + ⋯ + 𝑦 2 ' ℎ 𝑜 − 𝑁 = ∑ 𝑦(𝑗) ' ℎ(𝑜 − 𝑗) MN& 14

Convolution In Practice § The practical expression of convolution 2 2 𝑧 𝑜 = 𝑦 𝑜 ∗ ℎ 𝑜 = P 𝑦 𝑗 ' ℎ 𝑜 − 𝑗 = P ℎ(𝑗) ' 𝑦(𝑜 − 𝑗) MN& MN& – This represents input 𝑦 𝑜 as a streaming data to the filter ℎ 𝑜 § Matlab Animation Demo – http://mac.kaist.ac.kr/~juhan/ctp431/convolution_demo.html § The length of convolution output – If the length of 𝑦 𝑜 is M and the length of ℎ 𝑜 is N , the length of 𝑧 𝑜 is M+N-1 15

Feedback Filter + Output Input Filter 𝑧 𝑜 𝑦 𝑜 Delay § LTI digital filters allow to use the past outputs as input – Past outputs: 𝑧 𝑜 − 1 , 𝑧 𝑜 − 2 , … , 𝑧 𝑜 − 𝑂 § The whole system can be represented as – 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑏 - ' 𝑧 𝑜 − 1 + 𝑏 . ' 𝑧 𝑜 − 2 + ⋯ + 𝑏 R ' 𝑧 𝑜 − 𝑂 – This is a general form of Infinite Impulse Response (IIR) filter 16

A Simple Feedback Lowpass Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 1) § Signal flow graph + 𝑦 𝑜 𝑧 𝑜 𝑨 7- 𝑏 – When 𝑏 is slightly less than 1, it is called “Leaky Integrator” 17

A Simple Feedback Lowpass Filter: Impulse Response § Impulse response – 𝑧 0 = 𝑦 0 = 1 – 𝑧 1 = 𝑦 1 + 𝑏 ' 𝑧 0 = 𝑏 – 𝑧 2 = 𝑦 2 + 𝑏 ' 𝑧 1 = 𝑏 . – … – 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧 𝑜 − 1 = 𝑏 ; § Stability! – If 𝑏 < 1 , the filter output converges (stable) – If 𝑏 = 1 , the filter output oscillates (critical) – If 𝑏 > 1 , the filter output diverges (unstable) 18

A Simple Feedback Lowpass Filter: Frequency Response § More dramatic change than the simplest lowpass filter (FIR) – Phase response is not linear 𝑧 𝑜 = 𝑦 𝑜 + 0.9 ' 𝑧(𝑜 − 1) 19

Reson Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 2𝑠 ' cos𝜄 ' 𝑧 𝑜 − 1 − 𝑠 . ' 𝑧 𝑜 − 2 § Signal flow graph + 𝑧 𝑜 𝑦 𝑜 + 𝑨 7- 2𝑠 ' cos𝜄 + 𝑨 7- −𝑠 . 20

Reson Filter: Frequency Response § Generate resonance at a particular frequency – Control the peak height by 𝑠 and the peak frequency by 𝜄 For stability: 𝑠 < 1 21

Comb Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 𝑁 (Feedforward) 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 𝑂) (Feedback) § Signal flow graph + + 𝑦 𝑜 𝑧 𝑜 𝑦 𝑜 𝑧 𝑜 𝑨 72 𝑨 7R 𝑏 Feedback Feedforward 22

Comb Filter: Frequency Response § ”Combs” become shaper in the feedback type 𝑧 𝑜 = 𝑦 𝑜 + 𝑦(𝑜 − 8) 𝑧 𝑜 = 𝑦 𝑜 + 0.9 ' 𝑧(𝑜 − 8) 23

Perception of Time Delay § The 30 Hz transition – Given a repeated click sound (e.g. impulse train): • If the rate is less than 30Hz, they are perceived as discrete events. • As the rate is above 30 Hz, they are perceive as a tone – Demo: http://auditoryneuroscience.com/?q=pitch/click_train § Feedback comb filter: 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 𝑂) [ \ – If N < ]& ( 𝐺 _ : sampling rate): models sound propagation and reflection with energy loss on a string ( Karplus-strong model ) [ \ – If N < ]& ( 𝐺 _ : sampling rate): generate a looped delay 24

General Filter Form § The general form of digital Filters – 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑐 - ' 𝑦 𝑜 − 1 + 𝑐 . ' 𝑦 𝑜 − 2 + … + 𝑐 R ' 𝑦 𝑜 − 𝑁 +𝑏 - ' 𝑧 𝑜 − 1 + 𝑏 . ' 𝑧 𝑜 − 2 + ⋯ + 𝑏 R ' 𝑧 𝑜 − 𝑂 y(n) + x(n) b 0 Z -1 Z -1 x(n-1) y(n-1) -a 1 b 1 Z -1 Z -1 x(n-2) y(n-2) . . . . . . b 2 -a 2 Z -1 Z -1 y(n-N) x(n-M) -a N b M 25

Frequency Response § Sine-wave Analysis – 𝑦 𝑜 = 𝑓 9:; à 𝑦 𝑜 − 𝑛 = 𝑓 9:(;7a) = 𝑓 79:a 𝑦 𝑜 for any 𝑛 – Let’s assume that 𝑧 𝑜 = 𝐻 𝜕 𝑓 9(:;cd : ) à 𝑧 𝑜 − 𝑛 = 𝑓 79:a 𝑧 𝑜 for any 𝑛 § Putting this into the different equation 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑐 - ' 𝑓 79: ' 𝑦 𝑜 + 𝑐 . ' 𝑓 79.: ' 𝑦 𝑜 + … + 𝑐 2 ' 𝑓 79:2 ' 𝑦 𝑜 +𝑏 - ' 𝑓 79: ' 𝑧 𝑜 + 𝑏 . ' 𝑓 79.: ' 𝑧 𝑜 + ⋯ + 𝑏 R ' 𝑓 79:R ' 𝑧 𝑜 𝑧 𝑜 = 𝑐 & + 𝑐 - ' 𝑓 79: + 𝑐 . ' 𝑓 79.: + … + 𝑐 2 ' 𝑓 79:2 1 + 𝑏 - ' 𝑓 79: + 𝑏 . ' 𝑓 79.: + … + 𝑏 R ' 𝑓 79:R 𝑦(𝑜) 𝐼(𝜕) : frequency response 𝐼(𝜕) = 𝑐 & + 𝑐 - ' 𝑓 79: + 𝑐 . ' 𝑓 79.: + …+ 𝑐 2 ' 𝑓 79:2 𝐻 𝜕 = 𝐼(𝜕) : amplitude response 1 + 𝑏 - ' 𝑓 79: + 𝑏 . ' 𝑓 79.: + …+ 𝑏 R ' 𝑓 79:R 𝜄 𝜕 = ∠𝐼(𝜕) : phase response 26

Z-Transform § 𝑎 -transform – Define z to be a variable in complex plane: we call it z -plane – When z = e j ω (on unit circle), the frequency response is a particular case of the following form 𝐵(𝑨) = 𝑐 & + 𝑐 - ' 𝑨 7- + 𝑐 . ' 𝑨 7. + …+ 𝑐 2 ' 𝑨 72 𝐼 𝑨 = 𝐶(𝑨) 1 + 𝑏 - ' 𝑨 7- + 𝑏 . ' 𝑨 7. + …+ 𝑏 R ' 𝑨 7R – We call this 𝑨 -transform or the transfer function of the filter – z -1 corresponds to one sample delay: delay operator or delay element – Filters are often expressed as 𝑨 -transform: polynomial of 𝑨 7- 27