CTP431- Music and Audio Computing Digital Audio Effects Graduate - PowerPoint PPT Presentation

CTP431- Music and Audio Computing Digital Audio Effects Graduate School of Culture Technology KAIST Juhan Nam 1

Introduction § Amplitude – Gain, fade in/out, automation curve, compressor § Timbre – Filters, EQ, distortion, modulation, flanger, vocoder § Spatial effect – Delay, Reverberation, panning, binaural (HRTF) § Pitch – Pitch shifting (e.g. auto-tune), Transpose § Time stretching – Timing change, Tempo adjustment Source: http://www.uaudio.com/uad/downloads Source: https://www.izotope.com/en/products/repair-and-edit/rx-post-production-suite.html 2

Digital System Digital Input Output System 𝑦 𝑜 𝑧 𝑜 § Take the input signal 𝑦 𝑜 as a sequence of numbers and returns the output signal 𝑧 𝑜 as another sequence of numbers § We are particularly interested in linear systems that are composed of the following operations – Multiplication: 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 – Delaying: 𝑧 𝑜 = 𝑦 𝑜 − 1 – Summation: 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 1 3

Linear Time-Invariant (LTI) System Digital Input Output System 𝑦 𝑜 𝑧 𝑜 § Linearity – Homogeneity: if 𝑦 𝑜 → 𝑧 𝑜 , then a ' 𝑦 𝑜 → a ' 𝑧 𝑜 – Superposition: if 𝑦 - 𝑜 → 𝑧 - 𝑜 and 𝑦 . 𝑜 → 𝑧 . (n) , then 𝑦 - 𝑜 + 𝑦 . 𝑜 → 𝑧 - 𝑜 + 𝑧 . 𝑜 § Time-Invariance – If 𝑦 𝑜 → 𝑧 𝑜 , then 𝑦 𝑜 − 𝑂 → 𝑧 𝑜 − 𝑂 for any 𝑂 – This means that the system does not change its behavior over time 4

LTI System § LTI systems in frequency domain – No new sinusoidal components are introduced – Only existing sinusoids components changes in amplitude and phase. § Examples of non-LTI systems – Clipping – Distortion – Aliasing – Modulation 5

LTI Digital Filters § A LTI digital filters performs a combination of the three operations – 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑐 - ' 𝑦 𝑜 − 1 + 𝑐 . ' 𝑦 𝑜 − 2 + ⋯ + 𝑐 2 ' 𝑦 𝑜 − 𝑁 § This is a general form of Finite Impulse Response (FIR) filter 6

Two Ways of Defining LTI Systems § By the relation between input 𝑦 𝑜 and output 𝑧 𝑜 – Difference equation – Signal flow graph § By the impulse response of the system – Measure it by using a unit impulse as input – Convolution operation 7

The Simplest Lowpass Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 𝑦(𝑜 − 1) § Signal flow graph + 𝑦 𝑜 𝑧 𝑜 𝑨 7- “Delay Operator” 8

The Simplest Lowpass Filter: Sine-Wave Analysis § Measure the amplitude and phase changes given a sinusoidal signal input 9

The Simplest Lowpass Filter: Frequency Response § Plot the amplitude and phase change over different frequency – The frequency sweeps from 0 to the Nyquist rate 10

The Simplest Lowpass Filter: Frequency Response § Mathematical approach – Use complex sinusoid as input: 𝑦 𝑜 = 𝑓 9:; – Then, the output is: 𝑧 𝑜 = 𝑦 𝑜 + 𝑦 𝑜 − 1 = 𝑓 9:; + 𝑓 9:(;7-) = 1 + 𝑓 79: ' 𝑓 9:; = 1 + 𝑓 79: ' 𝑦(𝑜) – Frequency response: 𝐼 𝜕 = 1 + 𝑓 79: = 𝑓 9 > ? + 𝑓 79 > ? 𝑓 79 > . )𝑓 79 > : ? = 2cos ( ? : – Amplitude response: 𝐼(𝜕) = 2 cos . : – Phase response: ∠𝐼 𝜕 = − . 11

The Simplest Highpass Filter § Difference equation: 𝑧 𝑜 = 𝑦 𝑜 − 𝑦(𝑜 − 1) Frequency response § 12

Impulse Response Input Output Filter 𝑧 𝑜 = ℎ(𝑜) 𝑦 𝑜 = 𝜀 𝑜 ℎ 𝑜 § The filter output when the input is a unit impulse – 𝑦 𝑜 = 𝜀 𝑜 = 1, 0, 0, 0, … → 𝑧 𝑜 = ℎ(𝑜) § Characterizes the digital system as a sequence of numbers – A system is represented just like audio samples! 13

Examples: Impulse Response § The simplest lowpass filter – h 𝑜 = 1, 1 § The simplest highpass filter – h 𝑜 = 1, −1 § Moving-average filter (order=5) - - - - - – h 𝑜 = J , J , J , J , J § General FIR Filter – h 𝑜 = 𝑐 & , 𝑐 - , 𝑐 . , … , 𝑐 2 à A finite length of impulse response 14

Convolution § The output of LTI digital filters is represented by convolution operation between 𝑦 𝑜 and ℎ 𝑜 2 𝑧 𝑜 = 𝑦 𝑜 ∗ ℎ 𝑜 = P 𝑦(𝑗) ' ℎ(𝑜 − 𝑗) MN& § Deriving convolution – The input can be represented as a time-ordered set of weighted impulses • 𝑦 𝑜 = 𝑦 & , 𝑦 - , 𝑦 . , … , 𝑦 2 = 𝑦 & ' 𝜀 𝑜 + 𝑦 - ' 𝜀 𝑜 − 1 + 𝑦 . ' 𝜀 𝑜 − 2 + ⋯ + 𝑦 2 ' 𝜀 𝑜 − 𝑁 – By the linearity and time-invariance 2 • 𝑧 𝑜 = 𝑦 & ' ℎ 𝑜 + 𝑦 - ' ℎ 𝑜 − 1 + 𝑦 . ' ℎ 𝑜 − 2 + ⋯ + 𝑦 2 ' ℎ 𝑜 − 𝑁 = ∑ 𝑦(𝑗) ' ℎ(𝑜 − 𝑗) MN& 15

Convolution In Practice § The practical expression of convolution 2 2 𝑧 𝑜 = 𝑦 𝑜 ∗ ℎ 𝑜 = P 𝑦 𝑗 ' ℎ 𝑜 − 𝑗 = P ℎ(𝑗) ' 𝑦(𝑜 − 𝑗) MN& MN& – This represents input 𝑦 𝑜 as a streaming data to the filter ℎ 𝑜 § The length of convolution output – If the length of 𝑦 𝑜 is M and the length of ℎ 𝑜 is N , the length of 𝑧 𝑜 is M+N-1 16

Demo: Convolution 17

Feedback Filter + Output Input Filter 𝑧 𝑜 𝑦 𝑜 Delay § LTI digital filters allow to use the past outputs as input – Past outputs: 𝑧 𝑜 − 1 , 𝑧 𝑜 − 2 , … , 𝑧 𝑜 − 𝑂 § The whole system can be represented as – 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑏 - ' 𝑧 𝑜 − 1 + 𝑏 . ' 𝑧 𝑜 − 2 + ⋯ + 𝑏 R ' 𝑧 𝑜 − 𝑂 – This is a general form of Infinite Impulse Response (IIR) filter 18

A Simple Feedback Lowpass Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧(𝑜 − 1) § Signal flow graph + 𝑦 𝑜 𝑧 𝑜 𝑨 7- 𝑏 – When 𝑏 is slightly less than 1, it is called “Leaky Integrator” 19

A Simple Feedback Lowpass Filter: Impulse Response § Impulse response – 𝑧 0 = 𝑦 0 = 1 – 𝑧 1 = 𝑦 1 + 𝑏 ' 𝑧 0 = 𝑏 – 𝑧 2 = 𝑦 2 + 𝑏 ' 𝑧 1 = 𝑏 . – … – 𝑧 𝑜 = 𝑦 𝑜 + 𝑏 ' 𝑧 𝑜 − 1 = 𝑏 ; § Stability! – If 𝑏 < 1 , the filter output converges (stable) – If 𝑏 = 1 , the filter output oscillates (critical) – If 𝑏 > 1 , the filter output diverges (unstable) 20

A Simple Feedback Lowpass Filter: Frequency Response § More dramatic change than the simplest lowpass filter (FIR) – Phase response is not linear 𝑧 𝑜 = 𝑦 𝑜 + 0.9 ' 𝑧(𝑜 − 1) 21

Reson Filter § Difference equation 𝑧 𝑜 = 𝑦 𝑜 + 2𝑠 ' cos𝜄 ' 𝑧 𝑜 − 1 − 𝑠 . ' 𝑧 𝑜 − 2 § Signal flow graph + 𝑧 𝑜 𝑦 𝑜 + 𝑨 7- 2𝑠 ' cos𝜄 + 𝑨 7- −𝑠 . 22

Reson Filter: Frequency Response § Generate resonance at a particular frequency – Control the peak height by 𝑠 and the peak frequency by 𝜄 For stability: 𝑠 < 1 23

General Filter Form § The general form of digital Filters – 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑐 - ' 𝑦 𝑜 − 1 + 𝑐 . ' 𝑦 𝑜 − 2 + … + 𝑐 R ' 𝑦 𝑜 − 𝑁 +𝑏 - ' 𝑧 𝑜 − 1 + 𝑏 . ' 𝑧 𝑜 − 2 + ⋯ + 𝑏 R ' 𝑧 𝑜 − 𝑂 y(n) + x(n) b 0 Z -1 Z -1 x(n-1) y(n-1) -a 1 b 1 Z -1 Z -1 x(n-2) y(n-2) . . . . . . b 2 -a 2 Z -1 Z -1 y(n-N) x(n-M) -a N b M 24

Frequency Response § Sine-wave Analysis – 𝑦 𝑜 = 𝑓 9:; à 𝑦 𝑜 − 𝑛 = 𝑓 9:(;7Z) = 𝑓 79:Z 𝑦 𝑜 for any 𝑛 – Let’s assume that 𝑧 𝑜 = 𝐻 𝜕 𝑓 9(:;\] : ) à 𝑧 𝑜 − 𝑛 = 𝑓 79:Z 𝑧 𝑜 for any 𝑛 § Putting this into the different equation 𝑧 𝑜 = 𝑐 & ' 𝑦 𝑜 + 𝑐 - ' 𝑓 79: ' 𝑦 𝑜 + 𝑐 . ' 𝑓 79.: ' 𝑦 𝑜 + … + 𝑐 2 ' 𝑓 79:2 ' 𝑦 𝑜 +𝑏 - ' 𝑓 79: ' 𝑧 𝑜 + 𝑏 . ' 𝑓 79.: ' 𝑧 𝑜 + ⋯ + 𝑏 R ' 𝑓 79:R ' 𝑧 𝑜 𝑧 𝑜 = 𝑐 & + 𝑐 - ' 𝑓 79: + 𝑐 . ' 𝑓 79.: + … + 𝑐 2 ' 𝑓 79:2 1 + 𝑏 - ' 𝑓 79: + 𝑏 . ' 𝑓 79.: + … + 𝑏 R ' 𝑓 79:R 𝑦(𝑜) 𝐼(𝜕) : frequency response 𝐼(𝜕) = 𝑐 & + 𝑐 - ' 𝑓 79: + 𝑐 . ' 𝑓 79.: + … + 𝑐 2 ' 𝑓 79:2 𝐻 𝜕 = 𝐼(𝜕) : amplitude response 1 + 𝑏 - ' 𝑓 79: + 𝑏 . ' 𝑓 79.: + … + 𝑏 R ' 𝑓 79:R 𝜄 𝜕 = ∠𝐼(𝜕) : phase response 25

Z-Transform § 𝑎 -transform – Define z to be a variable in complex plane: we call it z -plane – When z = e j ω (on unit circle), the frequency response is a particular case of the following form 𝐵(𝑨) = 𝑐 & + 𝑐 - ' 𝑨 7- + 𝑐 . ' 𝑨 7. + … + 𝑐 2 ' 𝑨 72 𝐼 𝑨 = 𝐶(𝑨) 1 + 𝑏 - ' 𝑨 7- + 𝑏 . ' 𝑨 7. + … + 𝑏 R ' 𝑨 7R – We call this 𝑨 -transform or the transfer function of the filter – z -1 corresponds to one sample delay: delay operator or delay element – Filters are often expressed as 𝑨 -transform: polynomial of 𝑨 7- 26

Practical Filters § One-pole one-zero filters – Leaky integrator 1 z − 1 H ( z ) = b 0 + b – Moving average a 0 + a 1 z − 1 – DC-removal filters – Bass / treble shelving filter § Biquad filters 1 z − 1 + b 2 z − 2 – Reson filter H ( z ) = b 0 + b – Band-pass / notch filters a 0 + a 1 z − 1 + a 2 z − 2 – Equalizer: a set of biquad filters § Any high-order filter can be factored into a combination of one-pole one- zero filters or bi-quad filters! 27