CTP431- Music and Audio Computing Audio Signal Processing (Part #2) Graduate School of Culture Technology KAIST Juhan Nam 1
Types of Audio Signal Processing § Filter/EQ § Compressor § Delay-based Effects – Delay, reverberation § Spatial Effect – HRTF § Playback Rate Conversion – Resampling 2
Filters § Adjust the level of a certain frequency band – Lowpass – Highpass – Bandpass – Notch – Resonant Filter – Equalizer § Parameters – Cut-off/Center Frequency – Q: sharpness/resonance 3
Low-pass Filter § Transfer Function 1 + 2 z − 1 + 1 z − 2 H ( z ) = (1 − cos Θ α = sin Θ Θ = 2 π f c / f s ) (1 + α ) − 2cos Θ z − 1 + (1 − α ) z − 2 2 2 Q – fc : cut-off frequency, Q: resonance Lowpass Filters Lowpass Filters 30 30 20 20 Q =4 10 10 Q =2 Gain(dB) Gain(dB) f=400 f=1000 f=3000 f=8000 Q =1 0 0 Q =0.5 − 10 − 10 − 20 − 20 − 30 − 30 2 3 4 2 3 4 10 10 10 10 10 10 freqeuncy(log10) freqeuncy(log10) 4
High-pass Filter § Transfer Function 1 − 2 z − 1 + 1 z − 2 H ( z ) = (1 + cos Θ α = sin Θ ) Θ = 2 π f c / f s (1 + α ) − 2cos Θ z − 1 + (1 − α ) z − 2 2 2 Q Highpass Filters Highpass Filters 30 30 20 20 f=400 f=1000 f=3000 f=8000 Q =4 10 10 Q =2 Gain(dB) Gain(dB) Q =1 0 0 Q =0.5 − 10 − 10 − 20 − 20 − 30 − 30 2 3 4 2 3 4 10 10 10 10 10 10 freqeuncy(log10) freqeuncy(log10) 5
Band-pass filter § Transfer Function 1 − z − 2 H ( z ) = (sin Θ α = sin Θ Θ = 2 π f c / f s ) (1 + α ) − 2cos Θ z − 1 + (1 − α ) z − 2 2 2 Q Bandpass Filters Bandpass Filters 30 30 20 20 Q =4 10 10 Q =2 Gain(dB) Gain(dB) Q =1 0 f=400 f=1000 f=3000 f=8000 0 Q =0.5 − 10 − 10 − 20 − 20 − 30 − 30 2 3 4 2 3 4 10 10 10 10 10 10 freqeuncy(log10) freqeuncy(log10) 6
Notch filter § Transfer Function 1 − 2cos Θ z − 1 + z − 2 α = sin Θ H ( z ) = Θ = 2 π f c / f s (1 + α ) − 2cos Θ z − 1 + (1 − α ) z − 2 2 Q Notch Filters Notch Filters 30 30 20 20 10 10 f=400 f=1000 f=3000 f=8000 Gain(dB) Gain(dB) 0 0 Q =4 − 10 − 10 Q =2 Q =1 − 20 − 20 Q =0.5 − 30 − 30 2 3 4 2 3 4 10 10 10 10 10 10 freqeuncy(log10) freqeuncy(log10) 7
Equalizer § Transfer Function H ( z ) = (1 + α ⋅ A ) − 2cos Θ z − 1 + (1 + α ⋅ A ) z − 2 α = sin Θ Θ = 2 π f c / f s (1 + α / A ) − 2cos Θ z − 1 + (1 − α / A ) z − 2 2 Q EQ EQ 30 30 Q=1 Q=4 20 20 AdB=12 AdB=12 10 10 AdB=6 AdB=6 Gain(dB) Gain(dB) AdB=0 AdB=0 0 0 AdB= − 6 AdB= − 6 AdB= − 12 − 10 AdB= − 12 − 10 − 20 − 20 − 30 − 30 2 3 4 2 3 4 10 10 10 10 10 10 freqeuncy(log10) freqeuncy(log10) 8
References § Cookbook formulae for audio EQs based on biquad filter (R. Bristow- Johnson) – http://www.musicdsp.org/files/Audio-EQ-Cookbook.txt 9
Compressor § Audio effect unit for automatic gain control – Boost the level for soft signals and suppress it for loud signals – Typically used as a front-end processor in sound recording § Signal Processing Pipeline Envelop Gain Input Detector Curve X Output 10
Envelope Detector § Detecting the level of signal Full-wave Leaky Input envelope rectification Integrator § Different sensitivity for increasing (attack) and decreasing (release) levels – During attack: − 1( attack _ time * fs ) )( x ( n ) − y ( n − 1)) y ( n ) = y ( n − 1) + (1 − e – During release: − 1( release _ time * fs ) )( x ( n ) − y ( n − 1)) y ( n ) = y ( n − 1) + (1 − e 11
Gain Curve § Parameters – Threshold: level 1 – Attack/Release: sensitivity 0.5 – Ratio: amount of compression Before 0 compression – Knee: smoothing − 0.5 − 1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 time, samples 4 x 10 No compression 1 Output (dB) 1:2 Ratio 0.5 After 1:4 compression 0 1:10 Hard Knee − 0.5 − 1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 time, samples 4 x 10 Soft Knee Threshold Input (dB) Gain Curve 12
Delay-based Audio Effects § Types of delay-based audio effect – Delay – Chorus – Flanger – Reverberation 13
Delay + x(n) Delay Line feedback Wet Dry y(n) + § Delay effect – Generate repetitive loop delay – Feedback coefficient controls the amount of delayed input – Can be extended to stereo signals such that the delay output is “ping-ponged” between the left and right channels – The delay length is often synchronized with music tempo – The delayline is implemented as a “circular buffer” 14
Chorus LFOs Delay Line x(n) + Wet Dry y(n) + § Chorus effect – Gives the illusion of multiple voices playing in unison – By summing detuned copies of the input – Low frequency oscillators are used to modulate the position of output tops à This causes the pitch of the input (resampling!) 15
Flanger LFOs Delay Line x(n) Static tap Variable tap + Wet Dry y(n) + § Flanger effect – Originally generated by summing the output of two un-locked tape machines while varying their sync (used to be called “reel-flanging”) – Emulated by summing one static tap and variable tap in the delay line • Feed-forward combine filter where harmonic notches vary over frequency. – LFO is often synchronized with music tempo 16
Reverberation Listener Direct sound Sound Source Reflected sound § Natural acoustic phenomenon that occurs when sound sources are played in a room – Thousands of echoes are generated as sound sources are reflected against wall, ceiling and floors – Reflected sounds are delayed, attenuated and low-pass filtered: high-frequency component decay faster – The patterns of myriads of echoes are determined by the volume and geometry of room and materials on the surfaces 17
Reverberation § Room reverberation is characterized by its impulse response (IR) CCRMA Lobby Impulse Response – E.g. when a balloon pop is used as a sound source 1 0.8 0.6 § The room IR is composed of three parts direct path response amplitude 0.4 – Direct path early reflections – Early reflections 0.2 – Late-field reverberation: high echo density 0 -0.2 § RT60 late-field reverberation – The time that it takes the reverberation to decay -0.4 0 10 20 30 40 50 60 70 80 90 100 time - milliseconds by 60 dB from its peak amplitude 18
Artificial Reverberation § Mechanical reverb – Use metal plate and spring – Plate reverb: https://www.youtube.com/watch?v=XJ5OFpvX5Vs § Delayline-based reverb – Early reflections: feed-forward delayline – Late-field reverb: allpass/comb filter, feedback delay networks (FDN) – “Programmable” reverberation § Convolution reverb – Measure the impulse response of a room – Do convolution input with the measured IR 19
Delay-based Reverb _ x(n) + y(n) Z -M + - A reverb is constructed by cascading multiple AP or FFCF units AllPass filter / Comb filter (when one tap is absent) Z -M1 x(n) + y(n) + Z -M2 Z -M3 - The lengths of delaylines are chosen such that their greatest common factors a 11 a 12 a 13 is small (e.g. prime numbers) a 11 a 12 a 13 a 11 a 12 a 13 - The mixing matrix is chosen to be unitary (orthonormal) Feedback Delay Networks 20
Convolution Reverb § Measuring impulse responses measurement n (t) noise r (t) s (t) r ( t ) = s ( t ) ∗ h ( t ) + n ( t ) , h (t) test LTI measured sequence system response – If the input is a unit impulse, SNR is low – Instead, we use specially designed input signals • Golay code, allpass chirp or sine sweep: their magnitude responses are all flat but the signals are spread over time – The impulse response is obtained using its inverse signal or inverse discrete Fourier transform 21
Convolution Reverb sine sweep, s(t) sine sweep spectrogram 0.5 10 frequency - kHz amplitude s ( t ) 0 5 -0.5 0 0 500 1000 1500 0 200 400 600 800 1000 sine sweep response, r(t) sine sweep response spectrogram 1 10 frequency - kHz 0.5 amplitude r ( t ) 0 5 -0.5 -1 0 0 500 1000 1500 2000 0 500 1000 1500 2000 time - milliseconds time - milliseconds measured impulse response 0.08 0.06 0.04 ˆ amplitude h ( t ) 0.02 0 -0.02 ( J. Abel ) -0.04 0 100 200 300 400 500 600 700 800 900 1000 time - milliseconds 22
Spatial Hearing § A sound source arrives in the ears of a listener with differences in time and level – The differences are the main cues to identify where the source is. – We call them ITD (Inter-aural Time Difference) and IID (Inter-aural Intensity Difference) – ITD and IID are a function of the arrival angle. L IID R ITD 23
Head-Related Transfer Function (HRTF) § A filter measured as the frequency response that characterizes how a sound source arrives in the outer end of ear canal – Determined by the refection on head, pinnae or other body parts – Function of azimuth (horizontal angle) and elevation (vertical angle) 𝐼 " (𝜕, ∅, 𝜄) L 𝐼 ) (𝜕, ∅, 𝜄) R 24
Measured Head-Related Impulse Responses 25
Magnitude response of the HRIRs 26
Binaural Synthesis Left output ℎ " (𝑢, ∅, 𝜄) Input ℎ ) (𝑢, ∅, 𝜄) Right output § Rendering the spatial effect using the measured HRIRs as FIR filters – HRIRs are typically several hundreds sample long – Convolution or modeling by IIR filters § Individualization of HRTF is a issue 27
Recommend
More recommend
