Sound Synthesis Graduate School of Culture Technology (GSCT) Juhan - PowerPoint PPT Presentation

CTP 431 Music and Audio Computing Sound Synthesis Graduate School of Culture Technology (GSCT) Juhan Nam 1

Outlines § Brief history of sound synthesis § Additive Synthesis § Subtractive Synthesis – Analog synthesizers § Nonlinear Synthesis – Ring modulation / Frequency modulation – Wave-shaping § Sample-based Synthesis 2

Brief History § Telharmonium (Cahill, 1897) – Room-size additive synthesizer using electro-magnetic “tone wheels” – Transmitted through telephone lines (subscription only!) – Sound like organ: evolved into Hammond B3 organ (drawbars) – https://www.youtube.com/watch?v=PPlbXl81Rs0 § Theremin (Léon Theremin, 1928) – Two metal antennas recognize the relative position of hands by detecting the change of electro-magnetic fields – Each of them controls amplitude and pitch of a tone – https://www.youtube.com/watch?v=w5qf9O6c20o – https://www.youtube.com/watch?v=pSzTPGlNa5U 3

Brief History § Music Concrete (Pierre Schaeffer, 1948) – Creating sounds by splicing the pieces of tapes where sounds are recorded: sampling-based synthesis – Related to musical composition – https://www.youtube.com/watch?v=c4ea0sBrw6M – Mellotron (1963): https://www.youtube.com/watch?v=HdkixaxjZCM § RCA Synthesizer: Mark II (1957) – First programmable synthesizer – Room-size and off-line processing as a synthesizer and a sequencer – https://www.youtube.com/watch?v=rgN_VzEIZ1I § Moog Synthesizers (Moog, 1964) – Mini-moog (1971): the first popular synthesizer – “Switched-On-Bach” by Wendy Carlos – https://www.youtube.com/watch?v=usl_TvIFtG0 4

Brief History § Yamaha DX7 (1983) – FM synthesis, the first commercially successful synthesizer – Electronic piano sounds in 80’s pop music § Fairlight CMI (1979) – The first sampling-based digital synthesizer – https://www.youtube.com/watch?v=iOlPCpSmhRM § Kurzweil K250 (1983) – The first synthesizer that faithfully reproduced an acoustic grand piano § Yamaha VL-1 (1994) – The first commercially available physical-modeling synthesizer 5

Sound Synthesis Techniques § Categories – Additive synthesis – Subtractive synthesis – Non-linear: modulation / wave-shaping – Sample-based synthesis – Physical modeling Memory Programmability Reproducibility of Interpretability Computa=on (Storage) (by # of parameters) natural sounds of parameters power Addi=ve ** ***** **** **** **** Subtrac=ve * *** ** *** ** Non-linear * *** ** ** ** Sample-base ***** * ***** N/A * ~ *** Physical model *** ** **** ***** *** ~ ***** 6

Additive Synthesis § Synthesize sounds by adding multiple sine oscillators – Also called Fourier synthesis OSC Amp (Env) OSC Amp (Env) + . . . . . . OSC Amp (Env) 7

Sound Examples § Web Audio Demo – http://femurdesign.com/theremin/ – http://www.venlabsla.com/x/additive/additive.html § Examples (instruments) – Kurzweil K150 • https://soundcloud.com/rosst/sets/kurzweil-k150-fs-additive – Kawai K5, K5000 – Hammond Organ 8

Subtractive Synthesis § Synthesize sounds by filtering wide-band oscillators – Source-Filter model – Examples • Analog Synthesizers: oscillators + resonant lowpass filters • Voice Synthesizers: glottal pulse train + formant filters 20 20 20 10 10 10 0 0 0 Magnitude (dB) Magnitude (dB) Magnitude (dB) − 10 − 10 − 10 − 20 − 20 − 20 − 30 − 30 − 30 − 40 − 40 − 40 − 50 − 50 − 50 − 60 − 60 − 60 5 10 15 20 5 10 15 20 0 0.5 1 1.5 2 2.5 Frequency (kHz) Frequency (kHz) Frequency (kHz) 4 x 10 Filtered Source Source Filter 9


 Subtractive Synthesis § Moog Synthesizer SoN Envelope LFO Control Keyboard Physical Envelope Control Wheels Slides Pedal Parameter Parameter Parameter Audio Path Amp Oscillators Filter (e.g. filter Parameter = offset + depth*control cut-off frequency) (sta=c value) (dynamic value) 10


 Oscillators § Classic waveforms 2 2 1 0 0 0 − 2 − 1 − 2 0 50 100 150 200 0 50 100 150 200 0 50 100 150 200 20 20 20 Magnitude (dB) Magnitude (dB) Magnitude (dB) − 12dB/oct 0 − 6dB/oct − 6dB/oct 0 0 − 20 − 20 − 20 − 40 − 40 − 40 − 60 − 60 − 60 5 10 15 20 5 10 15 20 5 10 15 20 Frequency (kHz) Frequency (kHz) Frequency (kHz) Triangular Sawtooth Square § Modulation – Pulse width modulation – Hard-sync – More rich harmonics 11

Amp Envelop Generator § Amplitude envelope generation – ADSR curve: attack, decay, sustain and release – Each state has a pair of time and target level Amplitude AUack Decay Sustain (dB) Release Note On Note Off 12

Examples § Web Audio Demos – http://www.google.com/doodles/robert-moogs-78th-birthday – http://webaudiodemos.appspot.com/midi-synth/index.html – http://aikelab.net/websynth/ – http://nicroto.github.io/viktor/ § Example Sounds – SuperSaw – Leads – Pad – MoogBass – 8-Bit sounds: https://www.youtube.com/watch?v=tf0-Rrm9dI0 – TR-808: https://www.youtube.com/watch?v=YeZZk2czG1c 13

Modulation Synthesis § Modulation is originally from communication theory – Carrier: channel signal, e.g., radio or TV channel – Modulator: information signal, e.g., voice, video § Decreasing the frequency of carrier to hearing range can be used to synthesize sound § Types of modulation synthesis – Amplitude modulation (or ring modulation) – Frequency modulation § Modulation is non-linear processing – Generate new sinusoidal components 14

Ring Modulation / Amplitude Modulation § Change the amplitude of one source with another source – Slow change: tremolo – Fast change: generate a new tone OSC OSC Modulator Modulator OSC OSC x + x Carrier Carrier (1 + a m ( t )) A c cos(2 π f c t ) a m ( t ) A c cos(2 π f c t ) Ring Modula=on Amplitude Modula=on 15

Ring Modulation / Amplitude Modulation § Frequency domain – Expressed in terms of its sideband frequencies – The sum and difference of the two frequencies are obtained according to trigonometric identity – If the modulator is a non-sinusoidal tone, a mirrored-spectrum with regard to the carrier frequency is obtained carrier sideband sideband a m ( t ) = A m sin(2 π f m t )) f c -f m f c f c +f m 16

Examples § Tone generation – SawtoothOsc x SineOsc – https://www.youtube.com/watch?v=yw7_WQmrzuk § Ring modulation is often used as an audio effect – http://webaudio.prototyping.bbc.co.uk/ring-modulator/ 17

Frequency Modulation § Change the frequency of one source with another source – Slow change: vibrato – Fast change: generate a new (and rich) tone – Invented by John Chowning in 1973 à Yamaha DX7 OSC Modulator A c cos(2 π f c t + β sin(2 π f m t )) frequency OSC β = A m Carrier Index of modula=on f m 18

Frequency Modulation § Frequency Domain – Expressed in terms of its sideband frequencies – Their amplitudes are determined by the Bessel function – The sidebands below 0 Hz or above the Nyquist frequency are folded k = −∞ ∑ y ( t ) = A c J k ( β )cos(2 π ( f c + kf m ) t ) k = −∞ carrier sideband1 sideband1 sideband2 sideband2 sideband3 sideband3 f c -3f m f c -f m f c f c +f m f c -2f m f c +2f m f c +3f m 19

Bessel Function ( − 1) n ( β 2 ) k + 2 n ∞ ∑ J k ( β ) = n !( n + k )! n = 0 1 Carrier Sideband 1 Sideband 2 Sideband 3 Sideband 4 0.5 J_(k) 0 − 0.5 0 50 100 150 200 250 300 350 beta 20

Bessel Function 21

The Effect of Modulation Index 1 1 Amplitude Amplitude 0 0 − 1 − 1 0 500 1000 1500 2000 0 500 1000 1500 2000 Time (Sample) Time (Sample) 20 Magnitude (dB) 20 Magnitude (dB) Beta = 0 Beta = 1 0 0 − 20 − 20 − 40 − 40 − 60 − 60 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Frequency (kHz) Frequency (kHz) 1 1 Amplitude Amplitude 0 0 − 1 − 1 0 500 1000 1500 2000 0 500 1000 1500 2000 Time (Sample) Time (Sample) 20 20 Magnitude (dB) Magnitude (dB) Beta = 10 Beta = 20 0 0 − 20 − 20 − 40 − 40 − 60 − 60 0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1 Frequency (kHz) Frequency (kHz) f c = 500, f m = 50 22

“Algorithms” in DX7 hUp://www.audiocentralmagazine.com/yamaha-dx-7-riparliamo-di-fm-e-non-solo-seconda-parte/yamaha-dx7-algorithms/ 23

Examples § Web Audio Demo – http://www.taktech.org/takm/WebFMSynth/ § Sound Examples – Bell – Wood – Brass – Electric Piano – Vibraphone 24

Non-linear Synthesis (wave-shaping) § Generate a rich sound spectrum from a sinusoid using non- linear transfer functions (also called “distortion synthesis”) § Examples of transfer function: y = f(x) – y = 1.5x’ – 0.5x’ 3 x’=gx: g correspond to the “gain knob” of the distor=on – y = x’/(1+|x’|) – y = sin(x’) T 0 (x)=1, T 1 (x)=x, – Chebyshev polynomial: T k+1 (x) = 2xT k (x)-T k-1 (x) T 2 (x)=2x 2 -1, T 2 (x)=4x 3 -3x 1 1 Amplitude Amplitude 1 0 0 0.5 − 1 0 50 100 150 200 − 1 Amplitude 0 50 100 150 200 Time (Sample) Time (Sample) 0 20 Magnitude (dB) 20 Magnitude (dB) 0 0 − 0.5 − 20 − 20 − 40 − 1 − 40 − 60 − 1 − 0.5 0 0.5 1 5 10 15 20 Time (Sample) − 60 Frequency (kHz) 5 10 15 20 Frequency (kHz) 25