CTP431- Music and Audio Computing, Fall 2017 Introduction Graduate - PowerPoint PPT Presentation

CTP431- Music and Audio Computing, Fall 2017 Introduction Graduate School of Culture Technology, KAIST Juhan Nam 1

Who We Are § Instructor – Juhan Nam ( 남주한 ) – Assistant Professor in GSCT, KAIST – Music and Audio Computing Lab: http://mac.kaist.ac.kr § TAs: – Jongpil Lee ( 이종필 ), Ph.D. Student in GSCT, KAIST – Wonil Kim ( 김원일 ), M.S. Student in GSCT, KAIST 2

What Is This Course About? § Introduction to Music Technology Music Technology? 3

Music Technology § A whole set of different technologies that have changed the way that people are engaged in music as composer, performer and listener Music Composition Education Notation Distribution Entertainment Musical Instruments Production Music Music Performance Listening 4

Piano § Invented by Bartolomeo Cristofori (1720s) Cristofori’s FortePiano (1722) The Piano Action 5

Piano Steinway Model D 6

Piano § Characteristics – Rich harmonics – Sustained tone – Polyphonic and wide register Harmonics of Piano § Influence on music – Composers in the Romantic era: Chopin, Schumann, Liszt – Musical expressions – Tonal harmony: vertical relation of notes 7

Sound Recording § Phonautograph - Leon Scott (1857) – The first invention of sound recording but recording only – Recent research on image to sound restoration: http://firstsounds.org/ http://www.newyorker.com/tech/elements/new-sounds-old-voices http://edcarter.net/home/phonautogram/

Sound Recording § Phonograph: Thomas Edison (1877) Edison cylinder Grooves in Edison cylinder

Sound Recording § Gramophone: Emile Berliner (1887)

Effect of the Early Music Records § Globalization – Worldwide distribution of music – Music becomes industry Caruso <Vesti La Giubba> 1930s superstar, 왕수복 “The first million-seller record”

Effect of the Early Music Records § Musical Form – Changed by the limited duration (3-4 minute long) – “AABA” form (or 32-bar form) – Modern popular music • Jazz standards Souce: http://www.musicarrangerspage.com/251/what-is-a-bridge-interlude/ 12

Musical Tone § A pitched tone has a periodic waveform 0.4 0.2 amplitude 0 − 0.2 − 0.4 50 52 54 56 58 60 time − milliseconds

Fourier Series § Any periodic signal can be represented as a sum of harmonically related sine waves Joseph Fourier

Additive Synthesis § Synthesize sounds by adding multiple sine oscillators – Also called Fourier synthesis OSC Amp (Env) OSC Amp (Env) + . . . . . . OSC Amp (Env) 15

Telharmonium § Additive synthesizer using electro- magnetic “tone wheels” (Cahill, 1897) § Transmitted through telephone lines – Subscription only – The business failed Tone wheel

Telharmonium https://www.youtube.com/watch?v=TunkjJvbrHs

Evolved into Hammond Organ 18

Subtractive Synthesis § Synthesize sounds by sharping with filters 0.4 0.2 amplitude Amp Oscillators Filter 0 − 0.2 − 0.4 50 52 54 56 58 60 time − milliseconds 20 20 20 10 10 10 0 0 0 Magnitude (dB) Magnitude (dB) Magnitude (dB) − 10 − 10 − 10 − 20 − 20 − 20 − 30 − 30 − 30 − 40 − 40 − 40 − 50 − 50 − 50 − 60 − 60 − 60 0 0.5 1 1.5 2 2.5 5 10 15 20 5 10 15 20 Frequency (kHz) Frequency (kHz) 4 x 10 Frequency (kHz) Oscillator Filter Filtered Sound 19

MiniMoog (1970) 20

Moog Synthesizers 21

https://www.youtube.com/watch?v=usl_TvIFtG0

“Switched-On-Bach” by Wendy Carlos (1968) 23

Magnetic Tape Recording § High Fidelity – Nearly flat over audible frequency range § Malleable – Record audio can be edited § Multi-track recording – Record and playback simultaneously – Layer by layer recording

Music Concrete § Composition by tape editing – Cut – Splice – Reverse – Speed up/down https://www.youtube.com/watch?v=c4ea0sBrw6M

Pierre Schaeffer -- Études de bruits (1948)

Sample-based Synthesis Mellotron (1963)

28

Les Paul’s Innovation § Overdubbing – Ensemble Effects § Delay effects – Tape delay or phasing effects Les Paul

Digital Audio … 1 0 0 1 1 0 1 … … 0 1 0 0 1 1 0 …

Sample-based Sound Synthesis Samples + Digital Signal Processing … 1 0 0 1 1 0 1 … … 0 1 0 0 1 1 0 … Music Score Sound

Digital Audio Effects Digital Signal Processing … 1 0 0 1 1 0 1 … … 0 1 0 0 1 1 0 … Sound Transformed Sound

MIDI and Sequencer § Save and play musical performance Step sequencer MIDI sequencer (Drum Machine) 33

Digital Audio Workstation 34

Launch Pad 35

DJ Machines https://www.youtube.com/watch?v=44F0d2CbjM0

Max Mathews § Father of Computer Music § Developed the first programming language to synthesize sound, called Music-N (1957) § Invented Radio-baton, a baton-type controller for computer orchestra http://opinionator.blogs.nytimes.com/2011/06/08/the-first-computer-musician/?_r=0

Music-N (Max Mathews)

"Daisy Bell”, programmed by John Kelly and Carol Lockbaum and the accompaniment was programmed by Max Mathews.

Audio Programming Language MAX / MSP / Jitter SuperCollider

Web Audio Web Audio (HTML5 Standard) http://www.google.com/doodles/robert-moogs-78th-birthday 43

Web Audio https://tonejs.github.io

Course Goals § Understanding theoretical backgrounds in music technology today – Basic acoustics, digital audio – Sound synthesis, digital audio effects – Musical control and representation (e.g. MIDI) – Algorithmic composition – Music analysis and information retrieval § Hand-on experience with sound examples and code – Programming: web audio API (HTML/CSS/Javascript) – Generate, modifying and controlling sounds – Visualization and graphics – Interaction 45

What is Web Audio API? § High-level JavaScript API for processing and synthesizing audio in web applications – Contain a number of sound synthesis, processing, and analysis nodes § HTML5 standards supported in Chrome, Firefox and other web browsers – Replacing “Flash” for multimedia capability VS 46 Flash: ActionScript 1985 1990 1995 2000 2005 2010 2015

Why Web Audio API? § Based on existing language (i.e. JavaScript) § Easy to integrate with multimedia components – MIDI (e.g. webMIDI) – Graphics (e.g. webGL) – Input sensor: camera, microphone, keyboard, mouse § Free and no installation § Platform-independent (but browser-dependent) 47

Why Web Audio API? Stackoverflow developer survey results (2016) Most popular languages used on GitHub (2016) https://octoverse.github.com/ https://insights.stackoverflow.com/survey/2016#technology 48

Course Information § Course webpage – http://mac.kaist.ac.kr/~juhan/ctp431/ – Basic course info, schedule and resources § KLMS – Announcement – Q&A: discussion board – Homework submission – Grading 49

Lecture Format § Monday – Theory part – Read the slides before the class § Wednesday – Practice part – Web audio programming – Bring your laptop 50

Grading § Attendance: 10% – Attendance, participation in discussion, and so on § Assignments: 40% – Javascript programming using web audio § Midterm: 20% – Paper exam focusing on theories § Final Project: 30% – Proposal / Presentation / Submission (by sharing on the web using Github) 51

Pre-requisites § Basic literacy – Programming language: variable, control, loop, function, class – Signal processing: meaning of x, y, t and f, Fourier transform (hopefully…) – Music: basic music theory § HTML/CSS/Javascript: desired but not required 52