Music, Language and Computation Aline Honingh Guest lecture in - PowerPoint PPT Presentation

Music, Language and Computation Aline Honingh Guest lecture in Logic, Language and Computation

Music at the ILLC l Henkjan Honing – music cognition l Fleur Bouwer l Gabor Haden l Rens Bod – computational musicology l Aline Honingh l Monthly seminar/discussion group on music cognition and computation

Outline l Music and Language l Music and Computation l Research example: automatic classification of music

Music and Language

Commonalities between different forms of cognition l Language: “ List the sales of product in 2003 ” l Music: ( Mozart symph. 40) What do they have in common?

How do we perceive language and music? l Inherent to all forms of perception: a structuring process in groups, subgroup, sub-subgroups etc. l Groups in language form a tree-structure (Wundt 1880)

Grouping structure in music l Grouping structure represents how parts combine compositionally and recursively into a whole (Lerdahl and Jackendoff 1983)

Perceptual structure = perceived structure l Very controversial claim There exists one model that predicts the perceived structure in language, music , vision and other modalities … (cf. Newell 1999)

More commonalities between language and music processing l Perceived incremental l Alphabet l A to Z l A to G, #, b l Syntax l Language: strong relation between syntax and meaning l Music: three layers l Scale degrees l Chord structure l Key structure l Evolution l Music as language l Language as music (Mithen, 2005)

l Recursion l Language l [he dreams] l [he dreams that [he dreams]] l [he dreams that [he dreams that [he dreams]]] l Etc. l Music l Bach ’ s Canon per Tonos, a.k.a endless rising canon (see ``Godel, Escher, Bach ’’ by D. Hofstadter) l Brain l Commonalities between language and music are also found in neuroscience (`Music, Language and the Brain’, Patel 2008)

Music and Computation l Computational applications l Key finding l Pitch spelling l Segmentation l Score following l Automatic analysis l Classification (on basis of genre/composer/..) l …

Key finding l Automatic search for key G-minor

Pitch spelling Piano keys note names 1 3 6 8 10 0 2 4 5 7 9 11 9 F 6 Db A 1 G ? 5 B Gbb 11 4 0 C# E 7

Segmentation l Automatically segmenting music in phrases

Score following Process of automated listening to music and determining position in score

Automatic analysis l Automatic (harmonic) analysis

Genre classification Rock?? Jazz??

Computational musicology Useful for: l Musicology l Automatic analysis

Computational musicology Useful for: l Musicology l Automatic analysis l Cognitive science l Understanding of cognitive processes through modelling

Computational musicology Useful for: l Musicology l Automatic analysis l Cognitive science l Understanding of cognitive processes through modelling l Commercial application l Search machines for music l Music recommendations l Music notation software (Finale etc.) l Automatic accompaniment

Computational musicology Useful for: l Musicology l Automatic analysis l Cognitive science l Understanding of cognitive processes through modelling l Commercial application l Search machines for music l Music recommendations l Music notation software (Finale etc.) l Automatic accompaniment Not yet realized

Research example

Classification of music l Two music excerpts: l A: - B: l Would you classify C as belonging to A or to B? l C:

Classification of music l Two music excerpts: l A: - B: l Would you classify C as belonging to A or to B? l C: l A: Rock, B, C: Jazz

How do we model this? l We simplify the music with a model: a new representation l We compare representations

The representation 1 3 6 8 10 0 2 4 5 7 9 11 l Notes represented by numbers l Intervals have been shown to be more important than notes l Melody: transposed melody:

0 1 11 2 10 The representation 9 3 6 Interval Categories (ICs) 8 4 5 7 6 l IC n : all pitch class sets that are dominated by the interval n 1 1 1 2 2 3

0 1 11 2 10 The representation 9 3 6 Interval Categories (ICs) 8 4 5 7 6 l IC n : all pitch class sets that are dominated by the interval n 1 1 1 2 2 3 Belongs to IC1

The representation l Fine-grained level: sequence of categories IC5 IC3 IC3 IC3 l Broad level: statistics of categories Category Percentage of occurrence IC1 6.76 % IC2 5.63 % IC3 25.20 % IC4 22.94 % IC5 36.45 % IC6 3.03 % Distribution in Debussy’s Golliwogg’s cakewalk

Visual representation of IC distributions l Of the 6 categories, we choose 3 as dimensions l Each IC distribution can be represented as a point in a 3D space Category Percentage of occurrence 1 6.76 % 2 5.63 % 3 25.20 % 4 22.94 % 5 36.45 % 6 3.03 %

Categorisation of Art ( ‘ classical ’ ) music Musical period Period composers Middelages 500-1400 various Renaissance 1400-1600 Palestrina Baroque 1600-1750 Bach, Händel, Vivaldi Classical 1750-1830 Haydn, Mozart, Beethoven, Schubert Romantic 1830-1900 Brahms, Mahler, Tchaikovsky, Debussy, Mendelssohn, Sibelius Modern 1900-... Ravel, Stravinsky, Schoenberg, Webern

Genre classification l Rock versus Jazz

Concluding remarks and future research l Classification of music is possible on the basis of a very simply principle: intervals l What is it in the music that determines it genre? l Does interval information overrule temporal information? Or do they go hand in hand?

Thank you

Tonal-atonal classification experiment Number of correct Number of correct Total number of classified pieces of classified pieces of correct classified atonal music tonal music pieces of music IC method 19 53 72 (94.7 %) Alternative method 14 53 67 (88.2 %) Total number of 20 56 76 pieces IC algorithm : category 5 / categorie 1 > α : tonal otherwise: atonal Alternative method : for each bar: find key count notes not in this key total number of these notes > β : atonal otherwise: tonal