Rhythm Tree Rewriting Jean Bresson (Ircam) Pierre Donat-Bouillud - PowerPoint PPT Presentation

Rhythm Tree Rewriting Jean Bresson (Ircam) Pierre Donat-Bouillud (ENS Cachan/Rennes) Florent Jacquemard (INRIA, Ircam) IFIP WG1.6 July 13, 2014

general perspective music symbolic representations in particular traditional western music notation processing of tree structured representations � general long term goal: symbolic MIR problems 
 http://music-ir.org/mirex • querying bases of music scores • transformations, • melodic similarity detection, • genre classification, recommendation, • detection of repetitions, automatic segmentation, 
 musicological analysis � one particular target application: rhythm transcription

target application rhythm transcription: automatic generation of notation plan of the talk • context: computer assisted (Music) composition • problem: rhythm transcription • data structure: rhythm trees • term rewriting approach • weighted tree automata

1 Assisted Composition hardware and software tools for authoring music, production of music scores

music notation editors score printing import/export MIDI and MusicXML • Finale (MakeMusic) • Sibelius (Avid)

algorithmic composition environments solving musical problems visual programming languages based on Lisp • PWGL (Sibelius Academy) • OpenMusic (Ircam)

2 Automatic Music Transcription

automatic music transcription: goal acoustical recording (audio file) symbolic timed trace (71,0.0)(74,1.18)(46,1.73)(52,2.09)(5 (MIDI file, piano roll) notation (MusicXML…)

automatic music transcription: tasks acoustical recording audio pitch tracking (audio file) MIR onset detection symbolic symbolic timed trace beat tracking MIR (MIDI file, piano roll) tempo/metric extraction notation (MusicXML…) structural segmentation

automatic music transcription: applications recorded acoustical recording performance (audio file) symbolic algorithmic MIDI keyboard timed trace composition (score edition) (MIDI file) (OpenMusic…) model of performance agnostic = score + deviations notation (MusicXML…) this work, with focus on rhythm quantization

rhythm quantization (symbolic) 1. segmentation of the input timed trace 
 each segment with constant tempo 
 or known acceleration 2. identification of tempo / beat positions 3. local quantization on each segment 
 ( on-the-beat quantization ) • input: one segment = timed trace: 
 sequence s of pairs (onset,duration) in 𝐒 2 (ms) • output: sequence t of pairs (onset, kind), with onsets in 𝐄 , small discrete set 
 (admissible subdivisions of beat) kind = ‘note’ or ‘silence’ number of ‘notes’ = | s |

rhythm quantization : measures of quality timed trace transcription 1: high precision, high complexity transcription 2: lower precision, low complexity

alignment to grid Desain, Honing, de Rijk Longuet-Higgins Quantization of musical time Mental Processes: Studies in Cognitive Science, 1987 Music and Connectionnism, MIT Press 1991 0 1 depth 2 3 0 1/2 1 c grid of depth 3 and subdivision schema (3, 2, 2)

choice of the grid according to depth, number of divisions… with user parameters or heuristics which grid for this? G G D C D7 D7 D7 G 3 4 which grid for that?

heuristics for grid selection Pressing, Lawrence Transcribe: A comprehensive autotranscription program ICMC 1993 Agon, Assayag, Fineberg, Rueda Meudic Kant: A critique of pure quantization OMquantify ICMC 1994 PhD Ircam, 2005 • given a predefined set T of template grids 
 (user parameters) • align the input segment s to grids of T • select the best grid g ∈ T according to distance to s • return alignement of s to g 
 (converted to a score)

local quantization workflow OMquantify RT based approaches � � • generation of several grids • convert input seq. s 
 • select best grid g 
 into RT (according to 3 distances) • computations on RT(s) • align input seq. s to g � • convert the alignement 
 • return RT result as score into OpenMusic Rhythm Tree 
 return as score

3 Rhythm Trees

rhythms and durations rhythmic values (fractions of the period) = symbolic notation tempo (frequency) durations (ms)

rhythm trees (RT) in traditional western music notation: durations are defined by recursive subdivisions of units

rhythm CF grammars Lee The rhythmic interpretation of simple musical sequences Musical Structure and Cognition, 1985 � � � � � � � � � � � � 3 → ˘ “ · | < 1 + 4 1 + 4 1 · | 4 4 1 → ˇ “ | > | 8 1 + 8 1 4 4 4 4 4 1 → ˇ “( | ? | · · · 2 2 2 2 8 4 4 4 4 S → ¯ | < | 4 2 + 4 2 1 1 1 1 1 1 4 4 4 4 4 4 2 → ˘ “ | < | 4 1 + 4 1 4 1 1 1 → ˇ “ | > | 8 8 8 1 + 8 1 4 1 → ˇ “( | ? | · · · 8

OM rhythm trees (OMRT) trees are the standard data structure for OpenMusic Laurson Agon, Haddad, Assayag Patchwork: A Visual Programming Language Representation et rendu de structures rythmiques Helsinki: Sibelius Academy, 1996 JIM, 2002 4 3 1 2 1 1 2 1 1 1 1 list (4 (3 (1 (2 1)) 2 1 (1 (1 1 1)))) • labels in 𝐎 = durations • ∑ children = equal subdivisions of parent • notes = leaves • for numerical computations (not symbolic)

symbolic RT • all sibling represent equal durations • for symbolic computations (unary notation) terms over the following signature: • inner nodes: labelled with arity = prime numbers 2-13 • leaves: • n : note • r : rest • s : slur. Sum with previous leaf in dfs ordering • 1+ : add to next sibling 2 G ˇ ˇ ˇ ˇ 2 2 n n n n

ˇ — ˇ — ˇ ˇ symbolic RT (example 2) G 4 3 fl 3 fl —fl —fl fl fl 4 ˇ` ˇ ˇ ˇ . 3 3 1+ 1+ 1+ n n n 1+ n n n n n

ˇ — ˇ ˇ symbolic RT (example 3) G 4 3 fl —fl fl fl 4 ˘ . 3 1+ 1+ n n n n

advantage of RT representation over string representation • close to traditional music notation � • keep the integrity constraint 
 sum of durations = 1 � • groups of correlated events 
 reflected in the tree structure 
 = sequences of siblings 
 → preserved in local transformations

4 Local Quantization by RT Rewriting

principle 1. generate an initial tree t 0 from input s 
 with maximum precision, maximum complexity • t 0 complete • closest to s given maximum depth, signature • alignment to a complete grid � 2. simplify t 0 into t using a set of rewrite rules 
 of 2 kinds: • conservative rules (preserve durations) • simplifying rules (do not preserve durations) � 3. return score corresponding to t

ˇ ? ? > conservative rules p ( n , s , . . . , s ) → n merge-ns − → ˇ 8 ˇ p ( r , . . . , r ) → r merge-r − → p ( s , . . . , s ) → s merge-s − → r 8 ˇ r 8 ˇ 8 ˇ p ( x 1 , . . . , x m , t, 1+ , . . . , 1+ , n , y 1 , . . . , y n ) replace-s | {z } k → p ( x 1 , . . . , x m , t, n , s , . . . , s , y 1 , . . . , y n ) | {z } k t = n or t = r or t = s , t = p ( z 1 , . . . , z p )

ˇ ˇ ˇ ˇ ˇ ˇ ˇ ˇ ˇ > ˇ ˇ ˇ ˇ conservative rules (2) 3 3( n , 2( s , n ) , s ) → 2( n , n ) 3/2 − → ˇ ˇ ˇ 8 ˇ ˇ 8 5 5( n , s , 2( s , n ) , s , s ) → 2( n , n ) 5/2 − → ˇ ˇ = ˇ ˇ ˇ = ˇ 7/2, 11/2,… � � 2 2( n , 3( s , n , s )) , 2(3( s , s , n ) , s ) → 3( n , n , n ) 4/3 3 3 3 − → 8 ˇ ˇ ˇ ˇ ˇ 8 ˇ 5( n , 3( s , s , n ) , s , 3( s , n , s ) , s ) → 3( n , n , n ) 5/3 5 3 3 3 − → ˇ ; ˇ ˇ = ˇ ˇ ˇ ; ˇ …

conservative rules (application) equational theory of rhythm notation the set of conservative rules is • confluent and terminating � it can be used as a tool for • simplifying rhythm notations • identify equivalent rhythm notations

simplifying rules reduce p ( s , n , x 1 , . . . , x m ) p ( n , s , x 1 , . . . , x m ) → p ( s , r , x 1 , . . . , x m ) p ( r , r , x 1 , . . . , x m ) → inflate p ( s , . . . , s , r ) → p ( s , . . . , s ) | {z } | {z } m +1 m � � � � p ( s , . . . , s , n ) , p 0 ( s , x 1 . . . , x k ) p ( s , . . . , s ) , p 0 ( n , x 1 . . . , x k ) → p 1 p 1 | {z } | {z } m m +1 p 2 p 2 p 1 p 1 p 0 ( n , x 1 , . . . , x ` ) p 0 ( s , x 1 , . . . , x ` ) → x x p ( s , . . . , s ) p ( s , . . . , s , n ) | {z } | {z } m +1 m

TRS with regexp constraints • there is an exponential number of simplifying rules • currently implemented as LISP functions 
 (1 function represents a family of rules) • studying other compact rule-based representation, with • tree, context and function variables • regular constraints (variable in regular tree language) � � Kutsia, Marin Matching with Regular Constraints, 2005 � � problems: • matching • rewriting strategies (bottom-up) • characterization of set of descendants

example 0 0.45 1 initial tree (complete) 5 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 n s s s s s s s s r r r r r r r r r r r

example 0 0.45 1 normalize reduce normalize (conservative) (non-conservative) (conservative) 5 5 5 2 2 n s r r r n s r r n s r r 2 2 r r s r r r

example (2) 0 0.45 1 other initial tree (complete) 2 2 2 5 5 5 5 n s s s s s s s s r r r r r r r r r r r