music structure analysis
play

Music Structure Analysis Meinard Mller International Audio - PowerPoint PPT Presentation

Lecture Music Processing Music Structure Analysis Meinard Mller International Audio Laboratories Erlangen meinard.mueller@audiolabs-erlangen.de Book: Fundamentals of Music Processing Meinard Mller Fundamentals of Music Processing Audio,


  1. SSM Enhancement Block Enhancement  Feature smoothing  Coarsening Time (samples) Time (samples)

  2. SSM Enhancement Challenge: Presence of musical variations  Fragmented paths and gaps  Paths of poor quality  Regions of constant (high) similarity  Curved paths Idea: Enhancement of path structure

  3. SSM Enhancement Shostakovich Waltz 2, Jazz Suite No. 2 (Chailly) SSM

  4. SSM Enhancement Shostakovich Waltz 2, Jazz Suite No. 2 (Chailly) SSM

  5. SSM Enhancement Shostakovich Waltz 2, Jazz Suite No. 2 (Chailly) SSM

  6. SSM Enhancement Shostakovich Waltz 2, Jazz Suite No. 2 (Chailly) Enhanced SSM Filtering along main diagonal

  7. SSM Enhancement Idea: Usage of contextual information (Foote 1999)  Comparison of entire sequences  = length of sequences  = enhanced SSM smoothing effect

  8. SSM Enhancement SSM

  9. SSM Enhancement Enhanced SSM with Filtering along main diagonal

  10. SSM Enhancement Enhanced SSM with Filtering along 8 different directions and minimizing

  11. SSM Enhancement Idea: Smoothing along various directions and minimizing over all directions Tempo changes of -50 to +50 percent

  12. SSM Enhancement Path Enhancement Time (samples) Time (samples)

  13. SSM Enhancement Path Enhancement  Diagonal smoothing Time (samples) Time (samples)

  14. SSM Enhancement Path Enhancement  Diagonal smoothing  Multiple filtering Time (samples) Time (samples)

  15. SSM Enhancement Path Enhancement  Diagonal smoothing  Multiple filtering  Thresholding (relative)  Scaling & penalty Time (samples) Time (samples)

  16. SSM Enhancement Further Processing  Path extraction Time (samples) Time (samples)

  17. SSM Enhancement Further Processing  Path extraction  Pairwise relations 1 Time (samples) 2 3 4 5 6 7 100 200 300 400 Time (samples) Time (samples)

  18. SSM Enhancement Further Processing  Path extraction  Pairwise relations Grouping (transitivity)  1 Time (samples) 2 3 4 5 6 7 100 200 300 400 Time (samples) Time (samples)

  19. SSM Enhancement Further Processing  Path extraction  Pairwise relations Grouping (transitivity)  1 Time (samples) 2 3 4 5 6 7 100 200 300 400 Time (samples) 100 200 300 400 Time (samples) Time (samples)

  20. SSM Enhancement Example: Zager & Evans “In The Year 2525” I V1 V2 V3 V4 V5 V6 V7 B V8 O

  21. SSM Enhancement Example: Zager & Evans “In The Year 2525”

  22. SSM Enhancement Example: Zager & Evans “In The Year 2525” Missing relations because of transposed sections

  23. SSM Enhancement Example: Zager & Evans “In The Year 2525” Idea: Cyclic shift of one of the chroma sequences One semitone up

  24. SSM Enhancement Example: Zager & Evans “In The Year 2525” Idea: Cyclic shift of one of the chroma sequences Two semitones up

  25. SSM Enhancement Example: Zager & Evans “In The Year 2525” Idea: Overlay & Maximize Transposition-invariant SSM

  26. SSM Enhancement Example: Zager & Evans “In The Year 2525” Note: Order of enhancement steps important! Maximization Smoothing & Maximization

  27. Similarity Matrix Toolbox Meinard Müller, Nanzhu Jiang, Harald Grohganz SM Toolbox: MATLAB Implementations for Computing and Enhancing Similarity Matrices http://www.audiolabs-erlangen.de/resources/MIR/SMtoolbox/

  28. Overview Thanks:  Introduction  Jiang, Grosche  Feature Representations  Peeters  Cooper, Foote  Self-Similarity Matrices  Goto  Levy, Sandler  Mauch  Audio Thumbnailing  Sapp  Novelty-based Segmentation

  29. Audio Thumbnailing General goal: Determine the most representative section (“Thumbnail”) of a given music recording. Example: Zager & Evans “In The Year 2525” I V1 V2 V3 V4 V5 V6 V7 B V8 O Example: Brahms Hungarian Dance No. 5 (Ormandy) A1 A2 B1 B2 C A3 B3 B4 Thumbnail is often assumed to be the most repetitive segment

  30. Audio Thumbnailing Two steps Both steps are problematic! Paths of poor quality (fragmented, gaps)  1. Path extraction  Block-like structures  Curved paths  Noisy relations 2. Grouping (missing, distorted, overlapping)  Transitivity computation difficult Main idea: Do both, path extraction and grouping, jointly  One optimization scheme for both steps  Stabilizing effect  Efficient

  31. Audio Thumbnailing Main idea: Do both path extraction and grouping jointly  For each audio segment we define a fitness value  This fitness value expresses “how well” the segment explains the entire audio recording  The segment with the highest fitness value is considered to be the thumbnail  As main technical concept we introduce the notion of a path family

  32. Fitness Measure Enhanced SSM 1 200 180 0.5 160 140 0 120 −0.5 100 80 −1 60 40 −1.5 20 0 −2 0 50 100 150 200

  33. Fitness Measure Path over segment  Consider a fixed segment

  34. Fitness Measure Path over segment  Consider a fixed segment  Path over segment  Induced segment  Score is high

  35. Fitness Measure Path over segment  Consider a fixed segment  Path over segment  Induced segment  Score is high  A second path over segment  Induced segment  Score is not so high

  36. Fitness Measure Path over segment  Consider a fixed segment  Path over segment  Induced segment  Score is high  A second path over segment  Induced segment  Score is not so high  A third path over segment  Induced segment  Score is very low

  37. Fitness Measure Path family  Consider a fixed segment  A path family over a segment is a family of paths such that the induced segments do not overlap.

  38. Fitness Measure Path family  Consider a fixed segment  A path family over a segment is a family of paths such that the induced segments do not overlap. This is not a path family!

  39. Fitness Measure Path family  Consider a fixed segment  A path family over a segment is a family of paths such that the induced segments do not overlap. This is a path family! (Even though not a good one)

  40. Fitness Measure Optimal path family  Consider a fixed segment

  41. Fitness Measure Optimal path family  Consider a fixed segment  Consider over the segment the optimal path family, i.e., the path family having maximal overall score.  Call this value: Score(segment) Note: This optimal path family can be computed using dynamic programming.

  42. Fitness Measure Optimal path family  Consider a fixed segment  Consider over the segment the optimal path family, i.e., the path family having maximal overall score.  Call this value: Score(segment)  Furthermore consider the amount covered by the induced segments.  Call this value: Coverage(segment)

  43. Fitness Measure Fitness  Consider a fixed segment P := Score(segment) R := Coverage(segment)

  44. Fitness Measure Fitness  Consider a fixed segment  Self-explanation are trivial! P := Score(segment) R := Coverage(segment)

  45. Fitness Measure Fitness  Consider a fixed segment  Self-explanation are trivial!  Subtract length of segment P := Score(segment) - length(segment) R := Coverage(segment) - length(segment)

  46. Fitness Measure Fitness  Consider a fixed segment  Self-explanation are trivial!  Subtract length of segment  Normalization  [ 0 , 1 ] P := Score(segment) - length(segment) Normalize( )  [ 0 , 1 ] Normalize( ) R := Coverage(segment) - length(segment)

  47. Fitness Measure Fitness  Consider a fixed segment Fitness(segment) F := 2 • P • R / (P + R)  [ 0 , 1 ] P := Score(segment) - length(segment) Normalize( )  [ 0 , 1 ] Normalize( ) R := Coverage(segment) - length(segment)

  48. Thumbnail Fitness Scape Plot Fitness Segment length Segment length Segment center Segment center

  49. Thumbnail Fitness Scape Plot Fitness Segment length Fitness(segment) Segment length Segment center Segment center

  50. Thumbnail Fitness Scape Plot Fitness Segment length Segment center

Recommend


More recommend