Introduction Non negative-Matrix Factorization Spectrogram model Decomposition of musical spectrograms Conclusion TIME-DEPENDENT PARAMETRIC AND HARMONIC TEMPLATES IN NON-NEGATIVE MATRIX FACTORIZATION 13 th International Conference on Digital Audio Effects Romain Hennequin, Roland Badeau and Bertrand David Telecom ParisTech September 8, 2010 Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 1/26
Introduction Non negative-Matrix Factorization Spectrogram model Decomposition of musical spectrograms Conclusion Introduction Musical spectrograms decomposition (on a basis of notes) Decomposition based on Non-negative Matrix Factorization (NMF) Spectrogram models are introduced into decomposition methods: parametric harmonic atoms makes it possible to model slight pitch variations Potential applications: Multipitch estimation/transcription Source separation Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 2/26
Introduction Non negative-Matrix Factorization Spectrogram model Decomposition of musical spectrograms Conclusion Sommaire 1 Non negative-Matrix Factorization 2 Spectrogram model 3 Decomposition of musical spectrograms Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 3/26
Introduction Non negative-Matrix Factorization Principle Spectrogram model Issues Decomposition of musical spectrograms Proposed solution Conclusion Contents 1 Non negative-Matrix Factorization Principle Issues Proposed solution 2 Spectrogram model 3 Decomposition of musical spectrograms Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 4/26
Introduction Non negative-Matrix Factorization Principle Spectrogram model Issues Decomposition of musical spectrograms Proposed solution Conclusion Principle of NMF Low-rank approximation: R � V ≈ ˆ ˆ V = WH V ft = W fr H rt r =1 Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 5/26
Introduction Non negative-Matrix Factorization Principle Spectrogram model Issues Decomposition of musical spectrograms Proposed solution Conclusion Issues with NMF Pitch variations Low-rank approximation does not permit to model variations over time, such as slight pitch variations (vibrato. . . ). Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 6/26
Introduction Non negative-Matrix Factorization Principle Spectrogram model Issues Decomposition of musical spectrograms Proposed solution Conclusion Issues with NMF Original spectrogram NMF spectrogram R = 1 5 5 4 4 frequency (kHz) frequency (kHz) 3 3 2 2 1 1 0 0 50 100 150 50 100 150 time (frames) time (frames) Note with vibrato: Decomposition with a single atom. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 7/26
Introduction Non negative-Matrix Factorization Principle Spectrogram model Issues Decomposition of musical spectrograms Proposed solution Conclusion Issues with NMF Original spectrogram NMF spectrogram R = 3 5 5 4 4 frequency (kHz) frequency (kHz) 3 3 2 2 1 1 0 0 50 100 150 50 100 150 time (frames) time (frames) Note with vibrato: Decomposition with 3 atoms. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 8/26
Introduction Non negative-Matrix Factorization Principle Spectrogram model Issues Decomposition of musical spectrograms Proposed solution Conclusion Proposed solution What does an atom look like in a musical spectrogram? In a musical spectrogram most of the (non-percussive) elements are instruments notes which are generally harmonic tones. Parameters of interest are generally the fundamental frequency of these tones, and the shape of the amplitudes of the harmonics. Proposed method: parametric model of spectrogram with harmonic atoms. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 9/26
Introduction Non negative-Matrix Factorization Parametric spectrogram Spectrogram model Parametric atoms Decomposition of musical spectrograms Algorithm Conclusion Contents 1 Non negative-Matrix Factorization 2 Spectrogram model Parametric spectrogram Parametric atoms Algorithm 3 Decomposition of musical spectrograms Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 10/26
Introduction Non negative-Matrix Factorization Parametric spectrogram Spectrogram model Parametric atoms Decomposition of musical spectrograms Algorithm Conclusion Parametric spectrogram Time-varying atoms in NMF: R R � � W θ rt ˆ ˆ V ft = → V ft = W fr H rt fr H rt r =1 r =1 θ rt is a time-varying parameter associated to each atom. In this paper, θ rt is the fundamental frequency f rt of each atom. 0 Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 11/26
Introduction Non negative-Matrix Factorization Parametric spectrogram Spectrogram model Parametric atoms Decomposition of musical spectrograms Algorithm Conclusion Parametric atoms Parametric harmonic atom construction n h ( f rt 0 ) � W f rt a k g ( f − kf rt fr = 0 0 ) k =1 Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 12/26
Introduction Non negative-Matrix Factorization Parametric spectrogram Spectrogram model Parametric atoms Decomposition of musical spectrograms Algorithm Conclusion Parametric spectrogram Hypotheses of the model The harmonic part of notes is supposed to be stationary within an analysis frame. Interferences between harmonics are supposed to be negligible. Classical hypothesis of NMF about positive summation of parts. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 13/26
Introduction Non negative-Matrix Factorization Parametric spectrogram Spectrogram model Parametric atoms Decomposition of musical spectrograms Algorithm Conclusion Algorithm Parametric spectrogram R n h � � ˆ a k g ( f − kf rt V ft = 0 ) h rt r =1 k =1 � �� � f rt 0 W fr Learnt parameters A divergence between V and ˆ V is to be minimized w.r.t.: f rt 0 : the fundamental frequency of each atom at each frame a k : the amplitudes of harmonics (Atoms share the same set of amplitudes) h rt : the activation of each atom at each frame 0 , a k , h rt ) = D ( V ft | ˆ Cost function: C ( f rt V ft ) Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 14/26
Introduction Non negative-Matrix Factorization Parametric spectrogram Spectrogram model Parametric atoms Decomposition of musical spectrograms Algorithm Conclusion Algorithm Minimization Global optimization w.r.t. f rt is impossible (numerous local 0 minima in C ). ⇒ one atom is introduced for each MIDI note. Optimization thus becomes local (fine estimate of f rt 0 ). Minimization achieved with multiplicative update rules. Remark The proposed method is no longer a rank-reduction method but still reduces the data dimension. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 15/26
Introduction Decomposition Non negative-Matrix Factorization Improvement Spectrogram model Estimated frequency Decomposition of musical spectrograms Real signals Conclusion Contents 1 Non negative-Matrix Factorization 2 Spectrogram model 3 Decomposition of musical spectrograms Decomposition Improvement Estimated frequency Real signals Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 16/26
Introduction Decomposition Non negative-Matrix Factorization Improvement Spectrogram model Estimated frequency Decomposition of musical spectrograms Real signals Conclusion Decomposition of a synthetic spectrogram Original power spectrogram 40 5 35 30 4 25 Frequency (kHz) 20 3 15 10 2 5 0 1 −5 0 50 100 150 200 250 300 Time (frame) Spectrogram of the first bars of JS Bach’s first prelude played by a synthesizer. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 17/26
Introduction Decomposition Non negative-Matrix Factorization Improvement Spectrogram model Estimated frequency Decomposition of musical spectrograms Real signals Conclusion Obtained decomposition 70 −15 60 50 −20 Semitones 40 −25 30 20 −30 10 −35 50 100 150 200 250 300 Frames Activations for each MIDI note. Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 18/26
Introduction Decomposition Non negative-Matrix Factorization Improvement Spectrogram model Estimated frequency Decomposition of musical spectrograms Real signals Conclusion Obtained decomposition Decomposition Notes appear at the right place with decreasing amplitudes Numerous atoms activated at onset time Notes activated at octave, twelfth and double octave of the right note (note with many common partials). Romain Hennequin, Roland Badeau and Bertrand David Time-dependent parametric templates in NMF - slide 19/26
Recommend
More recommend
