Austrian Research Institute for Artificial Intelligence (OFAI) Thomas Grill Sound quality of textural audio: characterization, modeling and visualization ESI Modern Methods of Time-Frequency Analysis II, Time-frequency methods for the applied sciences, 2012-12-07
L'objet sonore – the sonorous object Textural : la masse, le timbre harmonique, le grain, l'allure Gestural : la dynamique, le profil mélodique, le profil de masse Pierre Schaeffer: Traité des objets musicaux (1966), Solfège de l‘objet sonore (1967) Trevor Wishart: On Sonic Art (1985) Rolf Inge Godøy: Chunking Sound for Musical Analysis , CMMR 2008 Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 2
Examples of textural sounds • Activity sounds: chip, sweep, rustle, typing, scroop, rasp, crumple, clap, rub, walking • Machine sounds: buzz, whir, hammer, grumble, drone, traffic • Natural sounds: fire, water (rain, waterfall, ocean), wind • Animal sounds: sea gulls, crickets, humming • Human utterances: babble, chatter G. Strobl, G. Eckel and D. Rocchesso. “Sound Texture Modeling: A Survey”. Proceedings of the 2006 Sound and Music Computing (SMC) International Conference . Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 3
Examples of textural sounds 12800 6400 frequency [Hz] 3200 1600 800 400 200 100 0.0 0.5 1.0 1.5 2.0 2.5 time [s] Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 4
Examples of textural sounds 12800 6400 frequency [Hz] 3200 1600 800 400 200 100 0.0 0.5 1.0 1.5 2.0 2.5 time [s] Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 5
2.2 Working Definition of Sound Textures First Time Constraint: Constant Long-term Characteristics A definition for a so~und texture could be quite wide, but we chose to restrict our working definition for many perceptual and conceptual reasons. First of all, there is no consensus among people as to what a sound texture might be; and more people will accept sounds that fit a more restrictive definition. The first constraint WC! put on our definition of a sound textures is that it should exhibit similar characteristics over time; that is, a two-second snippet of a texture should not differ significantly from another two-second snippet. To use another metaphor, one could say that any two snippets of a sound texture seem to be cut from the same rug [RIC79]. A sound texture is like wallpaper: it can have local structure and randomness, but the characteristics of the structure and randomness must remain constant on the large scale. This means that the pitch should not change like in a racing car, the rhythm should not increase or decrease, etc. This constraint also means that sounds in which the attack plays a great part (like many timbres) cannot be sound textures. A sound texture is characterized by its sustain. Figure 2.2.1 shows an interesting way of segregating sound tex- tures from other sounds, by showing how the “potential information content” increases with time. “Information” is taken here in the cog- nitive sense rather then the information theory sense. Speech or Textural sound music can provide new information at any time, and their “potential information content” is shown here as a continuously increasing function of time. Textures, on the other hand, have constant long term characteristics, which translates into a flattening of the potential A sound texture is like wallpaper: it can have local structure information increase. Noise (in the auditory cognitive sense) has and randomness, but the characteristics of the structure and somewhat less information than textures. randomness must remain constant on the large scale. FIGURE 2.2.1 Potential Information Content of A Sound Texture vs. Time speech music content sound texture noise b time Saint-Arnaud, N. (1995). Classification of sound textures. Master’s thesis, MIT Media Lab, Cambridge, MA, USA Sounds that carry a lot of meaning are usually perceived as a message. The semantics take the foremost position in the cognition, Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 6 downplaying the characteristics of the sound proper. We choose to work with sounds which are not primarily perceived as a message. Chapter 2 Human Perception of Sound Textures 24
Sound-based music and textural sound • Sound-based music: “art form in which the sound and not the musical note is the basic unit.” ➡ Acousmatic music and soundscape composition • Textural sound as "sound material" Landy, L. (2007). Understanding the Art of Sound Organization. The MIT Press, Cambridge, MA, USA. Truax, B. (2008). Soundscape composition as global music: Electroacoustic music as soundscape. Organised Sound, 13(2):103–109. Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 7
Low Frequency Orchestra plays Robert Lettner: Das Spiel vom Kommen und Gehen Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 8
Fundamental questions • How can textural sounds be described? • How can (textural) sounds be organized? • How can textural sounds and collections thereof be visualized? Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 10
Describing sounds • Predominant scheme: Semantic tagging (sound origin, recording context, etc.) • Sonic qualities are equally important/interesting, especially for abstract sounds or use in acousmatic composition • Description ⇨ Organization Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 11
Identification of perceptual qualities in textural sounds • What are the most significant qualities of textural sounds? ➡ Repertory grid technique used to elicit qualities ( personal constructs ) "ex nihilo", for a specific selection of subjects (interviewees) and objects under examination ( items ) • Interviewees (subjects) are asked to name differences between two randomly chosen sound examples ➡ Bipolar qualities spanning range from one sound to the other Grill, Flexer and Cunningham. Identification of perceptual qualities in textural sounds using the repertory grid method. Proceedings of the 6th Audio Mostly Conference , 2011 Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 12
Example • Straight differentiation : In which ways do two sounds differ? • Triads : Group three objects to form two groups, then name differences between groups Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 13
Repertory Grid for sounds • Elicitation of ~10 bipolar constructs per subject • Subjects rate all 20 sounds (grades 1 to 5) using own personal constructs 1 … 5 motion textural impulse high excentric evolutionary well-defined regular narrative pitched smooth static coherent continuous low contained repetitive diffused irregular static non-pitched porous A 4 4 4 1 2 4 4 2 4 3 3 B 5 3 5 5 5 1 3 1 5 2 1 C 4 5 2 2 4 -5 5 3 5 5 4 D 4 2 5 4 3 4 4 3 4 2 3 E 2 4 1 1 2 4 1 5 5 3 5 F 1 1 2 2 2 -3 2 5 5 4 5 G 5 5 5 5 5 2 1 2 5 1 1 H 4 3 3 1 2 5 1 1 5 2 4 I 4 2 2 2 2 5 2 2 4 1 4 J 2 1 5 3 1 -2 5 5 3 5 3 K 5 2 4 4 4 4 3 1 5 4 2 L 1 1 1 3 1 -2 1 5 5 5 5 M 4 5 5 1 2 2 3 2 5 3 2 N 3 1 4 4 1 4 4 5 5 4 2 O 4 2 4 3 3 -3 5 4 3 5 3 P 2 2 3 3 3 4 5 3 5 5 4 Q 5 5 5 3 5 -5 1 1 5 1 1 R 3 3 4 2 3 2 2 3 4 2 3 S 2 2 5 2 3 4 4 4 2 3 2 T 1 1 4 4 1 4 3 2 3 5 2 Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 14
high/low • 16 subjects • expert listeners • 202 constructs • mostly German ordered/chaotic Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 15
http://grrrr.org/test/classify Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 16
Inter-rater agreement Agreement α Agreement α Construct (core group)* (all n ≥ 10) high – low 0.588 0.519 ordered – chaotic 0.556 0.447 natural – artificial 0.551 0.492 smooth – coarse 0.527 0.420 tonal – noisy 0.523 0.435 homogeneous – heterogeneous 0.519 0.416 dense – sparse 0.492 0.342 edgy – flowing 0.465 0.376 static – dynamic 0.403 0.383 near – far 0.252 0.249 *nine subjects who took part in the elicitation process Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 17
Sounds along axis high–low low ⟶ ⟵ high Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 18
Pearson correlation between constructs Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 19
Pearson correlation between constructs Thomas Grill: Sound quality of textural audio: characterization, modeling and visualization 20
Recommend
More recommend
