Perceptually informed organization of textural sounds OFAI research - PowerPoint PPT Presentation

Austrian Research Institute for Artificial Intelligence (OFAI) Thomas Grill Perceptually informed organization of textural sounds OFAI research seminar, 2012-10-23

ringing cheeping gasping smashing piercing peeping whooping tinkling raucous chattering crooning bellowing sobbing bumping snarling growling pitch crying thumping burping croaking clattering yapping keening splashing yelping rustling volume squealing howling barking sniveling moaning pealing tone rattling grunting clanging coughing quacking whining gagging fizzing wheezing honking hissing bawling trumpeting swishing sneezing rumbling bubbling ripping cooing chirping shouting shuffling tearing popping roaring thunderous scratching snorting crashing crunching cackling tolling clucking silent tapping soothing crowing tranquil melodious cacophonous singing quiet tune loud tinkling noisy rhythmic mumbling twittering din beat blaring cawing racket chattering murmuring whistling clapping booming whispering mewing snapping snoring yelling mooing crackling sighing

Fundamental questions • How can digital sound material be described? • How can sounds be organized? • How can sounds and collections thereof be visualized? • Focus on sampled sound with textural characteristics Thomas Grill: Perceptually informed organization of textural sounds 4

2.2 Working Definition of Sound Textures First Time Constraint: Constant Long-term Characteristics A definition for a so~und texture could be quite wide, but we chose to restrict our working definition for many perceptual and conceptual reasons. First of all, there is no consensus among people as to what a sound texture might be; and more people will accept sounds that fit a more restrictive definition. The first constraint WC! put on our definition of a sound textures is that it should exhibit similar characteristics over time; that is, a two-second snippet of a texture should not differ significantly from another two-second snippet. To use another metaphor, one could say that any two snippets of a sound texture seem to be cut from the same rug [RIC79]. A sound texture is like wallpaper: it can have local structure and randomness, but the characteristics of the structure and randomness must remain constant on the large scale. This means that the pitch should not change like in a racing car, the rhythm should not increase or decrease, etc. This constraint also means that sounds in which the attack plays a great part (like many timbres) cannot be sound textures. A sound texture is characterized by its sustain. Figure 2.2.1 shows an interesting way of segregating sound tex- tures from other sounds, by showing how the “potential information content” increases with time. “Information” is taken here in the cog- nitive sense rather then the information theory sense. Speech or Textural sounds music can provide new information at any time, and their “potential information content” is shown here as a continuously increasing function of time. Textures, on the other hand, have constant long term characteristics, which translates into a flattening of the potential A sound texture is like wallpaper: it can have local structure information increase. Noise (in the auditory cognitive sense) has and randomness, but the characteristics of the structure and somewhat less information than textures. randomness must remain constant on the large scale. FIGURE 2.2.1 Potential Information Content of A Sound Texture vs. Time speech music content sound texture noise b time Saint-Arnaud, N. (1995). Classification of sound textures. Master’s thesis, MIT Media Lab, Cambridge, MA, USA Sounds that carry a lot of meaning are usually perceived as a message. The semantics take the foremost position in the cognition, Thomas Grill: Perceptually informed organization of textural sounds 5 downplaying the characteristics of the sound proper. We choose to work with sounds which are not primarily perceived as a message. Chapter 2 Human Perception of Sound Textures 24

Examples of audio textures • Natural sounds: fire, water (rain, waterfall, ocean), wind • Animal sounds: sea gulls, crickets, humming • Human utterances: babble, chatter • Machine sounds: buzz, whir, hammer, grumble, drone, traffic • Activity sounds: chip, sweep, rustle, typing, scroop, rasp, crumple, clap, rub, walking G. Strobl, G. Eckel and D. Rocchesso. “Sound Texture Modeling: A Survey”. Proceedings of the 2006 Sound and Music Computing (SMC) International Conference . Thomas Grill: Perceptually informed organization of textural sounds 6

Examples of audio textures windspiel1.aif Thomas Grill: Perceptually informed organization of textural sounds 7

Examples of audio textures tiere7.aif Thomas Grill: Perceptually informed organization of textural sounds 8

Sound-based music and textural sound • Sound-based music: “art form in which the sound and not the musical note is the basic unit.” ➡ Acousmatic music and soundscape composition ➡ Primacy of listening experience • Textural sound as "sound material" Landy, L. (2007). Understanding the Art of Sound Organization. The MIT Press, Cambridge, MA, USA. Truax, B. (2008). Soundscape composition as global music: Electroacoustic music as soundscape. Organised Sound, 13(2):103–109. Thomas Grill: Perceptually informed organization of textural sounds 9

Low Frequency Orchestra plays Robert Lettner: Das Spiel vom Kommen und Gehen Thomas Grill: Perceptually informed organization of textural sounds 10

Describing sounds • Predominant scheme: Semantic tagging (sound origin, recording context, etc.) • Sonic qualities are equally important/interesting, especially for abstract sounds or use in acousmatic composition • Description ⇨ Organization Thomas Grill: Perceptually informed organization of textural sounds 11

Identification of perceptual qualities in textural sounds • What are the most significant qualities of textural sounds? ➡ Repertory grid technique used to elicit qualities ( personal constructs ) "ex nihilo", for a specific selection of subjects (interviewees) and objects under examination ( items ) • Interviewees (subjects) are asked to name differences between two randomly chosen sound examples ➡ Bipolar qualities spanning range from one sound to the other Grill, Flexer and Cunningham. Identification of perceptual qualities in textural sounds using the repertory grid method. Proceedings of the 6th Audio Mostly Conference , 2011 Thomas Grill: Perceptually informed organization of textural sounds 12

Example • Straight differentiation : In which ways do two sounds differ? • Triads : Group three objects to form two groups, then name differences between groups Thomas Grill: Perceptually informed organization of textural sounds 13

Repertory Grid for sounds • Elicitation of ~10 bipolar constructs per subject • Subjects rate all 20 sounds (grades 1 to 5) using own personal constructs 1 … 5 motion textural impulse high excentric evolutionary well-defined regular narrative pitched smooth static coherent continuous low contained repetitive diffused irregular static non-pitched porous A 4 4 4 1 2 4 4 2 4 3 3 B 5 3 5 5 5 1 3 1 5 2 1 C 4 5 2 2 4 -5 5 3 5 5 4 D 4 2 5 4 3 4 4 3 4 2 3 E 2 4 1 1 2 4 1 5 5 3 5 F 1 1 2 2 2 -3 2 5 5 4 5 G 5 5 5 5 5 2 1 2 5 1 1 H 4 3 3 1 2 5 1 1 5 2 4 I 4 2 2 2 2 5 2 2 4 1 4 J 2 1 5 3 1 -2 5 5 3 5 3 K 5 2 4 4 4 4 3 1 5 4 2 L 1 1 1 3 1 -2 1 5 5 5 5 M 4 5 5 1 2 2 3 2 5 3 2 N 3 1 4 4 1 4 4 5 5 4 2 O 4 2 4 3 3 -3 5 4 3 5 3 P 2 2 3 3 3 4 5 3 5 5 4 Q 5 5 5 3 5 -5 1 1 5 1 1 R 3 3 4 2 3 2 2 3 4 2 3 S 2 2 5 2 3 4 4 4 2 3 2 T 1 1 4 4 1 4 3 2 3 5 2 Thomas Grill: Perceptually informed organization of textural sounds 14

high/low • 16 subjects • expert listeners • 202 constructs • mostly German ordered/chaotic Thomas Grill: Perceptually informed organization of textural sounds 15

http://grrrr.org/test/classify Thomas Grill: Perceptually informed organization of textural sounds 16

Inter-rater agreement Agreement α Agreement α Construct (core group)* (all n ≥ 10) high – low 0.588 0.519 ordered – chaotic 0.556 0.447 natural – artificial 0.551 0.492 smooth – coarse 0.527 0.420 tonal – noisy 0.523 0.435 homogeneous – heterogeneous 0.519 0.416 dense – sparse 0.492 0.342 edgy – flowing 0.465 0.376 static – dynamic 0.403 0.383 near – far 0.252 0.249 *nine subjects who took part in the elicitation process Thomas Grill: Perceptually informed organization of textural sounds 17

Sounds along axis high–low low ⟶ ⟵ high Thomas Grill: Perceptually informed organization of textural sounds 18

Pearson correlation between constructs Thomas Grill: Perceptually informed organization of textural sounds 19

Pearson correlation between constructs Thomas Grill: Perceptually informed organization of textural sounds 20