Austrian Research Institute for Artificial Intelligence (OFAI) Thomas Grill, Arthur Flexer (OFAI), Stuart Cunningham (CAIR) Identification of perceptual qualities in textural sounds using the repertory grid method Audio Mostly 2011, Coimbra/Portugal
Interfacing with recorded sounds • Sound designers and electronic musicians maintain huge collections of sampled digital sounds • Usual categorization scheme: Semantic tagging (sound origin, recording context, etc.) • Sonic qualities are equally important/interesting for musical use of samples as sound material • Samples are often non-instrumental sounds (atmospheres etc.) Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 3 Audio Mostly 2011, Coimbra/Portugal
cataRT : Interfacing with sounds using MIR-style audio descriptors Periodicity ⟶ SpectralCentroid ⟶ Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 4 Audio Mostly 2011, Coimbra/Portugal
Larger scope: Sample-based electronic instruments Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 5 Audio Mostly 2011, Coimbra/Portugal
Larger scope: Sample-based electronic instruments • For live-performance in improvised or composed music • No pre-listening • Be agile! • Be expressive! • Be present! ➡ Look into strategies of embodied interaction ➡ Have agents assist you (in clever ways) Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 6 Audio Mostly 2011, Coimbra/Portugal
Textural sounds • A sound texture should exhibit similar characteristics over time . It can have local structure and randomness but the characteristics of the fine structure must remain constant on the large scale. • A sound texture is characterized by its sustain. [...] High-level characteristics must be exposed within the attention span of a few seconds. [*] N. Saint-Arnaud and K. Popat. “Analysis and Synthesis of Sound Textures”, Auditory Scene Analysis, D. F. Rosenthal, Horoshi G. Okuno, editors. Lawrence Erlbaum Association, New Jersey, 1998 Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 7 Audio Mostly 2011, Coimbra/Portugal
Textural sounds (example) Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 8 Audio Mostly 2011, Coimbra/Portugal
Sounds and perception • Non-expert vs. expert listening • Connotation of sound vs. Reduced Listening [1] • Attribution of Personal Constructs (PCs) [2] [1] Pierre Schaeffer, Traité des objets musicaux, 1966 [2] George Kelly, The psychology of personal constructs, 1955 Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 9 Audio Mostly 2011, Coimbra/Portugal
Repertory Grid interview method • Systematic approach to uncover PCs • PCs represent world-views • Presentation of examples to represent a specific part of the world • Choice of subjects helps to discover group norms • Minimal intervention by the interviewer [*] George Kelly, The psychology of personal constructs, 1955 Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 10 Audio Mostly 2011, Coimbra/Portugal
Repertory Grid for sounds / 1 • Focusing on textural sounds, both natural and abstract • Hours of sounds boiled down to 100 examples (5–10 secs) • Again condensed to 20 most varied textures • Interviewees (subjects) are asked to name differences between two randomly chosen examples Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 11 Audio Mostly 2011, Coimbra/Portugal
Example • Straight differentiation : In which ways do two sounds differ? • Triads : Group three objects to form two groups, then name differences between groups Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 12 Audio Mostly 2011, Coimbra/Portugal
Test setup • Sounds mixed-down to mono, loudness normalized, about 5 seconds long • One close high-quality speaker (single source) • Subjects from the authors' research and artistic contexts • Subjects were triggered to focus on the perceived quality rather than on potential origins or sources • Sounds randomly chosen, no visible file names etc. Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 13 Audio Mostly 2011, Coimbra/Portugal
Repertory Grid for sounds / 2 • Elicitation of 10–15 bipolar constructs per subject • Subjects rate all 20 sounds (grades 1 to 5) using own personal constructs 1 … 5 motion textural impulse high excentric evolutionary well-defined regular narrative pitched smooth static coherent continuous low contained repetitive diffused irregular static non-pitched porous A 4 4 4 1 2 4 4 2 4 3 3 B 5 3 5 5 5 1 3 1 5 2 1 C 4 5 2 2 4 -5 5 3 5 5 4 D 4 2 5 4 3 4 4 3 4 2 3 E 2 4 1 1 2 4 1 5 5 3 5 F 1 1 2 2 2 -3 2 5 5 4 5 G 5 5 5 5 5 2 1 2 5 1 1 H 4 3 3 1 2 5 1 1 5 2 4 I 4 2 2 2 2 5 2 2 4 1 4 J 2 1 5 3 1 -2 5 5 3 5 3 K 5 2 4 4 4 4 3 1 5 4 2 L 1 1 1 3 1 -2 1 5 5 5 5 M 4 5 5 1 2 2 3 2 5 3 2 N 3 1 4 4 1 4 4 5 5 4 2 O 4 2 4 3 3 -3 5 4 3 5 3 P 2 2 3 3 3 4 5 3 5 5 4 Q 5 5 5 3 5 -5 1 1 5 1 1 R 3 3 4 2 3 2 2 3 4 2 3 S 2 2 5 2 3 4 4 4 2 3 2 T 1 1 4 4 1 4 3 2 3 5 2 Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 14 Audio Mostly 2011, Coimbra/Portugal
Repertory Grid for sounds / 3 • Comparison of PCs possible using (Euclidean) distances between 20-dimensional vectors characterizing each PC • Self-similarity matrix of PCs (in total 202 × 202 elements) • Low distance indicates synonymy of PCs (among all subjects) • Hierarchical clustering for grouping Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 15 Audio Mostly 2011, Coimbra/Portugal
high/low • 16 subjects • expert listeners • 202 constructs • mostly German ordered/chaotic Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 16 Audio Mostly 2011, Coimbra/Portugal
Distilled constructs PCs in German original PCs translated to English hoch – tief high – low hell – dunkel bright – dull regelmäßig – unregelmäßig ordered – chaotic geordnet – chaotisch coherent – erratic glatt – rau smooth – coarse weich – rau soft – raspy natürlich – künstlich natural – artificial analog – digital analog – digital statisch – dynamisch static – dynamic starr – bewegt rigid – eventful nahe – fern near – far klar – verschwommen clear – blurred kantig – rund edgy – flowing zerrissen – kompakt disjointed – continuous dicht – spärlich dense – sparse flächig – punktuell expansive – selective homogen – heterogen homogeneous – heterogeneous gleichförmig – differenziert uniform – differentiated tonhaltig – geräuschhaft tonal – noisy Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 17 Audio Mostly 2011, Coimbra/Portugal
Evaluate constructs on full set of sounds • Web-based form connected to a database • Rating of random examples from the full set of 100 sounds in respect to the distilled constructs • Posted to various forums (music-IR, auditory, etc.) • German and English version Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 18 Audio Mostly 2011, Coimbra/Portugal
http://grrrr.org/test/classify Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 19 Audio Mostly 2011, Coimbra/Portugal
Evaluation • 104 subjects in total • 59 (35 german, 24 english) with more than 10 ratings • >16800 grades on 10 qualities / 100 sounds • on average 9.4 constructs rated per sound • each item has been graded at least 10 times, on average more than 16 times • Variance normalization per user (to account for different grading habits) Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 20 Audio Mostly 2011, Coimbra/Portugal
Sounds along axis high–low low ⟶ ⟵ high Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 21 Audio Mostly 2011, Coimbra/Portugal
Sounds along axis ordered–chaotic chaotic ⟶ ⟵ ordered Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 22 Audio Mostly 2011, Coimbra/Portugal
Sounds along axis tonal–noisy noisy ⟶ ⟵ tonal Grill, Flexer, Cunningham: Identification of perceptual qualities in textural sounds using the repertory grid method 23 Audio Mostly 2011, Coimbra/Portugal
Recommend
More recommend
