Visualization of Perceptual Qualities in Textural Sounds DAF-x - PowerPoint PPT Presentation

Austrian Research Institute for Artificial Intelligence (OFAI) Thomas Grill (OFAI), Ulla Rauter Visualization of Perceptual Qualities in Textural Sounds DAF-x 2011, IRCAM/Paris/France

Multi-modal perception [*] Wolfgang Köhler, Gestalt psychology ,1929 Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 2 DAF-x 2011, IRCAM/Paris/France

Multi-modal perception • Ecological perception [Gibson] • Embodied sound and music cognition [Leman, Noë, Godøy, Jensenius] Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 3 DAF-x 2011, IRCAM/Paris/France

so far: Elicitation of perceptual qualities of sound textures • Repertory grid method : Systematic approach to uncover PCs, with minimal intervention by the interviewer • Selection of 20 varied textures presented to 16 interviewees • Elicitation of 10 "independent" qualities • Follow-up online survey using 100 textures: 104 subjects in total, >16800 rates • Evaluation : Interrater agreement, correlation measures Grill, Flexer, Cunningham, Identification of perceptual qualities in textural sounds using the repertory grid method , Audio Mostly 2011 Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 4 DAF-x 2011, IRCAM/Paris/France

so far: Relevant perceptual constructs for the description of textural sounds Agreement α Agreement α Construct (core group)* (all n ≥ 10) high – low 0.588 0.519 ordered – chaotic 0.556 0.447 natural – artificial 0.551 0.492 smooth – coarse 0.527 0.420 tonal – noisy 0.523 0.435 homogeneous – heterogeneous 0.519 0.416 dense – sparse 0.492 0.342 edgy – flowing 0.465 0.376 static – dynamic 0.403 0.383 near – far 0.252 0.249 *nine subjects who took part in the elicitation process Grill, Flexer, Cunningham, Identification of perceptual qualities in textural sounds using the repertory grid method , Audio Mostly 2011 Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 5 DAF-x 2011, IRCAM/Paris/France

Transferring auditory qualities to the visual domain • Various different sketches for graphic representations • Should allow iconic and continuous map representations • Intuitive first approach, then further iteratively refined Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 6 DAF-x 2011, IRCAM/Paris/France

Transferring auditory qualities to the visual domain • high–low ⇨ brightness and color hue (red–yellow) • ordered–chaotic ⇨ arrangement of elements • smooth–coarse ⇨ smoothness of element outline • tonal–noisy ⇨ color saturation (grey–colorful) • homogeneous–heterogeneous ⇨ variation in color-related parameters, especially hue Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 7 DAF-x 2011, IRCAM/Paris/France

Online survey: Visualization of textural sounds http://grrrr.org/test/texvis Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 8 DAF-x 2011, IRCAM/Paris/France

Online survey: Visualization of textural sounds http://grrrr.org/test/texvis Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 9 DAF-x 2011, IRCAM/Paris/France

Evaluation: Average RMS error Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 10 DAF-x 2011, IRCAM/Paris/France

Evaluation: Pearson correlation selection to reference Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 11 DAF-x 2011, IRCAM/Paris/France

Evaluation: Pearson correlation selection to reference Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 11 DAF-x 2011, IRCAM/Paris/France

Evaluation: Pearson correlation selection to reference Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 11 DAF-x 2011, IRCAM/Paris/France

Musical interface: Continuous map of textural sounds http://grrrr.org/test/texvis/map.html Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 12 DAF-x 2011, IRCAM/Paris/France

Thanks for your attention! • Your comments please! • Funded by Vienna Science and Technology Fund (WWTF project Audiominer MA09-024 ) • Get in contact: Thomas Grill – http://grrrr.org Grill, Rauter: Visualization of Perceptual Qualities in Textural Sounds 13 DAF-x 2011, IRCAM/Paris/France