Pacific Belltower sound installation for live sonification of earthquake Internet data PerMagnus Lindborg , PhD International Computer Music Conference ICMC–EMW, Shanghai, 18 October 2017 1 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg sonification ~ data-driven art earthquakes ~ concept implementation in Max parameter mapping & auditory perception 2 Wednesday, October 18, 2017
Wednesday, October 18, 2017
Pacific Belltower, with Chen Qiangbin (ICMC Chair) (sound, 2 min) & Ping Jin (ICMC Music Chair) Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg sonification …for discovery of patterns in data …for generating sonic material, i.e. ‘scientific sonification’ i.e. ‘aesthetic sonification’ “Sonification is the use of nonspeech audio to “Questions of aesthetics and convey information.… the transformation of data musicality remain open in the fi eld of relations into perceived relations in an acoustic soni fi cation.” (Barrass & Vickers 2011) signal for the purposes of facilitating ~ data-driven art? communication or interpretation.” (Kramer 1994) “Perceptualization” (Grinstein & Smith 1990) concrete listening modes / intentions (Schaeffer 1966; Tuuri & Eerola 2012) ars informatica ars musica ‘Aesthetic Perspective Space’ (Vickers & Hogg 2006) abstract Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg research-led practice (Dean & Smith) interactive parameter mapping perceptual principles, i.e. cross-modal association (Lindborg & Friberg 2015, Lindborg 2016) 2003 2005 2008 2013 2014 2015 2016 2017 Pacific Belltower Man bör kalla Swordles TreeTorika LWiOS MessagierEsquisse LW24 increased spatial Locust Wrath #2 resolution Locust Wrath 6 Wednesday, October 18, 2017
Wednesday, October 18, 2017 https://i.ytimg.com/vi/bKr5nbxVzSg/hqdefault.jpg 7 https://www.google.com.sg/search?q=earthquake&client=firefox-b- ab&source=lnms&tbm=isch&sa=X&ved=0ahUKEwjH47j8y6XQAhUMK o8KHWq1AAwQ_AUICCgB&biw=1452&bih=1101&dpr=0.9 https://en.wikipedia.org/wiki/2017_Chiapas_earthquake PerMagnus Lindborg Pacific Belltower
Pacific Belltower PerMagnus Lindborg Wat Rakhang, Bangkok Perchance, he for whom this bell tolls may be so ill, as that he knows not it tolls for him… No man is an island… Never send to know for whom the bell tolls; it tolls for thee. John Donne (1623) Children's Bell Tower, Bodega Bay, USA photo-s/07/b8/85/a8/children-s-bell-tower.jpg https://media-cdn.tripadvisor.com/media/ https://en.wikipedia.org/wiki/Bell_tower_(wat) 8 Wednesday, October 18, 2017
Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg http://www.iris.washington.edu/seismon/eventlist/index.phtml Internet data parsing parameter mapping visualisation sonification spatialisation 10 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg http://www.iris.washington.edu/ seismon/eventlist/index.phtml data parsing parameter mapping visualisation sonification spatialisation 11 Wednesday, October 18, 2017
Wednesday, October 18, 2017
time management ~ trigger a peal on the hour + quarters of hour Wednesday, October 18, 2017
retrieving data from a website using Jitter, formatting it for FTM (matrices) Wednesday, October 18, 2017
parsing text data scan line by line (consider regexp) Wednesday, October 18, 2017
mapping epicentre depth to pitch (f0) of FM synthesis bells Wednesday, October 18, 2017
mapping magnitude to FM harmonicity Wednesday, October 18, 2017
mapping geographic DIRECTION to spatialisation Wednesday, October 18, 2017
Wednesday, October 18, 2017
Wednesday, October 18, 2017
Wednesday, October 18, 2017
22 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Mapping strategies (Spence, “Crossmodal…tutorial” 2011) (Tuuri & Eerola, “Taxonomy… Listening Modes” 2012) (Huron, “Sweet anticipation” 2010) Lindborg PM (2016). “Interactive Sonification of Weather Data for The Locust Wrath” . Leonardo , MIT Press. 23 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Mapping strategies Auditory system: ecological approach (1 - reflexive ) “Is there an immediate danger!?” • innate responses • structural crossmodal associations peal of bells, on the hour urgency 24 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Mapping strategies Auditory system: ecological approach (2 - kinaestethic ) affordances of perceptual experiences: • “How does the sound physically manifest itself?” • enactive perception ( doing , cf. Noë 2004) • categorisation by sound source, early processing (Lindborg 2016 “Taxonomy…”) sound source recognised as bells, connotations 25 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Mapping strategies Auditory system: ecological approach (2 - kinaestethic ) “Where does the sound come from? Is it approaching or receding?” • processing of spatial cues is largely pre-attentive • gestural signatures (friend or foe) • kinaesthetic action-sound couplings (mostly acquired) spatialisation congruence w. reality geography 26 Lindborg & Liu “ LWiOS” Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Mapping strategies Auditory system: ecological approach (3 - connotative ) Kinaesthetic affordances of perceptual experiences • “What does the sound evoke in me? Is it aggressive or inviting?” • contextual orientations and anticipations • listening mode depends on emotional crossmodal associations cognitive appraisal of response alternatives (cf. “decisional consequences”) inharmonious harmonicity (FM synth) magnitude = more energetic shallow = more pitch epicentre depth dangerous = high pitch 27 Wednesday, October 18, 2017
28 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg listeners “might be able to recognise structural elements in music that represent the conventions or rules of the cultural field… [the] meanings of semantic listening [are] affordances of cultural behaviour” (Tuuri & Eerola 2012). border between aesthetic sonification and electroacoustic music is blurred: “the difference… is largely one of perspective.” (Vickers & Hogg 2006) sonification should embrace soundscape research (Gresham-Lancaster & Sinclair 2005) strong principles for aesthetic sonification can be found in experiences of natural, non-designed environments 29 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Aalto University Volume 2 Symposium, 2016 Nov. Yale-NUS , 2016 30 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Tests with parametric speakers - Acouspade ADM/NTU , 2017 31 Wednesday, October 18, 2017
Pacific Belltower PerMagnus Lindborg Thanks for setup help! Deoshor & Kimmie 32 Wednesday, October 18, 2017
Thanks for listening! Pacific Belltower … questions? sound installation for live sonification of earthquake Internet data PerMagnus Lindborg , PhD International Computer Music Conference ICMC–EMW, Shanghai, 18 October 2017 www.permagnus.org permagnus@ntu.edu.sg 33 Wednesday, October 18, 2017
Recommend
More recommend
