The Future of Prosody It’s about Time Dafydd Gibbon Bielefeld University Jinan University Speech Prosody 9, Poznań, 13 June 2018
TIME The Future of Prosody – relevant topics? Ethics of research ● responsibility for the use of our results: – big data analytics, deep learning, natural speech – mass collection of personal communication habits, surveillance Time ● new methods → new insights ● brainstorming – ‘thinking outside the box’ ● methodology in subfields of prosody – more structure / pattern oriented, less function oriented SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 2
TIME Time as a core concept Metatheoretical framework for Time functions, trajectories time discussion Discussion of YARD (Yet Another Rhythm Discussion) different • annotation mining paradigms • multiple oscillators • low frequency spectra: • amplitude modulation • amplitude demodulation Challenges for the future Evolution SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 3
Major Time Process Time Time Types Time Patterns Domains Strawson’s acoustic Gedankenexperiment : ● Can one conceive of an individual, i.e. an object or a person, in a spaceless world with a time dimension only? Conclusion: ● if changing sounds are assumed to have moving sources ● then the sources are interpretable as individual entities Following this basic ontology: ● segments of speech signals are dynamically changing events ● not static units: speech has changing frequencies and rhythms (though in practice it is often convenient to forget this) Strawson, Peter F. Individuals. An Essay in Descriptive Metaphysics . London: Methuen. 1959. SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 4
In memoriam This work is dedicated to the memory of Wiktor Jassem , emeritus of the Polish Academy of Science, distinguished phonetician and prosodist, pioneer in spectral analysis and speech synthesis, and authority on Polish phonetics and phonology, mentor and long-time friend, with whom I discussed the seeds of many ideas reflected in this presentation over some 30 years, in particular the use of the difference spectra discussed in the present talk. I would especially like to remember Grzegorz Dogil , formerly of Lublin, Poznań, Bielefeld and Stuttgart, whom many of you have known personally. Greg passed away much too soon six months ago. His highly productive early work in my department in Bielefeld, including points related to the content of this address, was a source of inspiration for us, and has inspired many more phoneticians since that time. SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 5
Acknowledgments to the strong tradition of logical and computational phonetic and phonological research in Poland, particularly in Poznań: 1. Wiktor Jassem , on acoustic edge detection and on hierarchies of intonation and rhythm In many publications 2. Tadeusz Batóg , on hierarchical mereological models in phonology Batóg, Tadeusz. The Axiomatic Method in Phonology . London: Routledge and Kegan Paul, 1967. 3. Maria Steffen-Batóg finite state computational phonology Steffen-Batóg, Maria. The problem of automatic phonemic transcription of written Polish. Biuletyn Fonograficzny . 14, pp. 75–86, 1973. And to Batóg & Steffen-Batóg on formalizing phonetic distance Steffen-Batóg, Maria and Tadeusz Batóg. A distance function in phonetics. Lingua Posnaniensis , XXIII, 47–58. 1980. Special thanks for valuable hints, comments, suggestions and data: Petra Wagner, Plinio Barbosa, Rosemarie Tracy, Alexandra Gibbon Yu Jue, Liang Jie, Liu Huangmei, Chen Wenjun (Shanghai) Lin Xuewei, Li Peng, He Linfang, Feng Baoyin, Bi Dan (Guangzhou) SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 6
TIME Thinking outside the box: – different methods and method combinations – cooperation with other disciplines ● computational linguistics ● biology, ethology, musicology – computational perspectives ● user ● script / software developer – (( exploration * confirmation *)* standardization *)* cycles – exploratory rather than confirmatory research ● sometimes: lots of statistics and few examples ● here: lots of examples and not so much statistics ● discovery by thinking outside the box, visualisation, analogy, ... SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 7
Alternative Discourse Prosody SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 8
Time: Types, Domains, Processes, Patterns, Stamps, Modulations The background: Time Major Processing Patterns: ● epochs of time from discourse to evolution Time Types Time Time static & ● domains of time from syllable to discourse Domains dynamic ● paradigmatic and syntagmatic patterns in time Heuristic methods of prosodic pattern analysis: Time Time Time Time Stamps: Stamps: Stamps: Stamps: ● effects on subphones, subphone elements in sequence 3D 2D 1D Annotation ● duration patterns of vocalic, syllabic, stress units Time Trees Relations Isochrony Mining ● pitch patterns at phone, syllable, word, … discourse rank Explanatory methods of prosodic pattern analysis: Rhythm AM: AM: is Multiple Spectral FM: AM & FM Oscillators, Zones, Discourse ● rhythm as similarity, isochrony and alternation Spectral Production Perception Modulation ● rhythm as oscillation and modulation in speech production Zones emulation emulation ● rhythm as demodulation of oscillation in speech perception SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 9
Time: Types, Domains, Processes, Patterns, Stamps, Modulations Time Major Processing Patterns: Time Types Time Time static & Domains dynamic Heuristic methods of prosodic pattern analysis: Time Time Time Time Stamps: Stamps: Stamps: Stamps: ● effects on subphones, subphone elements in sequence 3D 2D 1D Annotation ● duration patterns of vocalic, syllabic, stress units Time Trees Relations Isochrony Mining ● pitch patterns at phone, syllable, word, … discourse rank Explanatory methods of prosodic pattern analysis: Rhythm AM: AM: is Multiple Spectral FM: AM & FM Oscillators, Zones, Discourse ● rhythm as similarity, isochrony and alternation Spectral Production Perception Modulation ● rhythm as oscillation and modulation in speech production Zones emulation emulation ● rhythm as demodulation of oscillation in speech perception SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 10
Time: Types, Domains, Processes, Patterns, Stamps, Modulations Time Major Processing Patterns: Time Types Time Time static & Domains dynamic Time Time Time Time Stamps: Stamps: Stamps: Stamps: 3D 2D 1D Annotation Time Trees Relations Isochrony Mining Rhythm AM: AM: Explanatory methods of prosodic pattern analysis: is Multiple Spectral FM: ● rhythm as similarity, isochrony and alternation AM & FM Oscillators, Zones, Discourse ● rhythm as oscillation and modulation in speech production Spectral Production Perception Modulation Zones emulation emulation ● rhythm as demodulation of oscillation in speech perception SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 11
Time: Types, Domains, Processes, Patterns, Stamps, Modulations Time Major Processing Patterns: Time Types Time Time static & Domains dynamic Time Time Time Time Stamps: Stamps: Stamps: Stamps: 3D 2D 1D Annotation Time Trees Relations Isochrony Mining Rhythm AM: AM: is Multiple Spectral FM: AM & FM Oscillators, Zones, Discourse Spectral Production Perception Modulation Zones emulation emulation SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 12
Time Major Processing Patterns: Time Types Time Time static & Domains dynamic Time and Prosody: overview Events vs. objects Five Major Time Epochs Four Time Types: ● Categorial time: paradigmatic ● Rubber time: syntagmatic ● Clock time: time stamps ● Cloud time: real time Two kinds of processing time: ● recursion, time and space: ● finite vs. non-finite memory (space) ● linear vs. non-linear complexity (time) SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 13
Major Time Process Time Time Types Time Patterns Domains Categorial Time Various phonologies, (paradigmatic relations) but most explicitly ‘Rubber’ Time Event Phonology (syntagmatic relations) Thanks to Andras Kornai , for the concept ‘Rubber Time’ SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 14
Major Time Process Time Time Types Time Patterns Domains Categorial Time Various phonologies, (paradigmatic relations) but most explicitly ‘Rubber’ Time Event Phonology (syntagmatic relations) Speech Technology Clock Time ‘front ends’: ● Recognition ● Synthesis ● Identification Cloud Time Thanks to Andras Kornai , for the concepts ‘Rubber Time’ and ‘Clock Time’ SP9, Poznań, 13 June 2018 D. Gibbon, The Future of Prosody - It's about Time 15
Recommend
More recommend
