1 19.05.2009 Emotional Speech Synthesis State of the art 2009 Felix Burkhardt
outline � how to model and why simulate emotions? � emotions in speech � introduction to speech synthesis approaches � examples, examples, examples � conclusion and outlook 19.05.2009 2 Emotional Soeech Synthesis - Felix Burkhardt,
contents � how to model and why simulate emotions? � emotions in speech � overview on speech synthesis � examples, examples, examples � conclusion, outlook 19.05.2009 3 Emotional Soeech Synthesis - Felix Burkhardt,
emotion models anger joy � … everyone except a psychologist knows what an emotion is (Young 1973) despair categories, e.g. anger, joy, … � dimensions, e.g. activation, � neutral dominance, valence arousal appraisals, e.g. novelty, intrinsic � pleasantness, relevance, coping e content c potential, n a n boredom i m o d emotion cube sadness valence source: Burkhardt 2001 19.05.2009 4 Emotional Soeech Synthesis - Felix Burkhardt,
w hy model emotional behaviour? � aspects of emotion modeling in human-machine interaction: source: Batliner et al 2006 19.05.2009 5 Emotional Soeech Synthesis - Felix Burkhardt,
applications of emotional tts � fun, e.g. emotional greetings � prosthesis � emotional chat avatars � gaming, believable characters time � adapted dialog design � adapted persona design � target-group specific advertising � … � believable agents � … � artificial humans 19.05.2009 6 Emotional Soeech Synthesis - Felix Burkhardt,
aspects of emotional tts 19.05.2009 7 Emotional Soeech Synthesis - Felix Burkhardt,
contents � why simulate emotions? � emotions in speech � overview on speech synthesis � examples, examples, examples � conclusion, outlook 19.05.2009 8 Emotional Soeech Synthesis - Felix Burkhardt,
speech features descriptive layers of speech source: Reynolds et al 2003 19.05.2009 9 Emotional Soeech Synthesis - Felix Burkhardt,
emotion in speech neutral angry happy bored frightened sad spectrograms from emotional acted speech source: TUB emotional database 19.05.2009 10 Emotional Soeech Synthesis - Felix Burkhardt,
emotional data? � actors vs. reality � Berlin EmoDB: 10 actors x 7 emotions x 10 sentences � alternatives � induced data, e.g. Aibo � television, radio data EmoDB: Burkhardt et al 2005 19.05.2009 11 Emotional Soeech Synthesis - Felix Burkhardt,
how to describe emotion? � EmotionML, incubator group at W3C � Example, embedded in SSML: <speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US"> <voice gender="female"> <prosody contour="(0%,+20Hz)(10%,+30%)(40%,+10Hz)"> Hi, am sad know but start getting angry... </prosody> </voice> <emotion> <category name="sadness„ set="basic" intensity="0.6"/> <timing start="10%" end="50%"/> </emotion> <emotion> <category name="anger" set="basic" intensity="0.4"/> <timing start="50%" end="100%"/> </emotion> </speak> http://www.w3.org/2005/Incubator/emotion/ 19.05.2009 12 Emotional Soeech Synthesis - Felix Burkhardt,
loquendo tts director source: Loquendo 19.05.2009 13 Emotional Soeech Synthesis - Felix Burkhardt,
contents � why simulate emotions? � emotions in speech � introduction to speech synthesis approaches � examples, examples, examples � conclusion, outlook 19.05.2009 14 Emotional Soeech Synthesis - Felix Burkhardt,
speech synthesis taxonomy speech synthesis systems re (copy)-synthesis, voice transformation voice response systems arbitary speech synthesizers voice conversion text-to-speech concept-to-speech (unknown input) (input from text-generation system) 19.05.2009 15 Emotional Soeech Synthesis - Felix Burkhardt,
tts process chain NLP natural DSP digital language speech phonetic transcription processing processing prosody track preprocessing unit concatenation / search morpho-syntactic analysis prosody fitting transpcription edge smoothing prosody modeling 19.05.2009 16 Emotional Soeech Synthesis - Felix Burkhardt,
synthesis approaches signal modeling system modeling articulatory synthesis vocal tract shape synthesis pseudo articulatory rule based data based expert systems statistical model generated non-uniform unit selection concatenative synthesis formant synthesis HMM hidden markov models ANN neural nets coding of units type of units syllables, diphones, parametric coded waveform coded allophones, LPC linear predictive coding PCM subsegments MFCC mel frequency cepstral LDM (linear delta mod.) MBR multi band resynthesis formants hybrid approaches MBRPSOLA, RELP 19.05.2009 17 Emotional Soeech Synthesis - Felix Burkhardt,
historic development natural sounding domain dependent non-uniform unit selection e.g. RealSpeak PSOLA based synthesis e.g. Elan formant synthesis e.g. Dec Talk articulatory van Kempelen flexible not flexible 1780 …. 1980 1990 2000 historic modern artificial sounding domain independent 19.05.2009 18 Emotional Soeech Synthesis - Felix Burkhardt,
system modeling 19.05.2009 19 Emotional Soeech Synthesis - Felix Burkhardt,
source filter model source: Klatt80 formant synthesizer (Klatt 1980) 19.05.2009 20 Emotional Soeech Synthesis - Felix Burkhardt,
contents � why simulate emotions? � emotions in speech � overview on speech synthesis � examples, examples, examples � conclusion, outlook 19.05.2009 21 Emotional Soeech Synthesis - Felix Burkhardt,
examples: emofilt � open source Java program based on MBROLA synthesis engine. � NOT a complete text-to-speech system � prosody filter between natural language and digital speech signal processing modules � as multilingual as MBROLA which currently supports 35 languages. 19.05.2009 22 Emotional Soeech Synthesis - Felix Burkhardt,
examples: emoSpeak � emoSpeak is integrated into the MARY text-to- speech framework by DFKI. � Marc Schröder investigated in his ph.d. thesis, how to assign rule-based modification of speech to emotional dimensions. � the system can be freely dowloaded source: Schröder 2004 19.05.2009 23 Emotional Soeech Synthesis - Felix Burkhardt,
examples voice conversion Murtaza Bulut et al, PSOLA - LPC neutral angry USC conversion Greg Beller, IRCAM Phase vocoder neutral sad 19.05.2009 24 Emotional Soeech Synthesis - Felix Burkhardt,
examples voice transformation Olivier Rosec Mixed LF + harmonic woman model FranceTelecom 2009 as boy as man man breathy whispery tense Shiva Sundaram Laughter synthesis by LPC synthesis and USC 2007 mass-spring model 19.05.2009 25 Emotional Soeech Synthesis - Felix Burkhardt,
examples formant synthesis AffectEditor DEC Talk prosody sad angry rules J. Cahn, MIT 1998 EmoSyn prosody rules + neutral sad phonation model Burkhardt, 2000 angry crying content 19.05.2009 26 Emotional Soeech Synthesis - Felix Burkhardt,
examples diphone synthesis MARY prosody rules for joy angry dimensions M. Schröder, DFKI three inventories for soft, normal and tense speech EmoFilt prosody rules neutral joy Burkhardt, 1999 19.05.2009 27 Emotional Soeech Synthesis - Felix Burkhardt,
examples statistical based Tokyo Institute, HMM models spectral neutral joy Kobayashi Lab and prosodic features 19.05.2009 28 Emotional Soeech Synthesis - Felix Burkhardt,
examples unit selection fun personality voices Damian Shouty CTTS with expressive product research units extralinguistic units Katrin 19.05.2009 29 Emotional Soeech Synthesis - Felix Burkhardt,
examples non human Oudeyer: Sony pet concatenative happy sad robots MIT Kismet robot formant synthesis anger fear 19.05.2009 30 Emotional Soeech Synthesis - Felix Burkhardt,
examples singing vocal tract lab 2007 donna nobis Peter Birkholz articulatory pavarobotti 1993 aria Ingo Titze Articulatory Bell Labs Gerstman & 1961 articulatory, first bicycle Mathews, song ever 19.05.2009 31 Emotional Soeech Synthesis - Felix Burkhardt,
more examples … http://emosamples.syntheticspeech.de 19.05.2009 32 Emotional Soeech Synthesis - Felix Burkhardt,
contents � why simulate emotions? � emotions in speech � overview on speech synthesis � examples, examples, examples � conclusion, outlook 19.05.2009 33 Emotional Soeech Synthesis - Felix Burkhardt,
conclusion � emotions are part of natural speech � simulation possible by either � modeling the process � including emotional data � still text to speech fights with intelligible, neutral speech � first steps: speaking styles, extralinguistics � first apps: fun, gaming 19.05.2009 34 Emotional Soeech Synthesis - Felix Burkhardt,
Recommend
More recommend
