Speech Synthesis and Perception with Envelope Cue B ACKGROUND I - PowerPoint PPT Presentation

Signals and Systems Speech Synthesis and Perception with Envelope Cue

目录 B ACKGROUND I MPLEMENTATION R ESULTS D ISCUSSION I MPROVEMENT

B ACKGROUND | P ART 1

History - Artificial Cochlea • First extra-auricular electric simulation 1748 • Invention of an electrical stimulating system 1905 • Electrode placed in the acoustic nerve produced a copy of the speech waveform. 1930

• The first true cochlea implant was implanted by the American otologist William Bill House 1961 • FDA allowed them to be implanted in adults. 1984 • The implants are approved for infants over 12 months old. 2000

I MPLEMENTATION | P ART 2

Figure 1. The operation of a four-channel cochlear implant . Reprinted from "Introduction to cochlear implants," by P . C. Loizou, 1999, IEEE Engineering in Medicine and Biology Magazine, vol. 18, no. 1.

synthesize.m-modulation band = 8; order = 4

synthesize.m-8 band pass filters order = 4

add_ssn.m SNR=-5

GUI.m

R ESULTS D ISCUSSION | P ART 3

Task1 Variation in Channel Number N=1 • Butter Filters: Order = 4 N=2 • 𝑔 𝑑𝑣𝑢𝑝𝑔𝑔 = 50𝐼𝑨 N=4 • N=8 • N=16 • N=20 • N=32 •

Why N is limited? • Instability of filters • Interference between electrodes • Continuous interleaved sampling

Task2 Variation in Cut-off Frequency Implement tone- Describe how the LPF Set the number vocoder by cut-off frequency affects changing the LPF of bands N=4. the intelligibility of synthesized sentence. cut-off frequency .

Task2 Results and Conclusion N=4 • 𝑔 𝑑𝑣𝑢𝑝𝑔𝑔 = 20Hz • 𝑔 𝑑𝑣𝑢𝑝𝑔𝑔 = 50Hz • 𝑔 𝑑𝑣𝑢𝑝𝑔𝑔 = 100Hz • 𝑔 𝑑𝑣𝑢𝑝𝑔𝑔 = 400Hz

Task3 Noise & Variation in Band Number Describe how the Implement Generate a number of bands affects Set LPF tone-vocoder the intelligibility of noisy signal cut-off by changing synthesized sentence, at SNR frequency and compare findings the number of -5 dB with those obtained to 50 Hz bands in task 1

Task3 Results and Conclusion N=2 • N=4 • N=6 • N=8 • N=16 •

Task4 Noise & Variation in Cut-off Frequency Describe how the Implement LPF cut-off Generate a Set the tone-vocoder frequency affects noisy signal number by changing the intelligibility at of bands the LPF cut-off of synthesized SNR -5 dB to N=6 frequency sentence

Task4 Noise & Variation in Cut-off Frequency

English & Chinese Comparison • Synthesized speech is likely to lose its tone. • Chinese: tonal; English: non-tonal Processed ：

English & Chinese Comparison Reprinted from " 电子耳蜗言语处理策略的频谱特征研究 ." by 陈又圣 , et al. (2017) 生物医学工程学杂志 34(5): 760-766.

How about music?

I MPROVEMENT | P ART 4

Noise Reduction S. V. Vaseghi, Advanced Digital Signal Processing and Noise Reduction . 2008.

Noise Reduction using Wiener filters Original • Noisy • Noise Reduced • Synthesized (noisy) • Synthesized (noise reduced) •

Reference ： [1] A. Mudry and M. Mills, "The early history of the cochlear implant: a retrospective," (in eng), JAMA Otolaryngol Head Neck Surg, vol. 139, no. 5, pp. 446-53, May 2013. [2] R. V. Shannon, F. G. Zeng, V. Kamath, J. Wygonski, and M. Ekelid, "Speech recognition with primarily temporal cues," (in eng), Science, vol. 270, no. 5234, pp. 303-4, Oct 13 1995. [3] S. V. Vaseghi, Advanced Digital Signal Processing and Noise Reduction. Wiley, 2008. [4] Chen, F., et al. (2015). "Evaluation of noise reduction methods for sentence recognition by mandarin- speaking cochlear implant listeners." Ear and hearing 36(1): 61-71. 陈又圣 , et al. (2017). " 电子耳蜗言语处理策略的频谱特征研究 ." 生物医学工程学杂志 34(5): 760-766. [5] 龚树生 , and 郝瑾 , “国产人工耳蜗 , 任重道远 , ” 中国医学文摘 : 耳鼻咽喉科学 , vol. 28, no. 5, pp. 231-236, [6] 2013.

感谢观看 | THANK YOU