Dual-Channel Acoustic Detection of Nasalization Statuses Dual-Channel Acoustic Detection of X. Niu & J. van Santen Nasalization Statuses Introduction Method Dual-channel acoustic model Xiaochuan Niu Dual-channel analysis method Adviser: Jan P . H. van Santen Nasalization feature extraction Nasalization detector Experiments Center for Spoken Language Understanding Simulation OGI School of Science & Engineering at OHSU Speech materials Detection tasks Results November 27, 2007 in SRI Conclusion
Outline Dual-Channel Acoustic Introduction 1 Detection of Nasalization Statuses Method 2 X. Niu & J. Dual-channel acoustic model van Santen Dual-channel analysis method Introduction Nasalization feature extraction Method Dual-channel Nasalization detector acoustic model Dual-channel analysis method Nasalization feature Experiments 3 extraction Nasalization detector Simulation Experiments Speech materials Simulation Speech materials Detection tasks Detection tasks Results Results Conclusion Conclusion 4
Velopharyngeal control during speech Dual-Channel Acoustic Detection of Nasalization Appropriate control of Statuses the VP port X. Niu & J. Nasal tract van Santen Closure: fricatives, plosives, non-nasal Introduction vowels Method Velum Dual-channel Opening: nasals, acoustic model Dual-channel nasal vowels, analysis method Nasalization feature nasalized vowels Oral tract extraction Velopharyngeal Nasalization detector Lack of coordination port Experiments Resonance: hypo- or Simulation Pharyngeal Speech materials tract hyper-nasality Detection tasks Glottal folds Results Airflow: nasal Conclusion emission
Statuses of nasal resonance Dual-Channel Acoustic Detection of Nasalization Different oral-nasal articulatory configurations that can Statuses be identified perceptually from acoustic signals X. Niu & J. van Santen Vo Oral opening only (e.g. non-nasal vowels) Introduction Ns Nasal opening only (e.g. nasals) Method Nv Oral & nasal opening simultaneously (e.g. nasalized Dual-channel vowels) acoustic model Dual-channel analysis method Research motivation: non-invasive detection of Nasalization feature extraction nasalization statuses for Nasalization detector Understanding the VP control mechanism during Experiments Simulation normal nasalization Speech materials Detection tasks Analysis and enhancement of disordered speech with Results resonance problems Conclusion
Statuses of nasal resonance Dual-Channel Acoustic Detection of Nasalization Different oral-nasal articulatory configurations that can Statuses be identified perceptually from acoustic signals X. Niu & J. van Santen Vo Oral opening only (e.g. non-nasal vowels) Introduction Ns Nasal opening only (e.g. nasals) Method Nv Oral & nasal opening simultaneously (e.g. nasalized Dual-channel vowels) acoustic model Dual-channel analysis method Research motivation: non-invasive detection of Nasalization feature extraction nasalization statuses for Nasalization detector Understanding the VP control mechanism during Experiments Simulation normal nasalization Speech materials Detection tasks Analysis and enhancement of disordered speech with Results resonance problems Conclusion
Acoustic features of nasalization (review) Dual-Channel Acoustic Detection of Nasalization Single-channel spectral characteristics of nasalized Statuses vowels X. Niu & J. van Santen Qualitative observations Reduced amplitude and/or upward-shift of F 1 Introduction Pole-zero pair in F 1 region Method Pole-zero pair in 200-500 Hz, etc. Dual-channel acoustic model Quantitative features Dual-channel analysis method Nasalization feature Parameters of spectral “flatness” extraction General spectral envelop features (MFCC, etc.) Nasalization detector Experiments Dual-channel acoustic measurement Simulation Speech materials Energy balance ( nasalance ): E n / ( E n + E m ) Detection tasks Results Oral-nasal transfer ratio function Conclusion (ONTRIF) analysis (Niu et al. 2005)
Outline Dual-Channel Acoustic Introduction 1 Detection of Nasalization Statuses Method 2 X. Niu & J. Dual-channel acoustic model van Santen Dual-channel analysis method Introduction Nasalization feature extraction Method Dual-channel Nasalization detector acoustic model Dual-channel analysis method Nasalization feature Experiments 3 extraction Nasalization detector Simulation Experiments Speech materials Simulation Speech materials Detection tasks Detection tasks Results Results Conclusion Conclusion 4
Dual-channel acoustic model Dual-Channel Acoustic Detection of Transmission-line model for a lossy cylindrical acoustic Nasalization Statuses tube (Flanagan, 1972) X. Niu & J. dx van Santen U+dU U Introduction L/2 R/2 L/2 + R/2 + P P+dP Method L w C P P+dP Dual-channel G U U+dU acoustic model R w Dual-channel − − C analysis method w Nasalization feature extraction Nasalization detector Acoustic waves in a tube modeled as electrical waves in Experiments a transmission line Simulation Speech materials Sound pressure ( P ) vs. Voltage Detection tasks Results Volume velocity ( U ) vs. Current Conclusion Circuit parameters are determined by the physical properties of the tube
Dual-channel acoustic model Dual-Channel Acoustic Detection of » P g – » A p B p – » P v – Nasalization = U pi C p D p U po + Statuses Pnl Uni Uno » – » – » – P v A m B m P ml Tn X. Niu & J. Znl = U mi C m D m U mo − van Santen » P v – » A n B n – » P nl – Usub Upi Upo Pg = U ni C n D n U no + + + Introduction Pv Pml Umi Umo U po U mi + U ni Ug Tp Tm = Method Zsub Zml − − U sub = U g + U pi − Dual-channel acoustic model Dual-channel analysis method A circuit network represents voiced sound production Nasalization feature extraction Nasalization detector through nasal and oral channels (Childers, 2000) Experiments Transmission properties of acoustic waves through Simulation Speech materials vocal tracts are modeled by chain-matrix equations Detection tasks Results Coupling effects result from the constraints applied by Conclusion the boundary equations
Dual-channel acoustic model Dual-Channel Acoustic Detection of » P g – » A p B p – » P v – Nasalization = U pi C p D p U po + Statuses Pnl Uni Uno » – » – » – P v A m B m P ml Tn X. Niu & J. Znl = U mi C m D m U mo − van Santen » P v – » A n B n – » P nl – Usub Upi Upo Pg = U ni C n D n U no + + + Introduction Pv Pml Umi Umo U po U mi + U ni Ug Tp Tm = Method Zsub Zml − − U sub = U g + U pi − Dual-channel acoustic model Dual-channel analysis method A circuit network represents voiced sound production Nasalization feature extraction Nasalization detector through nasal and oral channels (Childers, 2000) Experiments Transmission properties of acoustic waves through Simulation Speech materials vocal tracts are modeled by chain-matrix equations Detection tasks Results Coupling effects result from the constraints applied by Conclusion the boundary equations
Dual-channel acoustic model Dual-Channel Acoustic Detection of » P g – » A p B p – » P v – Nasalization = U pi C p D p U po + Statuses Pnl Uni Uno » – » – » – P v A m B m P ml Tn X. Niu & J. Znl = U mi C m D m U mo − van Santen » P v – » A n B n – » P nl – Usub Upi Upo Pg = U ni C n D n U no + + + Introduction Pv Pml Umi Umo U po U mi + U ni Ug Tp Tm = Method Zsub Zml − − U sub = U g + U pi − Dual-channel acoustic model Dual-channel analysis method A circuit network represents voiced sound production Nasalization feature extraction Nasalization detector through nasal and oral channels (Childers, 2000) Experiments Transmission properties of acoustic waves through Simulation Speech materials vocal tracts are modeled by chain-matrix equations Detection tasks Results Coupling effects result from the constraints applied by Conclusion the boundary equations
Outline Dual-Channel Acoustic Introduction 1 Detection of Nasalization Statuses Method 2 X. Niu & J. Dual-channel acoustic model van Santen Dual-channel analysis method Introduction Nasalization feature extraction Method Dual-channel Nasalization detector acoustic model Dual-channel analysis method Nasalization feature Experiments 3 extraction Nasalization detector Simulation Experiments Speech materials Simulation Speech materials Detection tasks Detection tasks Results Results Conclusion Conclusion 4
Dual-channel analysis method Dual-Channel Acoustic Detection of Nasalization Oral-nasal transfer ratio function (ONTRIF) Statuses X. Niu & J. T n / m ( ω ) ≡ P n ( ω ) P m ( ω ) = ( A m Z ml + B m ) Z nr van Santen ( A n Z nl + B n ) Z mr Introduction Method Dual-channel Properties of the ONTRIF acoustic model Dual-channel analysis method T n / m ( ω ) is independent of the acoustic system below Nasalization feature extraction the VP port; Nasalization detector Poles stem from the transfer admittance of the nasal Experiments Simulation cavity; zeros stem from the transfer admittance of the Speech materials oral cavity; sinuses result in pole-zero pairs; Detection tasks Results T n / m ( ω ) can be estimated from dual-channel signals Conclusion directly.
Recommend
More recommend
