Vocal Biomarkers for Monitoring Neurological Disorders Thomas F. Quatieri Senior Technical Staff, MIT Lincoln Laboratory Faculty Harvard-MIT Health Science Technology May 7, 2015
MIT Lincoln Laboratory Federally Funded Research and Development Center Massachusetts Institute of Technology MIT Lincoln Laboratory, Lexington, Massachusetts Structure: Ten Divisions (e.g., Homeland Protection, Communication Systems, Cyber Security) with about eight groups within each division Bioengineering Systems and Technology Group: Preserve and enhance human health and performance through monitoring, analysis, and interventions • New group ~3 years: Highly interdisciplinary • Staff: ~50 scientists, engineers, students, support • Funding sources: DoD, NIH, Internal • Broad technical areas: Biomedical research, synthetic biology, bioinformatics, biometrics Speech, hearing, and neuro-cognitive analysis JAC Review- 2 TFQ 02/27/15
Speech, Hearing and Neuro-cognitive Analysis Motivation, Objective, Approach Motivation � Traumatic Brain Injury Cognitive • Many conditions that effect Overload Depression cognitive performance � Fragmented Sleep Lou Gehrig’s • Includes neurological and stress Disease conditions � Psychological Parkinson’s Fear Danger Objective Post Traumatic Environmental Stress Disorder • Simple, sensitive method to detect and Hot, Cold, Alzheimer’s Altitude monitor a condition Disease • Distinguish across conditions � Approach: Vocal biomarkers � • Reflect underlying neurophysiological changes that alter speech motor control � • Reflect coordination changes Gait across speech production components, as well with other modalities � JAC Review- 3 TFQ 02/27/15
MIT Lincoln Research Focus Research in vocal biomarkers Depression AVEC 2013 Data (2013 AVEC Depression 11 Challenge): B Phonetic A 10 - • Audio from 50 train/50 test timing RMSE Articulatory subjects 9 - coordination MIT LL Objective: 8 • Predict BECK depression 1 2 3 Team Rank assessment score from audio Evolving research areas Mild Traumatic Brain Injury Data: Full Season Athlete Collection (Purdue) Reaction Time 1.0 Cognitive ALS Timing 0.8 Subjects Ages Sport True Positive Load 0.6 8 female 14–18 Coordination Vocal 0.4 Parkinson’s 24 male 14–18 Fatigue 0.2 Objective: Detect cognitive 0 0.1 0.2 0.3 0.4 0.5 False Alarm impairment (using IMPACT) Dementia Prediction of cognitive status in elderly From laboratory to mobile device Imbalanced Data ROC for Averaged Dataset Data: audio from 200 elderly 1 0.8 True Positive Rate Depression: Traumatic brain Effectiveness of drug 0.6 MIT LL/MIT BCS Equal Error Rate Launching (EER) = 18% injury: treatment: 0.4 worldwide Apps Piloting Apps for Interest in Apps for 0.2 Objective: (with Satra NCAA depression monitoring 0 0 0.2 0.4 0.6 0.8 1 • Detect cognitive impairment Ghosh, MIT BCS) False Positive Rate JAC Review- 4 TFQ 02/27/15
Making an Impact Research Areas Data Collections Modeling Advanced Feature Extraction Clinical Acceptance JAC Review- 5 TFQ 02/27/15
Making an Impact Research Areas Scientific Explain old and guide Data Foundation new vocal biomarkers Collections Modeling Advanced Feature Extraction Clinical Acceptance JAC Review- 6 TFQ 02/27/15
Making an Impact Research Areas Scientific Explain old and guide Data Foundation new vocal biomarkers Collections Modeling Advanced Feature Extraction Clinical Acceptance JAC Review- 7 TFQ 02/27/15
Research Areas Databases Large-scale behavioral collections • Audio databases with on-body platforms – Option to extract vocal features and remove audio • Collections with related modalities (e.g., robust wireless EEG) Imaging collections Gait • Brain, vocal tract, and vocal fold imaging – Improved real-time MRI; ultra high-speed 3D video • Ultimate is simultaneous measurements during speaking Improved protocols • Speaking tasks that illicit specific parts of the brain and speech motor processes • Speaking tasks that bring out specific neural and motor components effected by different neurological conditions JAC Review- 8 TFQ 02/27/15
Research Areas Modeling Simple approximate view of Need for model-based approaches to enhance speech production scientific foundation for use of vocal biomarkers Concept Different brain regions Sentences and words Auditory and tactile self-monitoring Computational neural modeling Cognitive • Basic neural circuitry of speech production Syllables and phonemes Prosodics • Modulation by non-speech networks (e.g., limbic) Limbic Phonetic representation: Position/state of articulators/ • Disturbances in the distressed brain folds • Directions into Velocities of Articulators (DIVA) Timing and coordination of articulators and vocal folds model is one basis Neural signaling Muscle activation Speech Computational physiological modeling • Understanding of multitude of muscles and their coordination in speech production • Disorders both in articulatory and laryngeal (vocal fold) movement JAC Review- 9 TFQ 02/27/15
Research Areas Advanced Feature Extraction • Mapping of changes in neural and physiological models to changes in the acoustic signal • Robust and high-resolution signal processing to reflect dynamic and subtle aspects of complex changes in neural and physiological systems, beyond standard features JAC Review- 10 TFQ 02/27/15
Clinical Acceptance • Objective measures as an aid, not replacement • Early identification of neurologic disease onset • Prediction of relapse or recovery • Prediction should be specific as well as sensitive – Many sub-classes of speech disorders common to a variety of neurological disorders • Monitoring should be personalized with biofeedback JAC Review- 11 TFQ 02/27/15
Acknowledgments Daryush Mehta and Bob Hillman – MGH Voice Rehabilitation Center Jordan Green – MGH Speech and Feeding Disorders Lab Satra Ghosh – MIT Brain and Cognitive Science Dept. Visar Berisha – Arizona State, Speech and Hearing Science JAC Review- 12 TFQ 02/27/15
Publications Trevino, A., Quatieri, T. F. and Malyska, N., “Phonologically-based biomarkers for major depressive disorder,” EURASIP Journal on Advances in Signal Processing: Special Issue on Emotion and Mental State Recognition from Speech, 42:2011–2042, 2011. Williamson, J.R., Quatieri, T.F., Helfer, B.S., Horwitz, R., Yu, B., and Mehta, D.D., “Vocal biomarkers of depression based on motor incoordination,” in Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge, 2013, pp. 41–48. Helfer, B.S., T. F. Quatieri, Williamson, J.R., Mehta, D.D., Horwitz, R ., and B. Yu, “Classification of depression state based on articulatory precision.,” in Interspeech, 2013, pp. 2172–2176. Quatieri, T. F. and Malyska, N., “Vocal-source biomarkers for depression: A link to psychomotor activity,” Proceedings of Interspeech, 2012. Horwitz, R., Quatieri, T.F., Helfer, B.S., Yu, B., Williamson, J.R., and Mundt, J., “On the Relative Importance of Vocal Source, System, and Prosody in Human Depression.” IEEE Body Sensor Network Conference, Cambridge, MA, May 2013. Helfer, B.S., Quatieri, T.F., Williamson, J.R., Keyes, L., Evans, B., Greene, W.N., Vian, T., Lacirignola, J., Shenk, T., Talavage, T., Palmer, J., and Heaton, K., "Articulatory dynamics and coordination in classifying cognitive change in preclinical mTBI," Interspeech 2014. Yu, B., Quatieri, T.F., Williamson, J.R., and Mundt, J., "Prediction of cognitive performance in an animal fluency task based on rate and articulatory markers," Interspeech 2014. L. Keyes, J. Su, T. Quatieri, B. Evans, J. Lacirignola, T. Vian, W. Greene, D. Strom, and A. Dai, “FY12 Line-Supported Bio-Medical Initiative Program: Multi-modal Early Detection Interactive Classifier (MEDIC) for Mild Traumatic Brain Injury (mTBI) Triage,” MIT Lincoln Laboratory Project Report LSP-41, 2012. JAC Review- 13 TFQ 02/27/15
Recommend
More recommend
