detecting depression with audio text sequence modeling of
play

Detecting Depression with Audio/Text Sequence Modeling of Interviews - PowerPoint PPT Presentation

Aug 17, 2022 •570 likes •1.23k views

Detecting Depression with Audio/Text Sequence Modeling of Interviews Tuka Alhanai, Mohammad Ghassemi, and James Glass Massachusetts Institute of Technology Cambridge MA USA 2 nd September, Interspeech 2018 Email: tuka@mit.edu Website:

  1. Detecting Depression with Audio/Text Sequence Modeling of Interviews Tuka Alhanai, Mohammad Ghassemi, and James Glass Massachusetts Institute of Technology Cambridge MA USA 2 nd September, Interspeech 2018 Email: tuka@mit.edu Website: talhanai.com Github: github.com/talhanai

  2. How have you been feeling lately? Healthy: “I’ve been feeling good lately” Depressed: “*sigh* stressed [um] lately I’ve been really sad and I don’t know why” 2

  3. Hypothesis : Depression can be modeled using sequences of a spoken interaction. 3

  4. Hypothesis : Depression can be modeled using sequences of a spoken interaction. 4

  5. Hypothesis : Depression can be modeled using sequences of a spoken interaction. Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 5

  6. Hypothesis : Depression can be modeled using sequences of a spoken interaction. Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 6

  7. Hypothesis : Depression can be modeled using sequences of a spoken interaction. Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 7

  8. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 8

  9. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 9

  10. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 10

  11. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 11

  12. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 12

  13. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 13 on Audio/Visual Emotion Challenge . ACM, 2016.

  14. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 14 on Audio/Visual Emotion Challenge . ACM, 2016.

  15. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 15 on Audio/Visual Emotion Challenge . ACM, 2016.

  16. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 16 on Audio/Visual Emotion Challenge . ACM, 2016.

  17. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 17 on Audio/Visual Emotion Challenge . ACM, 2016.

  18. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering Yang, Le, et al. "Decision tree based depression classification from audio video and language information." Proceedings of the 6th International Workshop 18 on Audio/Visual Emotion Challenge . ACM, 2016.

  19. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 19

  20. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering keyword: “therapy” “sad” “down” Williamson, James R., et al. "Detecting depression using vocal, facial and semantic communication cues." Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge . ACM, 2016. 20

  21. Current Solutions 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering keyword: “therapy” [Topic 1 features, “sad” Topic 2 features, “down” Topic 3 features] Williamson, James R., et al. "Detecting depression Gong, Yuan, and Christian Poellabauer. "Topic using vocal, facial and semantic communication Modeling Based Multi-modal Depression Detection." cues." Proceedings of the 6th International Workshop Proceedings of the 7th Annual Workshop on on Audio/Visual Emotion Challenge . ACM, 2016. 21 Audio/Visual Emotion Challenge . ACM, 2017.

  22. Our Solution 1. Clinical: Interview/questionnaire 2. Automated : Context-dependent Feature engineering 22

  23. Our Solution 1. Clinical: Interview/questionnaire 2. Automated : Context-independent Feature engineering free 23

  24. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 24

  25. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 25

  26. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 26

  27. Q : What are some things you really like about L.A.? A : I like the weather, I like the opportunities. Q : How easy was it for you to get used to living in L.A.? A : It took a minute, somewhat easy. Q : What are some things you don't really like about L.A.? A : Congestion. Q: What'd you study at school? A: um I took up business and administration. Q: Cool are you still doing that? A: Yeah I am. Here and there, I’m on a break right now but I plan on going back in the uh next semester. Q: What’s your dream job? A: uh probably open up my own business. 27

  28. Method … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 28 …

  29. Method 1/0 depression … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 29 …

  30. Method 1/0 depression bi - LSTM bi - LSTM text … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 30 …

  31. Method 1/0 depression bi - LSTM bi - LSTM bi - LSTM bi - LSTM bi - LSTM text audio … Q: What are some things you really like about L.A.? A: I like the weather, I like the opportunities. Q: How easy was it for you to get used to living in L.A.? A: It took a minute, somewhat easy. Q: What are some things you don't really like about L.A.? A: Congestion. 31 …

Recommend

Modeling language as a sequence of tokens CMSC 470 Marine Carpuat Beyond MT: Encoder-Decoder

Modeling language as a sequence of tokens CMSC 470 Marine Carpuat Beyond MT: Encoder-Decoder

Modeling language as a sequence of tokens CMSC 470 Marine Carpuat Beyond MT: Encoder-Decoder can be used as Conditioned Language Models P(Y|X) to generate text Y based on some input X Given some text, how to segment it into a sequence of

447 views • 43 slides
EE E6820: Speech & Audio Processing & Recognition Lecture 10: ASR: Sequence Recognition

EE E6820: Speech & Audio Processing & Recognition Lecture 10: ASR: Sequence Recognition

EE E6820: Speech & Audio Processing & Recognition Lecture 10: ASR: Sequence Recognition 1 Signal template matching 2 Statistical sequence recognition 3 Acoustic modeling 4 The Hidden Markov Model (HMM) Dan Ellis

615 views • 34 slides
MORE TITLE TEXT . Pregnancy-Related Depression: Health Department Activities Pregnancy-Related

MORE TITLE TEXT . Pregnancy-Related Depression: Health Department Activities Pregnancy-Related

Title Text MORE TITLE TEXT . Pregnancy-Related Depression: Health Department Activities Pregnancy-Related Depression as a Priority Area Title Text MORE TITLE TEXT . 2016-2020 Statewide MCH Needs Assessment 2011-2015 Develop

318 views • 31 slides
Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of

Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of

Garrett Bingham Sequence to Sequence Video to Text Venugopalan et al. Given a variable-length sequence of video frames, Prev: Title Slide generate a variable-length natural language Problem description of the video. Next: Motivation

850 views • 15 slides
The Gibbs Sampler CSE 527 Lecture 9 Lawrence, et al. Detecting Subtle Sequence

The Gibbs Sampler CSE 527 Lecture 9 Lawrence, et al. Detecting Subtle Sequence

The Gibbs Sampler CSE 527 Lecture 9 Lawrence, et al. Detecting Subtle Sequence Signals: A The Gibbs Sampler Gibbs Sampling Strategy for Multiple Sequence Alignment, Science 1993 The Double Helix Some DNA Binding Domains

62 views • 5 slides
A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples Qiang Zeng ,

A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples Qiang Zeng ,

A Multiversion Programming Inspired Approach to Detecting Audio Adversarial Examples Qiang Zeng , Jianhai Su, Chenglong Fu, Golam Kayas, Lannan Luo, Xiaojiang Du, Chiu C. Tan, and Jie Wu DSN 2019 2 3 Audio AE generation Open the front

353 views • 20 slides
Text 1. A text is a sequence of characters 2. Each character is taken from a finite alphabete

Text 1. A text is a sequence of characters 2. Each character is taken from a finite alphabete

Computer Architecture applied computer science 02.03 Representation of non-numerical sets urbino worldwide campus 02 Information theory 02.03 Representation of non-numerical sets Texts Images Signals (Audio/Video)

420 views • 6 slides
Pierce Gierloff Health Depression Essay Depression Depression is a very serious, but also very

Pierce Gierloff Health Depression Essay Depression Depression is a very serious, but also very

Pierce Gierloff Health Depression Essay Depression Depression is a very serious, but also very treatable disorder that concerns millions of people every day, it can be defined as a feeling of severe despondency and dejection. Depression has

309 views • 4 slides
Where do the improvements come from in sequence-to-sequence neural TTS? Oliver Watts Gustav

Where do the improvements come from in sequence-to-sequence neural TTS? Oliver Watts Gustav

Where do the improvements come from in sequence-to-sequence neural TTS? Oliver Watts Gustav Eje Henter Jason Fong Cassia Valentini-Botinhao Input feature Text TEXT extraction analysis com- Input layer Hidden layers Output layer

795 views • 48 slides
Sequence to Sequence Video to Text Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue Raymond

Sequence to Sequence Video to Text Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue Raymond

Sequence to Sequence Video to Text Subhashini Venugopalan, Marcus Rohrbach, Jeff Donahue Raymond Mooney, Trevor Darrell, Kate Saenko Outline Objective Experimental Setup Current model. A Simple Extension. How is information

474 views • 24 slides
Text Classification and Sequence Labeling Graham Neubig Text Classification

Text Classification and Sequence Labeling Graham Neubig Text Classification

CMU CS11-737: Multilingual NLP Text Classification and Sequence Labeling Graham Neubig Text Classification Given an input text X , predict an output label y Topic Classification food food politics politics I like

792 views • 53 slides
On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition Kazuki Irie, Rohit

On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition Kazuki Irie, Rohit

On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition Kazuki Irie, Rohit Prabhavalkar, Anjuli Kannan, Antoine Bruguier, David Rybach, Patrick Nguyen Interspeech 2019, Graz, Austria

742 views • 25 slides
The Great Depression Outcome: Causes of The Great Depression Causes of The Great Depression 1. A

The Great Depression Outcome: Causes of The Great Depression Causes of The Great Depression 1. A

The Great Depression Outcome: Causes of The Great Depression Causes of The Great Depression 1. A Depression is: a. One phase in a larger business cycle b. Characterized by a vicious cycle of: i. Low consumer/trust/confidence ---> Low demand for

435 views • 9 slides
Review: Acoustic Modeling x Observations; sequence of 40d feature vectors. Lecture 6

Review: Acoustic Modeling x Observations; sequence of 40d feature vectors. Lecture 6

Review: Acoustic Modeling x Observations; sequence of 40d feature vectors. Lecture 6 word sequence. Language Modeling/Pronunciation Modeling HMM/GMM framework lets us model P ( x | ) . . . How likely feature vectors are given

608 views • 36 slides
Neural AMR : Sequence-to-Sequence Models for Parsing and Generation annis Konstas joint work

Neural AMR : Sequence-to-Sequence Models for Parsing and Generation annis Konstas joint work

Neural AMR : Sequence-to-Sequence Models for Parsing and Generation annis Konstas joint work with Srinivasan Iyer, Mark Yatskar, Yejin Choi, Luke Zettlemoyer AMR graph Generate from AMR graph text Decoder Encoder text Attention AMR

1.32k views • 116 slides
The Rise in Depressive Disorder The Costs of Depression Rates of depression have radically

The Rise in Depressive Disorder The Costs of Depression Rates of depression have radically

Out of the Blue: Challenging Myths Myths about depression: Six non-medication ways to Cause is known (biochemical and genetic) Despite the ads one sees on TV, the cause of depression is not known and has not been established as

359 views • 24 slides
Detecting Outliers in HMM modeling through Relative Entropy with Applications to Change-Point

Detecting Outliers in HMM modeling through Relative Entropy with Applications to Change-Point

Detecting Outliers in HMM modeling through Relative Entropy with Applications to Change-Point Detection V. Perduca, G. Nuel JOBIM 2012 V. Perduca (MAP 5 - Paris Descartes) Detecting Outliers in HMM modeling JOBIM 2012 1 / 17 Change-point

429 views • 27 slides
Use Cases and Interaction Diagrams (Sequence and Collaboration) Hours 6, 7, 9, and 10 Scenarios

Use Cases and Interaction Diagrams (Sequence and Collaboration) Hours 6, 7, 9, and 10 Scenarios

Use Cases and Interaction Diagrams (Sequence and Collaboration) Hours 6, 7, 9, and 10 Scenarios The starting point # A scenario is a sequence of steps # Scenarios are presented in text format # No UML diagram contains the text # Drives use

258 views • 6 slides
Detecting a Change of Style Using Text Statistics Kamil Safin Aleksandr Ogaltsov Antiplagiat

Detecting a Change of Style Using Text Statistics Kamil Safin Aleksandr Ogaltsov Antiplagiat

Detecting a Change of Style Using Text Statistics Kamil Safin Aleksandr Ogaltsov Antiplagiat Company Moscow Institute of Physics and Technology Higher School of Economics 1 / 10 PAN18 competition Tasks Author identification task.

206 views • 10 slides
Causes of The Great Depression The Great Depression is one of the most misunderstood events

Causes of The Great Depression The Great Depression is one of the most misunderstood events

Causes of The Great Depression The Great Depression is one of the most misunderstood events in American history Some point to the Crash of the Stock Market as the cause of the Depression Not true. Some blame Herbert Hoover,

1.61k views • 148 slides
Parallel Latent Change Modeling of Depression and Pain to Predict Relapse During Buprenorphine

Parallel Latent Change Modeling of Depression and Pain to Predict Relapse During Buprenorphine

Parallel Latent Change Modeling of Depression and Pain to Predict Relapse During Buprenorphine and Suboxone Treatment Stanford Division of Pain Medicine Presenter: Noel A. Vest, Ph.D. October 17, 2019 @n o e l v e s t @n o e l v e s t

556 views • 17 slides
Detecting Changes and Anomalies in Noisy Text Streams Jerry Wright Networking and Services

Detecting Changes and Anomalies in Noisy Text Streams Jerry Wright Networking and Services

CoCITe Noise Mixture Distributions Results Summary Detecting Changes and Anomalies in Noisy Text Streams Jerry Wright Networking and Services Research Lab AT&T Labs Research 15 February 2010 Noisy Text Streams CoCITe Noise

350 views • 34 slides
Modeling with UML Chapter 2, lecture 1, Overview: modeling with UML What is modeling?

Modeling with UML Chapter 2, lecture 1, Overview: modeling with UML What is modeling?

Object-Oriented Software Engineering Using UML, Patterns, and Java Modeling with UML Chapter 2, lecture 1, Overview: modeling with UML What is modeling? What is UML? Use case diagrams Class diagrams Sequence diagrams Bernd

357 views • 22 slides
12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic

12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic

12/6/2013 Detecting Fakes Image Forensics: Detecting Forged Photos 1.Detecting photorealistic graphics 2.Detecting manipulated images Photo manipulation is the application of image editing techniques to photographs in order to create an

306 views • 13 slides

More recommend