text mining on clinical data
play

Text Mining on Clinical Data Robert McHardy Outline Motivation - PowerPoint PPT Presentation

Dec 15, 2022 •345 likes •1.18k views

Institut fr Maschinelle Sprachverarbeitung Text Mining on Clinical Data Robert McHardy Outline Motivation Medical Entity Recognition Anonymization of Medical Reports Knowledge-based Biomedical Word Sense Disambiguation

  1. Institut für Maschinelle Sprachverarbeitung Text Mining on Clinical Data Robert McHardy

  2. Outline • Motivation • Medical Entity Recognition • Anonymization of Medical Reports • Knowledge-based Biomedical Word Sense Disambiguation • Extraction of Potential Adverse Drug Events • Resources Universität Stuttgart 5.12.2017 2

  3. Motivation — Different Users Universität Stuttgart 5.12.2017 3

  4. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not Universität Stuttgart 5.12.2017 4

  5. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible Universität Stuttgart 5.12.2017 4

  6. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible • We don‘t want to suffer from unsafe drugs Universität Stuttgart 5.12.2017 4

  7. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible • We don‘t want to suffer from unsafe drugs • Researchers want to use the data Universität Stuttgart 5.12.2017 4

  8. Motivation — Why do we need Text Mining on Clinical Data? • Doctors need to know if a drug is safe to use or not • As fast as possible • We don‘t want to suffer from unsafe drugs • Researchers want to use the data • It has to be anonymized Universität Stuttgart 5.12.2017 4

  9. Motivation — PubMed, again! Universität Stuttgart 5.12.2017 5

  10. Unified Medical Language System Metathesaurus (UMLS) Universität Stuttgart 5.12.2017 6

  11. Medical Entity Recognition — Overview • Abacha and Zweigenbaum: Consists of two parts • Detecting phrases referring to medical entities • Assigning semantic categories to the found entities Universität Stuttgart 5.12.2017 7

  12. Medical Entity Recognition — Overview Universität Stuttgart 5.12.2017 8

  13. Medical Entity Recognition — Overview Type 1 diabetes T1D Diabetes type 1 IDDM Juvenile diabetes Universität Stuttgart 5.12.2017 8

  14. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] Universität Stuttgart 5.12.2017 9

  15. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Many tools for NP chunking available Universität Stuttgart 5.12.2017 9

  16. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Many tools for NP chunking available • Maximum recall is desired Universität Stuttgart 5.12.2017 9

  17. Medical Entity Recognition — Noun Phrase Chunking Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Many tools for NP chunking available • Maximum recall is desired • Open-domain tools like IMS‘ TreeTagger are suitable Universität Stuttgart 5.12.2017 9

  18. Medical Entity Recognition — MetaMap and the UMLS • MetaMap is a tool which maps noun phrases in raw text to UMLS concepts • This is done according to a matching score Universität Stuttgart 5.12.2017 10

  19. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap Universität Stuttgart 5.12.2017 11

  20. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools Universität Stuttgart 5.12.2017 11

  21. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs Universität Stuttgart 5.12.2017 11

  22. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs • Some ambiguity is left Universität Stuttgart 5.12.2017 11

  23. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs • Some ambiguity is left • UMLS can provide several concepts for a term Universität Stuttgart 5.12.2017 11

  24. Medical Entity Recognition — MetaMap and the UMLS • Three problems with MetaMap • Noun chunking performance is worse than with specialized NLP tools • Medical entity detection often finds verbs and general words which aren‘t MEs • Some ambiguity is left • UMLS can provide several concepts for a term • and several semantic categories for a concept Universität Stuttgart 5.12.2017 11

  25. Medical Entity Recognition — MetaMap and the UMLS Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] Cold temperature Common cold Cold ( term) Cold storage ( term) Cold storage Chronic obstructive lung disease Universität Stuttgart 5.12.2017 12

  26. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking Universität Stuttgart 5.12.2017 13

  27. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list Universität Stuttgart 5.12.2017 13

  28. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list • Search in specialized lists for candidate terms Universität Stuttgart 5.12.2017 13

  29. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list • Search in specialized lists for candidate terms • Annotate entities with MetaMap Universität Stuttgart 5.12.2017 13

  30. Medical Entity Recognition — MetaMap+ • Use tools like TreeTagger for the NP chunking • Filter NPs with a stop-word list • Search in specialized lists for candidate terms • Annotate entities with MetaMap • Filter frequent errors and too broad semantic types Universität Stuttgart 5.12.2017 13

  31. Medical Entity Recognition — MetaMap+ • Voting mechanism to disambiguate semantic categories Universität Stuttgart 5.12.2017 14

  32. Medical Entity Recognition — Support Vector Machines (SVMs) • Word level features: • words of the NP • number of words of the NP • window of words around the NP • Orthographical features: • first letter capitalized • all letters upper-/lowercase • contains abbreviation(s) • POS tags Universität Stuttgart 5.12.2017 15

  33. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O Universität Stuttgart 5.12.2017 16

  34. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O • B-x: Begin of a phrase of class x Universität Stuttgart 5.12.2017 16

  35. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O • B-x: Begin of a phrase of class x • I-x: Intermediate part of a phrase of class x Universität Stuttgart 5.12.2017 16

  36. Medical Entity Recognition — BIO-CRFs Pharmacodynamic studies, including positron-emission tomography (PET) and computed tomography (CT) […] • Words are annotated with the the tags B, I and O • B-x: Begin of a phrase of class x • I-x: Intermediate part of a phrase of class x • O: Outside entities Universität Stuttgart 5.12.2017 16

  37. Medical Entity Recognition — BIO-CRFs • Word level features: • The word itself • Window of words • Lemmas • Orthographical features: • Upper/lowercase • contains a digit • pre- and suffixes • POS tags • (Semantic category of word (provided by MetaMap+)) Universität Stuttgart 5.12.2017 17

  38. Medical Entity Recognition — Evaluation • Corpus contains discharge summaries and progress notes • De-identified and annotated by hand • Entities: Problem, Treatment and Test • Overall 76,665 sentences Universität Stuttgart 5.12.2017 18

  39. Medical Entity Recognition — Evaluation Setting Precision Recall F-Score MetaMap 15.52 16.10 15.80 MetaMap+ 48.68 56.46 52.28 SVM 43.65 47.16 45.33 BIO-CRF 70.15 83.31 76.17 BIO-CRF-Hybrid 72.18 83.78 77.55 Universität Stuttgart 20.01.2016 19

  40. Anonymization of Medical Reports Universität Stuttgart 20.01.2016 20

  41. Anonymization of Medical Reports — What is anonymization? • De-Identification Universität Stuttgart 5.12.2017 21

  42. Anonymization of Medical Reports — What is anonymization? • De-Identification • Completely remove all personal health information Universität Stuttgart 5.12.2017 21

Recommend

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com Tutorial on Machine Learning with R The Melbourne Data Science Week 2017 1 June 2017 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies .

733 views • 57 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011 Machine Learning & Computational Biology Research Group MPIs Tbingen .: Data Mining in Bioinformatics, Page 1 What is text mining?

813 views • 31 slides
2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118 / 179 Text Mining Goals To learn key problems and techniques in the mining one of the most common types of data To learn how to represent text

738 views • 59 slides
Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Motivation for Text Mining Motivation for Text Mining Approximately 90% of the Worlds data is held in unstructured formats Text Mining Text Mining Web pages Emails Technical documents Corporate documents Books

456 views • 9 slides
CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Text Data: Topic Model Instructor: Yizhou Sun yzsun@cs.ucla.edu December 4, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

418 views • 30 slides
Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao

Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao

Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao http://www.RDataMining.com Short Course on R and Data Mining University of Canberra 7 October 2016 1 Chapter 10: Text Mining, in R and Data Mining:

798 views • 40 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tbingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 What is text mining? Definition Text mining is the use

603 views • 31 slides
Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early stage demen1a: the SAMS framework Christopher Bull, Dommy Asfiandy, Ann Gledson, Joseph

284 views • 14 slides
Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow) & David Robinson (DataCamp) https://www.tidytextmining.com/ R for Data Science: Garrett Grolemund & Hadley Wickham http://r4ds.had.co.nz/

866 views • 15 slides
Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire Beatrice Alex balex@inf.ed.ac.uk Digital scholarship: day of ideas 2, Edinburgh, 02/05/2013 OVERVIEW What is text mining? Text Mining in digital

484 views • 27 slides
CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu

CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu

CS6220: DATA MINING TECHNIQUES Text Data: Topic Models Instructor: Yizhou Sun yzsun@ccs.neu.edu February 17, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Decision Tree;

337 views • 17 slides
Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar,

Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar,

Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar, University of Edinburgh November 19th 2013 Monday, 25 November 2013 MINING WHAT DATA? Electronic text or things that can be turned into it. Born

911 views • 51 slides
Data Mining 2020 Text Classification Naive Bayes Ad Feelders Universiteit Utrecht Ad Feelders

Data Mining 2020 Text Classification Naive Bayes Ad Feelders Universiteit Utrecht Ad Feelders

Data Mining 2020 Text Classification Naive Bayes Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Data Mining 1 / 49 Text Mining Text Mining is data mining applied to text data. Often uses well-known data mining

597 views • 49 slides
MINING Text Data: Nave Bayes Instructor: Yizhou Sun yzsun@cs.ucla.edu December 7, 2017

MINING Text Data: Nave Bayes Instructor: Yizhou Sun yzsun@cs.ucla.edu December 7, 2017

CS145: INTRODUCTION TO DATA MINING Text Data: Nave Bayes Instructor: Yizhou Sun yzsun@cs.ucla.edu December 7, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

903 views • 26 slides
Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt

Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt

Data Mining in Bioinformatics Day 9: String & Text Mining in Bioinformatics Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tbingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 Why compare sequences?

786 views • 25 slides
Text mining with ngram variables Matthias Schonlau, Ph.D. University of Waterloo, Canada

Text mining with ngram variables Matthias Schonlau, Ph.D. University of Waterloo, Canada

Text mining with ngram variables Matthias Schonlau, Ph.D. University of Waterloo, Canada What to do with text data? The most common approach to dealing with text data is as follows: Step 1: encode text data into numeric variables

673 views • 41 slides
Web Mining and Recommender Systems T ext Mining Learning Goals Introduce the topic of text

Web Mining and Recommender Systems T ext Mining Learning Goals Introduce the topic of text

Web Mining and Recommender Systems T ext Mining Learning Goals Introduce the topic of text mining Describe some of the difficulties of dealing with textual data Administrivia Midterm will be handed out after class next Monday Nov 9

891 views • 87 slides
Text Mining in Clinical Domain: Dealing with Noise Author: Hoang Nguyen, Jon Patrick Source:

Text Mining in Clinical Domain: Dealing with Noise Author: Hoang Nguyen, Jon Patrick Source:

Text Mining in Clinical Domain: Dealing with Noise Author: Hoang Nguyen, Jon Patrick Source: KDD16 Advisor: Jia-Ling Koh Speaker: Avon Yu Date: 2018/12/4 1 Outline Introduc*on Method Experiment Conclusion 2

388 views • 24 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 16, 2015 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

936 views • 38 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

724 views • 54 slides
SAMS: Data and Text Mining for Early Detection of Alzheimers Disease November, 2016 Dr

SAMS: Data and Text Mining for Early Detection of Alzheimers Disease November, 2016 Dr

SAMS: Data and Text Mining for Early Detection of Alzheimers Disease November, 2016 Dr Christopher Bull Aim of talk What is SAMS Data Capture Problems and solutions to acquiring this type of text/data NLP Tools used

725 views • 33 slides
CS145: INTRODUCTION TO DATA MINING Sequence Data: Similarity Search Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Sequence Data: Similarity Search Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING Sequence Data: Similarity Search Instructor: Yizhou Sun yzsun@cs.ucla.edu November 27, 2017 Methods to be Learnt Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

614 views • 46 slides
CS145: INTRODUCTION TO DATA MINING 09: Vector Data: Clustering Basics Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING 09: Vector Data: Clustering Basics Instructor: Yizhou Sun

CS145: INTRODUCTION TO DATA MINING 09: Vector Data: Clustering Basics Instructor: Yizhou Sun yzsun@cs.ucla.edu October 27, 2017 Methods to Learn Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for Text

713 views • 39 slides

More recommend