text mining for historical documents motivation and case
play

Text Mining for Historical Documents Motivation and Case Studies - PowerPoint PPT Presentation

Aug 27, 2023 •107 likes •288 views

Text Mining for Historical Documents Motivation and Case Studies Caroline Sporleder Computational Linguistics/MMCI Universit at des Saarlandes Wintersemester 2011/12 22.02.2012 Caroline Sporleder Text Mining for Historical Documents IT

  1. Text Mining for Historical Documents Motivation and Case Studies Caroline Sporleder Computational Linguistics/MMCI Universit¨ at des Saarlandes Wintersemester 2011/12 22.02.2012 Caroline Sporleder Text Mining for Historical Documents

  2. IT and Cultural Heritage: Why bother? (1) Museums, archives and libraries possess large collections of data artefacts books, manuscripts meta-data: catalogues, field books, reports etc. More and more digitisation projects governments have come to see CH as a valuable asset digitised data can be accessed more easily digitisation as a safeguard against data loss Caroline Sporleder Text Mining for Historical Documents

  3. IT and Cultural Heritage: Why bother? (2) Digitisation offers opportunities easier data access (searching, browsing) presentation of data (visualisation) knowledge discovery support for curation (partial automisation, consistency checking) Caroline Sporleder Text Mining for Historical Documents

  4. IT and Cultural Heritage: Why bother? (3) But to make the most of digitised data, we need sophisticated tools information retrieval (data indexing and searching) information extraction (linguistic data analysis) automatic data linking discovery of trends and interdependencies data presentation (for experts and non-experts) meta-data enrichment (linguistic disambiguation, semantic tagging, automatic transcription of audio data etc.) semi-automatic curation (data completion, error detection, consistency enforcement) ⇒ text mining and natural language processing (NLP) play a big role because much of primary and most meta-data are textual Caroline Sporleder Text Mining for Historical Documents

  5. Case Study: Naturalis The Dutch National Museum of Natural History Caroline Sporleder Text Mining for Historical Documents

  6. Naturalis: The Collection (1) more than 10 million specimens: 5,250,000 insects 2,290,000 invertebrates 1,000,000 vertebrates 1,160,000 fossils 440,000 stones and minerals 150,000 species 10% of the Earth’s biodiversity Caroline Sporleder Text Mining for Historical Documents

  7. Naturalis: The Collection (2) Caroline Sporleder Text Mining for Historical Documents

  8. Data and Meta-Data For each of the 10M specimens a label attached to the specimen, providing basic details (biological name, where and when found, inventory number) an entry in a register book usually an entry in a field book Additionally, for many specimens an entry in a specimen data base a photo meta-data in the form of research papers etc. written about them Also: domain ontologies, taxonomic descriptions, maps etc. Caroline Sporleder Text Mining for Historical Documents

  9. Digitisation Efforts Convert field and register books into data bases take high quality digital photos of pages transcribe them manually Caroline Sporleder Text Mining for Historical Documents

  10. Digitisation of Fieldbooks Caroline Sporleder Text Mining for Historical Documents

  11. Digitisation of Fieldbooks Caroline Sporleder Text Mining for Historical Documents

  12. Example: Typist Guidelines Caroline Sporleder Text Mining for Historical Documents

  13. Example: Typist Guidelines Caroline Sporleder Text Mining for Historical Documents

  14. Example: Typist Guidelines Caroline Sporleder Text Mining for Historical Documents

  15. Transcription of Fieldbooks all fieldbooks relating to Reptiles and Amphibians Collection 15,000 handwritten pages manually transcribed by typists simple guidelines on how to deal with non-ASCII characters text written in the margins illegible passages etc. transcriptions completed in around 8 months < 5% error rate Caroline Sporleder Text Mining for Historical Documents

  16. Fieldbook Transcript 1 ex. Phyllobates femoralis At base of tree on small island, primary forest, 20.45-22.00 u. RMNH 23865 Lithodytes lineatus, Brownsberg, aan voet, onder stuk rot hout, 13.07.1968, 8.45 u., RMNH 26076 Dorsolateraal strepen heldergeel, tekening op dijen vuurrood, veel feller als bij Phyllobates femoralis. Gonyocephalus auritus Meyer, 3 ex. (1 juv.), Misool. Hoedt 1867. RMNH 17656 Eleutherodactylus zeuctotylus 1 [vrouw] Lelygebergte, 4 km N.O. van airstrip, distr. Marowijne, Suriname, 19-VIII-1975, onder stuk hout, 610m, l [plus] d M. S. Hoogmoed. Caroline Sporleder Text Mining for Historical Documents

  17. What can you do with it? (1) Caroline Sporleder Text Mining for Historical Documents

  18. What can you do with it? (2) Caroline Sporleder Text Mining for Historical Documents

Recommend

Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Text Mining Text Mining Web pages Emails Technical documents Corporate documents

Motivation for Text Mining Motivation for Text Mining Approximately 90% of the Worlds data is held in unstructured formats Text Mining Text Mining Web pages Emails Technical documents Corporate documents Books

456 views • 9 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com Tutorial on Machine Learning with R The Melbourne Data Science Week 2017 1 June 2017 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies .

733 views • 57 slides
Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow)

Text Mining with R Ben Williams 2018 Resources Text Mining with R: Julia Silge (StackOverflow) &amp; David Robinson (DataCamp) https://www.tidytextmining.com/ R for Data Science: Garrett Grolemund &amp; Hadley Wickham http://r4ds.had.co.nz/

866 views • 15 slides
Text Mining on Clinical Data Robert McHardy Outline Motivation Medical Entity

Text Mining on Clinical Data Robert McHardy Outline Motivation Medical Entity

Institut fr Maschinelle Sprachverarbeitung Text Mining on Clinical Data Robert McHardy Outline Motivation Medical Entity Recognition Anonymization of Medical Reports Knowledge-based Biomedical Word Sense Disambiguation

1.18k views • 83 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 21 to March 4, 2011 Machine Learning &amp; Computational Biology Research Group MPIs Tbingen .: Data Mining in Bioinformatics, Page 1 What is text mining?

813 views • 31 slides
2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118

2. Text Mining D-BSSE Karsten Borgwardt Data Mining II Course, Basel Spring Semester 2016 118 / 179 Text Mining Goals To learn key problems and techniques in the mining one of the most common types of data To learn how to represent text

738 views • 59 slides
Support Vector Machines for Large Scale Text Mining in R Ingo Feinerer 1 Alexandros Karatzoglou 2

Support Vector Machines for Large Scale Text Mining in R Ingo Feinerer 1 Alexandros Karatzoglou 2

Support Vector Machines for Large Scale Text Mining in R Ingo Feinerer 1 Alexandros Karatzoglou 2 1 Vienna University of Technology, Austria 2 Telefonica Research, Spain COMPSTAT2010 Motivation Machine learning and data mining require

451 views • 13 slides
Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview

Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview

Text Mining in Search Engines By: DJ Ambler With special thanks to the Internet Overview What is text mining? How is it used in search engines? Text Mining Definition A way to extract meaning from text Structuring, deriving

314 views • 10 slides
Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10

Data Mining in Bioinformatics Day 4: Text Mining Karsten Borgwardt February 25 to March 10 Bioinformatics Group MPIs Tbingen Karsten Borgwardt: Data Mining in Bioinformatics, Page 1 What is text mining? Definition Text mining is the use

603 views • 31 slides
Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early

Combining data mining and text mining for detec1on of early stage demen1a: the SAMS framework Christopher Bull, Dommy Asfiandy, Ann Gledson, Joseph

284 views • 14 slides
Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire

Digital history and big data: Text mining historical documents on trade in the British Empire Beatrice Alex balex@inf.ed.ac.uk Digital scholarship: day of ideas 2, Edinburgh, 02/05/2013 OVERVIEW What is text mining? Text Mining in digital

484 views • 27 slides
Web Mining and Recommender Systems T ext Mining Learning Goals Introduce the topic of text

Web Mining and Recommender Systems T ext Mining Learning Goals Introduce the topic of text

Web Mining and Recommender Systems T ext Mining Learning Goals Introduce the topic of text mining Describe some of the difficulties of dealing with textual data Administrivia Midterm will be handed out after class next Monday Nov 9

891 views • 87 slides
Text Mining and Historical Research Beatrice Alex balex@inf.ed.ac.uk MSc Historical Research,

Text Mining and Historical Research Beatrice Alex balex@inf.ed.ac.uk MSc Historical Research,

Text Mining and Historical Research Beatrice Alex balex@inf.ed.ac.uk MSc Historical Research, University of Edinburgh, March 21st 2014 Friday, 21 March 2014 OVERVIEW What is text mining? Types of text analyses. Trading Consequences: text

698 views • 36 slides
1 Introduction The Text Mining Process Text representation Learning Conclusion Introduction

1 Introduction The Text Mining Process Text representation Learning Conclusion Introduction

Introduction The Text Mining Process Text representation Learning Conclusion Introduction The Text Mining Process Text representation Learning Conclusion Aim of presentation Strategies in Identifying Issues Addressed in Legal Reports

640 views • 5 slides
BioMinT: Biological Text Mining EU FP5 Quality of Life Project Dr. Dipl.-Ing. Alexander K.

BioMinT: Biological Text Mining EU FP5 Quality of Life Project Dr. Dipl.-Ing. Alexander K.

BioMinT: Biological Text Mining EU FP5 Quality of Life Project Dr. Dipl.-Ing. Alexander K. Seewald sterreichisches Forschungsinstitut fr Artificial Intelligence Motivation Economic and business pressures are forcing drug companies to

293 views • 13 slides
Assisted Curation: Does Text Mining Really Help? (Alex et al. 2008) by Benedict Fehringer

Assisted Curation: Does Text Mining Really Help? (Alex et al. 2008) by Benedict Fehringer

Assisted Curation: Does Text Mining Really Help? (Alex et al. 2008) by Benedict Fehringer Seminar: Unlocking the Secrets of the Past: Text Mining for Historical Documents Supervisor: Dr. Caroline Sporleder (and Martin Schreiber)

900 views • 64 slides
Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao

Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao

Twitter Data Analysis with R Text Mining and Social Network Analysis 1 Yanchang Zhao http://www.RDataMining.com Short Course on R and Data Mining University of Canberra 7 October 2016 1 Chapter 10: Text Mining, in R and Data Mining:

798 views • 40 slides
Hotspot Mapper for World War II Unlocking the Secrets of the Past: Text Mining for Historical

Hotspot Mapper for World War II Unlocking the Secrets of the Past: Text Mining for Historical

Hotspot Mapper for World War II Unlocking the Secrets of the Past: Text Mining for Historical Documents Mariona Coll Ardanuy Seyed Mehdi Khodadad Hosseini Ehsan Khoddam Mohammadi Nikolina Koleva Peter Stahl Demo 2 Historical Motivation

419 views • 23 slides
Introduzione al text mining Outline Introduzione e concetti di base Motivazioni,

Introduzione al text mining Outline Introduzione e concetti di base Motivazioni,

Complex Data Mining &amp; Workflow Mining Introduzione al text mining Outline Introduzione e concetti di base Motivazioni, applicazioni Concetti di base nellanalisi dei dati complessi Text/Web Mining Concetti di base sul

1.34k views • 109 slides
Motivations Automated Text Mining and Vast Quantities of Text Available Knowledge Discovery

Motivations Automated Text Mining and Vast Quantities of Text Available Knowledge Discovery

A Visual Approach to Motivations Automated Text Mining and Vast Quantities of Text Available Knowledge Discovery Scientific Literature News Articles and Blogs Doctoral Dissertation by Email Andrey A. Puretskiy

336 views • 7 slides
Enabling Digital history: Text mining big historical document collections on trade in the British

Enabling Digital history: Text mining big historical document collections on trade in the British

Enabling Digital history: Text mining big historical document collections on trade in the British Empire Beatrice Alex balex@inf.ed.ac.uk PQIS All Team Meeting, ProQuest, April 23rd 2014 TEXT MINING D escribes a set of linguistic, statistical

408 views • 39 slides
Graph-Based Methods for M ltili Multilingual Text and Web l T t d W b Mining Mining Mark

Graph-Based Methods for M ltili Multilingual Text and Web l T t d W b Mining Mining Mark

Graph-Based Methods for M ltili Multilingual Text and Web l T t d W b Mining Mining Mark Last Department of Information Systems Engineering p y g g Ben-Gurion University of the Negev In cooperation with H Horst Bunke (University of

864 views • 53 slides
Massive Text Corpora Jialu Liu*, Jingbo Shang*, Chi Wang, Xiang Ren, Jiawei Han University of

Massive Text Corpora Jialu Liu*, Jingbo Shang*, Chi Wang, Xiang Ren, Jiawei Han University of

Mining Quality Phrases from Massive Text Corpora Jialu Liu*, Jingbo Shang*, Chi Wang, Xiang Ren, Jiawei Han University of Illinois at Urbana-Champaign SIGMOD 2015, May 2015 * Equal Contribution Outline Motivation: Why Phrase Mining?

785 views • 29 slides

More recommend