data mining in the humanities
play

Data Mining in the Humanities Text Data Beatrice Alex - PowerPoint PPT Presentation

Jul 16, 2023 •398 likes •911 views

Data Mining in the Humanities Text Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar, University of Edinburgh November 19th 2013 Monday, 25 November 2013 MINING WHAT DATA? Electronic text or things that can be turned into it. Born

  1. Data Mining in the Humanities Text ⊂ Data Beatrice Alex balex@inf.ed.ac.uk DigitalHSS Seminar, University of Edinburgh November 19th 2013 Monday, 25 November 2013

  2. MINING WHAT DATA? Electronic text or things that can be turned into it. Born electronic text (research papers, literature, tweets, blogs, comments on blogs etc.). Meta data (collection and document level). Image subtitles (Flickr image titles and subtitles). Video/audio transcripts (YouTube transcripts, TED talks, MOOC transcripts etc.) DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  3. TEXT MINING D escribes a set of linguistic, statistical and/or machine learning techniques that model and structure the information content of textual resources. Turns unstructured text into structured data (e.g. relational database or linked data) . Is very useful for analysing large text collections automatically (overcoming data paralysis). Goal in DHSS research: By analysing large amounts of digitised data, help HSS scholars to discover novel patterns and explore hypotheses. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  4. TYPES OF ANALYSES Word or n-gram frequencies, concordances or collocations analysis. Named entity recognition. Grounding, e.g. geo-referencing. Relation extraction. Clustering, e.g. topic modelling. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  5. NAMED ENTITY RECOGNITION Identification and classification of entity mentions in the text where entity refers to things like: Persons Locations Dates ... Often used for improving access to collections. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  6. CONNECTED HISTORIES http://www.connectedhistories.org DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  7. OLD BAILEY ONLINE http://www.oldbaileyonline.org DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  8. GROUNDING Linking entity mentions in text to a unique identifier, e.g.: Wikipedia pages Lat/longs or Geonames IDs Gene ontologies Goal is to disambiguate between mentions with the same surface form. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  9. GEO-REFERENCING http://placenames.org.uk DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  10. GEO-REFERENCING http://placenames.org.uk DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  11. GEO-REFERENCING Ian Gregory: Mapping the Lakes followed by Spacial Humanities Recognises the importance of geo- referencing in DH research. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  12. RELATION EXTRACTION Identifying relations between entities in text or in meta data Triples: person - author_of - book title, commodity - traded_at - location, person - born_in location DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  13. RELATION EXTRACTION ChartEX, Discovering spatial descriptions and relationships in medieval charters, October 2013. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  14. RELATION EXTRACTION ChartEX, Discovering spatial descriptions and relationships in medieval charters, October 2013. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  15. RELATION EXTRACTION ChartEX, Discovering spatial descriptions and relationships in medieval charters, October 2013. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  16. CLUSTERING Documents or words are clustered into groups based on word probabilities and other features. Single membership clustering versus multi- membership clustering. Hierarchical clustering. Topic modelling. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  17. HIERARCHICAL CLUSTERING Shakespeare Project and Visualising English Print Allison et al., 2011, Literary Lab Pamphlet 1. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  18. TOPIC MODELLING Analysis of Martha Ballard’s diary (27 years of daily entries) by Cameron Blevins DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  19. TOPIC MODELLING Cameron Blevins’ blog Analysis of Martha Ballard’s diary (27 years of daily entries) by Cameron Blevins DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  20. Ted Underwood’s blog DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  21. SUMMARY SO FAR Different types of text analyses applied to historical and literary research but new opportunities are endless. Text mining can assist scholars in their research but it is not replacing them! Traditional scholarship is well suited to close reading. DHSS scholars can focus on questions which can be answered by computational methods. Human interpretation is vital. Visualisation of TM output is important. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  22. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  23. PROJECT OVERVIEW JISC/SSHRC Digging into Data Challenge II Jan 2012 - Dec 2013 Text mining, data extraction and information visualisation to explore big historical datasets. Focus on how commodities were traded across the globe in the 19th century. Help historians to discover novel patterns and explore new research questions. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  24. PROJECT TEAM Ewan Klein, Bea Alex, Claire Grover, Richard Tobin: text mining Colin Coates, Jim Clifford: historical analysis James Reid, Nicola Osborne : data management, social media Aaron Quigley, Uta Hinrichs: information visualisation DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  25. TRADITIONAL HISTORICAL RESEARCH Global Fats Supply 1894-98 Gillow and the Use of Mahogany in the Eighteenth Century, Adam Bowett, Regional Furniture, v.XII, 1998. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  26. DOCUMENT COLLECTIONS Collection # of Documents # of Images House of Commons Parliamentary Papers 118,526 6,448,739 (ProQuest) Early Canadiana Online 83,016 3,938,758 Directors’ Letters of 14,340 n/a Correspondence (Kew) Confidential Prints (Adam 1,315 140,010 Matthews) Foreign and Commonwealth Office 1,000 41,611 Collection Asia and the West (Gale) 4,725 948,773 (OCRed: 450,841) DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  27. SYSTEM Lexicons & Gazetteers Annotated Documents Text Mining Documents XML 2 RDB Query Interface Commodities Commodities Ontology RDB S O K S Visualisation DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  28. MINED INFORMATION Example sentence: Normalised and grounded entities: commodity: cassia bark [concept: Cinnamomum cassia] date: 1871 (year=1871) location: Padang (lat=-0.94924;long=100.35427;country=ID) location: America (lat=39.76;long=-98.50;country=n/a) quantity + unit: 6,127 piculs DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  29. MINED INFORMATION Example sentence: Extracted entity attributes and relations: origin location: Padang destination location: America commodity–date relation: cassia bark – 1871 commodity–location relation: cassia bark – Padang commodity–location relation: cassia bark – America DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  30. EDINBURGH GEOPARSER DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  31. COMMODITY LEXICON Seed set from customs import records. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  32. COMMODITY LEXICON Seed set from customs import records. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  33. COMMODITY LEXICON Seed set from customs import records. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  34. SIBLING ACQUISITION ? DBpedia ? DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  35. LEXICON BOOTSTRAPPING Seed lexicon ~600 Extended lexicon ~17,000 With pluralisation of ~20,500 single word entries DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  36. EVALUATION DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

  37. INTERMEDIATE RESULTS Lexicon with 20,476 entries and 16,928 concepts. The prototype detected 31,169,104 commodities in 7 billion words. They correspond to 5,841 different commodities (4,466 concepts) and cover 28.5% of commodities in the lexicon. DigitalHSS Seminar, University of Edinburgh, November 19th 2013 Monday, 25 November 2013

Recommend

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data

DATA MINING LECTURE 2 What is data? The data mining pipeline What is Data Mining? Data mining is the use of efficient techniques for the analysis of very large collections of data and the extraction of useful and possibly unexpected

2.4k views • 94 slides
Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques

What is Web Mining? Wh t i W b Mi i What is Web Mining? Wh t i W b Mi i ? ? Web Mining Web Mining Web Mining Web Mining Web mining is the use of data mining techniques to automat cally d scover and extract nformat on automatically

774 views • 20 slides
Web Mining Web Mining Web mining is the use of data mining techniques to automatically

Web Mining Web Mining Web mining is the use of data mining techniques to automatically

What is Web Mining? What is Web Mining? Web Mining Web Mining Web mining is the use of data mining techniques to automatically discover and extract information from Web documents/services (Etzioni, 1996, CACM 39(11)) Web mining aims to

566 views • 22 slides
DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is

DATA MINING LECTURE 1 Introduction What is data mining? After years of data mining there is still no unique answer to this question. A tentative definition: Data mining is the use of efficient techniques for the analysis of very large

1.28k views • 62 slides
THE DATA MINING PIPELINE What is data? The data mining pipeline: collection, preprocessing,

THE DATA MINING PIPELINE What is data? The data mining pipeline: collection, preprocessing,

DATA MINING THE DATA MINING PIPELINE What is data? The data mining pipeline: collection, preprocessing, mining, and post-processing Sampling, feature extraction and normalization Exploratory analysis of data basic statistics What is data

2.11k views • 139 slides
Scotlands National Collections and the Digital Humanities, Edinburgh, 14/02/2014 PROJECT

Scotlands National Collections and the Digital Humanities, Edinburgh, 14/02/2014 PROJECT

Scotlands National Collections and the Digital Humanities, Edinburgh, 14/02/2014 PROJECT OVERVIEW JISC/SSHRC Digging into Data Challenge II Jan 2012 - Dec 2013 Text mining, data extraction and information visualisation to explore big

551 views • 15 slides
Key domain analysis: mining text in the humanities and social sciences Paul Rayson Department

Key domain analysis: mining text in the humanities and social sciences Paul Rayson Department

Key domain analysis: mining text in the humanities and social sciences Paul Rayson Department of Computing Lancaster University Dawn Archer Department of Humanities University of Central Lancashire A talk of two halves Motivation

361 views • 23 slides
Introduction What is data mining? to Data Mining: On what kind of data? Data Mining

Introduction What is data mining? to Data Mining: On what kind of data? Data Mining

Introduction Motivation: Why data mining? Introduction What is data mining? to Data Mining: On what kind of data? Data Mining Data mining functionalities Major issues in data mining 2 Motivation: Necessity is

438 views • 14 slides
LECTURE 1: INTRODUCTION TO DATA MINING Dr. Dhaval Patel CSE, IIT-Roorkee What is data mining?

LECTURE 1: INTRODUCTION TO DATA MINING Dr. Dhaval Patel CSE, IIT-Roorkee What is data mining?

LECTURE 1: INTRODUCTION TO DATA MINING Dr. Dhaval Patel CSE, IIT-Roorkee What is data mining? Data mining is also called knowledge discovery and data mining (KDD) Data mining is extraction of useful patterns from data sources , e.g.,

1.29k views • 50 slides
Data Mining: A Powerful Data Mining: A Powerful Tool for Data Cleaning Tool for Data Cleaning

Data Mining: A Powerful Data Mining: A Powerful Tool for Data Cleaning Tool for Data Cleaning

Data Mining: A Powerful Data Mining: A Powerful Tool for Data Cleaning Tool for Data Cleaning Jiawei Han Department of Computer Science University of Illinois at Urbana-Champaign Nov. 4, 2003 1 Data mining for data quality assurance

721 views • 40 slides
Modeling Data the different views on Data Mining Views on Data Mining Fitting the data

Modeling Data the different views on Data Mining Views on Data Mining Fitting the data

Modeling Data the different views on Data Mining Views on Data Mining Fitting the data Density Estimation Learning being able to perform a task more accurately than before Prediction use the data to predict future data

517 views • 29 slides
Data Mining: Concepts and Techniques Web Mining Li Xiong Slides credits: Jiawei Han and

Data Mining: Concepts and Techniques Web Mining Li Xiong Slides credits: Jiawei Han and

Data Mining: Concepts and Techniques Web Mining Li Xiong Slides credits: Jiawei Han and Micheline Kamber; Anand Rajaraman, Jeffrey D. Ullman Olfa Nasraoui Bing Liu 4/9/2008 1 Web Mining Web mining vs. data mining Structure (or lack

997 views • 60 slides
Introduction to Web Mining What is Web Mining? Discovering useful information from the

Introduction to Web Mining What is Web Mining? Discovering useful information from the

CS 345A Data Mining Lecture 1 Introduction to Web Mining What is Web Mining? Discovering useful information from the World-Wide Web and its usage patterns Web Mining v. Data Mining Structure (or lack of it) Textual information and

744 views • 31 slides
Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Contents Text Mining Concept Tasks Twitter Data Analysis with R Twitter Extracting Tweets

Text Mining with R Yanchang Zhao http://www.RDataMining.com R and Data Mining Course Beijing University of Posts and Telecommunications, Beijing, China July 2019 Chapter 10: Text Mining, in R and Data Mining: Examples and Case Studies

904 views • 64 slides
Introduction to Data Mining Methods and Tools by Michael Hahsler Agenda What is Data Mining?

Introduction to Data Mining Methods and Tools by Michael Hahsler Agenda What is Data Mining?

Introduction to Data Mining Methods and Tools by Michael Hahsler Agenda What is Data Mining? Data Mining T asks Relationship to Statistics, Optimization, Machine Learning and AI T ools Data Legal, Privacy and Security

1.02k views • 69 slides
Open Data in the Humanities: Data Sharing and Publication for Triadic Co-Creation Asanobu KITAM

Open Data in the Humanities: Data Sharing and Publication for Triadic Co-Creation Asanobu KITAM

Open Data in the Humanities: Data Sharing and Publication for Triadic Co-Creation Asanobu KITAM OTO Center for Open Data in the Humanities (CODH) Joint Support-Center for Data Science Research Research Organization and Information and Systems

589 views • 32 slides
Week 5 Video 2 Relationship Mining Causal Mining Causal Data Mining These slides developed in

Week 5 Video 2 Relationship Mining Causal Mining Causal Data Mining These slides developed in

Week 5 Video 2 Relationship Mining Causal Mining Causal Data Mining These slides developed in partnership with Stephen Fancsali, Carnegie Learning, Inc. Causal Data Mining Distinct from prediction or correlation mining The goal is not

491 views • 33 slides
Course Content Principles of Knowledge Introduction to Data Mining Discovery in Data

Course Content Principles of Knowledge Introduction to Data Mining Discovery in Data

Course Content Principles of Knowledge Introduction to Data Mining Discovery in Data Data warehousing and OLAP Data cleaning Fall 2004 Data mining operations Chapter 9: Web Mining Data summarization

582 views • 30 slides
Da Data Min a Minin ing What at i is it? Ch. 1 I-DM- 1 IRDM 15/16 What is Data Mining?

Da Data Min a Minin ing What at i is it? Ch. 1 I-DM- 1 IRDM 15/16 What is Data Mining?

Da Data Min a Minin ing What at i is it? Ch. 1 I-DM- 1 IRDM 15/16 What is Data Mining? Data mining is the process of extracting hidden patterns from data. An Unethical Econometric practice of massaging and manipulating the

475 views • 24 slides
Computational humanities Computational humanities 2019-07-17 Michael Piotrowski humanities.

Computational humanities Computational humanities 2019-07-17 Michael Piotrowski humanities.

1/37 Computational humanities Computational humanities 2019-07-17 Michael Piotrowski humanities. this summer school Franco Moretti (Hackler and Kirsten 2016, p. 5) tive and computational mean something. But, frankly, I

373 views • 19 slides
Web Mining Andreas Andersson Gustav Strmberg Sandra Stendahl Introduction Web mining o

Web Mining Andreas Andersson Gustav Strmberg Sandra Stendahl Introduction Web mining o

2011-11-30 Web Mining Andreas Andersson Gustav Strmberg Sandra Stendahl Introduction Web mining o Structure mining o Content mining o Usage mining Web usage mining o Acquire the data o Preprocess the data o Detect patterns in the data o

84 views • 5 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

724 views • 54 slides
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun

CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 16, 2015 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data Network Classification

936 views • 38 slides
CS145: INTRODUCTION TO DATA MINING Sequence Data: Sequential Pattern Mining Instructor: Yizhou

CS145: INTRODUCTION TO DATA MINING Sequence Data: Sequential Pattern Mining Instructor: Yizhou

CS145: INTRODUCTION TO DATA MINING Sequence Data: Sequential Pattern Mining Instructor: Yizhou Sun yzsun@cs.ucla.edu November 27, 2017 Methods to Learn Vector Data Set Data Sequence Data Text Data Logistic Regression; Nave Bayes for

616 views • 42 slides

More recommend