subject indexing
play

subject indexing Matias Frosterus, Jarmo Saarikko & Okko - PowerPoint PPT Presentation

Apr 15, 2023 •545 likes •937 views

20 million URIs and the overhaul of the Finnish library sector subject indexing Matias Frosterus, Jarmo Saarikko & Okko Vainonen The National Library of Finland SWIB19, Hamburg, Germany, 26-Nov-2019 KANSALLISKIRJASTO The Overhaul of

  1. 20 million URIs and the overhaul of the Finnish library sector subject indexing Matias Frosterus, Jarmo Saarikko & Okko Vainonen The National Library of Finland SWIB19, Hamburg, Germany, 26-Nov-2019 KANSALLISKIRJASTO

  2. The Overhaul of subject indexing in Finnish libraries: 2019  The goal:  moving from monolingual thesauri to • multilingual, • machine-readable, • interlinked • SKOS vocabularies 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 2

  3. The Overhaul of subject indexing in Finnish libraries: 2019  The motivation: • Indexing in one language allows for searching in another • Links to other vocabularies allows for interoperability • Moving from terms to concepts with URIs makes updating easier 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 3

  4. The vocabularies YSA  General Finnish Thesaurus YSA was the most used thesaurus in Finland • Developed since the 1980s • Used to describe all of the non-fictional literature published in Finland • Monolingual 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 4

  5. The vocabularies YSA  Swedish language counterpart called Allärs • Finnish-Swedish, to be precise • Very slightly different Allärs structure due to linguistic differences 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 5

  6. The vocabularies YSA  In 2018 MUSA, a thesaurus of music terms was absorbed into YSA MUSA • Cilla, the Swedish language counterpart of Allärs MUSA, absorbed respectively into Allärs Cilla 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 6

  7. The vocabularies YSO YSA MUSA YSO places Allärs Cilla In 2003 FinnONTO research project began work on the General Finnish Ontology YSO 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 7

  8. General Finnish Ontology YSO  Based on YSA and Allärs • Places as a separate vocabulary YSO Places  From terms to concepts identified by URIs  Concepts based on Finnish and Swedish • Translated into English  Complete hierarchy and clearly defined semantics  Linked • to Finnish ontologies of other domains • Library of Congress Subject Headings, Wikidata 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 8

  9. The vocabularies YSO YSA MUSA YSO places Allärs Cilla FGF SEKO Two more vocabularies for the conversion 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 9

  10. Scope expanded  Many vocabularies  Dismantling subfields used in subject indexing ” chains ”  New MARC fields 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 10

  11. Lessons learned: Communication Project Complex Live status itself details Schedules Guides Push-type Pull-type messages messages Metadata Library directors Others affected specialists 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 11

  12. Conversion: Authority Records Various Various REST Library Various Library Various API Systems Library Systems Library Systems Systems 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 12

  13. Conversion: Authority Records Various Various REST Library Various Library Various API Systems Library Systems Library Systems Systems SKOS to MARC Converter 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 13

  14. SKOS Record for yso:p16239 yso:p16239 a skos:Concept, <http://www.yso.fi/onto/yso-meta/Concept> ; skos:prefLabel "morgon"@sv, "aamu"@fi, "morning"@en ; skos:broader yso:p5264 ; skos:exactMatch koko:p17356, ysa:Y109535, allars:Y23054 ; skos:closeMatch <http://id.loc.gov/authorities/subjects/sh2004006540> ; dc:modified "2017-05-10"^^xsd:date ; skos:inScheme yso: . 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 14

  15. MARC Authority File for yso:p16239 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 15

  16. Conversion: Bibliographic Records Various Various Library BIB Various Library Various Systems Library records Systems Library Systems Systems 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 16

  17. Conversion: Bibliographic Records Various Various Library BIB Various Library Various Systems Library records Systems Library Systems Systems BIB Converter BIB records 2019-11- SWIB19 Frosterus, Saarikko & Vainonen 17 26

  18. Two sets of rules SKOS to  An expert group made up of indexing MARC specialists from various national groups Converter and libraries  Two sets of rules BIB • SKOS to MARC for authority records Converter • BIB conversion rules • Separate rules for fiction and non- fiction and music/film due to different indexing rules 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 18

  19. Dismantling subfields in subject indexing  New subject indexing rules use only one subfield for each term • Existing records had not been converted  All in all proved to be a very complex task • Same MARC fields and subfields but different conventions for different types of content • Specific “labels” that changed the meaning of subfields • The conventions had changed over time and older ones were difficult to re-engineer 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 19

  20. Example of Conversion 650#7 |a hard rock |z Finland |y 2000-2009 |2 allars The publication is about Finnish rock music 648 #7 |a 2000-2009 650 #7 |a hard rock |2 yso/swe |0 http://www.yso.fi/onto/yso/p29778 651 #7 |a Finland |2 yso/swe |0 http://www.yso.fi/onto/yso/p94426 The publication is a music score, recording or video 370 #7 |g Finland |2 yso/swe |0 http://www.yso.fi/onto/yso/p94426 388 1# |a 2000-2009 655 #7 |a hard rock |2 slm/swe |0 http://urn.fi/URN:NBN:fi:au:slm:s828 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 20

  21. Coverage of the conversion  National union catalog Melinda  Local library databases employing various library systems (Voyager, Koha, Axiell Aurora, etc.) • Both universities and public libraries  Other systems that were using YSA/Allärs • E.g., government institutions 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 21

  22. Lessons learned: Unwritten conventions • History has a tendency to accumulate • Including experts widely is key 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 22

  23. Coding the conversions  2 programs • SKOS to MARC authorities • Changing terms in MARC BIB-records  Open source Python3 code  Available to libraries and library system providers • https://github.com/NatLibFi/Finto- data/tree/master/tools/finto-skos-to-marc • https://github.com/NatLibFi/yso-marcbib 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 23

  24. Lessons learned: Complexity of programming  Original plan • Take each term and switch it to the label of the same concept in the other vocabulary  Reality • Metadata in data • Meanings of terms were interdependent • Content type affected the use of MARC fields • Many analyses had to be done before selecting the ” correct ” term 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 24

  25. Process of BIB-record conversion Repair MARC21 Select field DB places records Select term Select rule Marc21 Finto YSO Find matching Write error FGF term field SEKO Write record checklist Create new field Remove old Removed Sort fields Write old field field fields 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 25

  26. Conversion of MARC BIB records  Conversion analyzed fields: Select field • 648, 650, 651, 655 • Field was analyzed only if subfield |2 value was ysa, allars, musa or cilla  Conversion created fields: • 257 , 370 , 382 , 388 , 648, 650, 651, 653 , 655  For YSO and FGF terms we also added language independent concept URIs to the |0 -subfield 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 26

  27. Finding place subfields first  Identify and concatenate place Repair places subfields that are concatenated in the vocabulary (e.g. city districts)  650#7 |aJAZZ |zHelsinki |zEira Search for ” Helsinki - - Eira ” label in the SKOS-vocabulary 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 27

  28. Coding the conversion: matching the concepts Repair places ysa:Y116934 skos:exactMatch yso:p116934 . 370## |g Eira (Helsinki) |2 yso/fin |0 http://... 370## |g Eira (Helsingfors) |2 yso/swe |0 http://... 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 28

  29. Using the subfield 8 to identify connected terms  Example of a symphony composed in 1900 and performed in 2019 Create new field 650#7 |a sinfoniat |y 1900 |z Helsinki |2 ysa 650#7 |a sinfoniat |y 2019 |z Wien |2 ysa 650#7 |a sinfoniaorkesterit |2 ysa 370#7 |81\u |g Helsinki |2 yso/fin |0 http://www.yso.fi/onto/yso/p94137 370#7 |82\u |g Wien |2 yso/fin |0 http://www.yso.fi/onto/yso/p106956 382#1 |a sinfoniaorkesteri |2 seko |0 http://urn.fi/urn:nbn:fi:au:seko:00936 388#7 |81\u |a 1900 ‡ 2yso/fin 388#7 |82\u |a 2019 ‡ 2yso/fin 655#7 |81\u |82\u |a sinfoniat |2 slm/fin |0 http://urn.fi/URN:NBN:fi:au:slm:s917  MARC21 subfield 8 links all related fields  Years are not (yet) authorized in Finnish thesauri 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 29

  30. Sorting the fields  We tried to keep the original order of Sort fields first occurence of terms  New fields were sorted according to field number, 2nd indicator, vocabulary identifier  We checked and removed any duplicate fields 2019-11-26 SWIB19 Frosterus, Saarikko & Vainonen 30

Recommend

Distributed Indexing Indexing, session 8 CS6200: Information Retrieval Slides by: Jesse Anderton

Distributed Indexing Indexing, session 8 CS6200: Information Retrieval Slides by: Jesse Anderton

Distributed Indexing Indexing, session 8 CS6200: Information Retrieval Slides by: Jesse Anderton Distributing Indexing The scale of web indexing makes it infeasible to maintain an index on a single computer. Instead, we distribute the task

383 views • 36 slides
Indexing Presentation - The Basics Attached is the slide deck for a short presentation on indexing

Indexing Presentation - The Basics Attached is the slide deck for a short presentation on indexing

Steve Stedman Freelance SQL Server Consultant http://stevestedman.com Indexing Presentation - The Basics Attached is the slide deck for a short presentation on indexing that was created to educate software developers on the basics of indexing.

173 views • 4 slides
PDL Basics of Indexing and Threading Outline Motivation Indexing Dimension

PDL Basics of Indexing and Threading Outline Motivation Indexing Dimension

PDL Basics of Indexing and Threading Outline Motivation Indexing Dimension manipulation Slicing Parent and child relation Threading Function's signature The core and extra dimensions References: 1.

470 views • 31 slides
Bitmap Indexing and related indexing techniques Presented by: El Ghailani Maher Outline I

Bitmap Indexing and related indexing techniques Presented by: El Ghailani Maher Outline I

Bitmap Indexing and related indexing techniques Presented by: El Ghailani Maher Outline I ntroduction ! W hy I ndexing? ! Factors that determ ine the convenient I ndexing ! technique Criteria to develop a new indexing technique ! Bitm

516 views • 41 slides
Proof-Theoretic Foundations of Indexing in Logic Programming Iliano Cervesato iliano@cmu.edu

Proof-Theoretic Foundations of Indexing in Logic Programming Iliano Cervesato iliano@cmu.edu

Indexing over Predicates Indexing over Terms Beyond Horn Clauses Conclusions Proof-Theoretic Foundations of Indexing in Logic Programming Iliano Cervesato iliano@cmu.edu Carnegie Mellon University Supported by grant NPRP 4-341-1-059, Usable

867 views • 55 slides
Index Construction Dictionary, postings, scalable indexing, dynamic indexing Web Search 1

Index Construction Dictionary, postings, scalable indexing, dynamic indexing Web Search 1

Index Construction Dictionary, postings, scalable indexing, dynamic indexing Web Search 1 Overview Indexes Query Indexi xing Ranki king Applica cation Results Documents User Information Query y Query analys ysis proce cess

611 views • 38 slides
Chapter 6 Hash-Based Indexing Efficient Support for Equality Search Hash-Based Indexing Static

Chapter 6 Hash-Based Indexing Efficient Support for Equality Search Hash-Based Indexing Static

Hash-Based Indexing Torsten Grust Chapter 6 Hash-Based Indexing Efficient Support for Equality Search Hash-Based Indexing Static Hashing Hash Functions Architecture and Implementation of Database Systems Extendible Hashing Summer 2016

930 views • 39 slides
Chapter V: Indexing &amp; Searching Information Retrieval &amp; Data Mining Universitt des

Chapter V: Indexing &amp; Searching Information Retrieval &amp; Data Mining Universitt des

Chapter V: Indexing &amp; Searching Information Retrieval &amp; Data Mining Universitt des Saarlandes, Saarbrcken Winter Semester 2011/12 Chapter V: Indexing &amp; Searching* V.1 Indexing &amp; Query processing Inverted indexes, B +

627 views • 29 slides
Lecture 19: Motion Sparse stereo matching Indexing scenes Indexing scenes Tuesday, Nov

Lecture 19: Motion Sparse stereo matching Indexing scenes Indexing scenes Tuesday, Nov

11/20/2007 Review Problem set 3 Dense stereo matching Lecture 19: Motion Sparse stereo matching Indexing scenes Indexing scenes Tuesday, Nov 20 Sources of error in Effect of window size correspondences Low-contrast /

459 views • 10 slides
Capabilities Capabilities Indexing and Publishing Indexing and Publishing Jason M. Coposky

Capabilities Capabilities Indexing and Publishing Indexing and Publishing Jason M. Coposky

Capabilities Capabilities Indexing and Publishing Indexing and Publishing Jason M. Coposky June 25-28, 2019 @jason_coposky iRODS User Group Meeting 2019 Executive Director, iRODS Consortium Utrecht, Netherlands 1 iRODS Capabilities

667 views • 23 slides
Indexing CS6320 1/29/2018 Shachi Deshpande, Yunhe Liu Content Motivation for Indexing

Indexing CS6320 1/29/2018 Shachi Deshpande, Yunhe Liu Content Motivation for Indexing

Indexing CS6320 1/29/2018 Shachi Deshpande, Yunhe Liu Content Motivation for Indexing B-tree B-tree basics The cost of B-tree operations B-tree variants B-tree in multi-user Environments Learned Index

1.98k views • 61 slides
Indexing and Searching Indexing and Searching TDT4215 TDT4215 Indexing &amp; Searching 3

Indexing and Searching Indexing and Searching TDT4215 TDT4215 Indexing &amp; Searching 3

1 Indexing Approaches Indexing Approaches R. Baeza-Yates and B. R. Baeza-Yates and B. Ribeiro-Neto Ribeiro-Neto: : Modern Informa Modern Information Retrie ion Retrieval, l, Chapter 8. Chapt Chapter Chapter 8 1999 . 1999 1999

371 views • 13 slides
INDEXING - 1 Tree-Structured Indices Tree-structured indexing techniques support both

INDEXING - 1 Tree-Structured Indices Tree-structured indexing techniques support both

INDEXING - 1 Tree-Structured Indices Tree-structured indexing techniques support both range searches and equality searches . ISAM : static structure; B+ tree : dynamic, adjusts gracefully under inserts and deletes. - 2 ISAM

206 views • 8 slides
Media Indexing &amp; Retrieval Media Indexing &amp; Retrieval Prepared by Ling Guan Jose Lay

Media Indexing &amp; Retrieval Media Indexing &amp; Retrieval Prepared by Ling Guan Jose Lay

20/09/2018 Media Indexing &amp; Retrieval Media Indexing &amp; Retrieval Prepared by Ling Guan Jose Lay Paisarn Muneesawang Ning Zhang Rui Zhang 1 Background More and more media are becoming available online. In the past decade, we

323 views • 20 slides
Indexing and Searching Indexing and Searching Berlin Chen 2005 References: 1. Modern

Indexing and Searching Indexing and Searching Berlin Chen 2005 References: 1. Modern

Indexing and Searching Indexing and Searching Berlin Chen 2005 References: 1. Modern Information Retrieval, chapter 8 2. Information Retrieval: Data Structures &amp; Algorithms, chapter 5 3. G.H. Gonnet, R.A. Baeza-Yates, T. Snider,

511 views • 37 slides
Indexing Extracts from: Witten, Moffat, and Bell, Managing Gigabytes , 2nd ed., Morgan

Indexing Extracts from: Witten, Moffat, and Bell, Managing Gigabytes , 2nd ed., Morgan

Indexing Extracts from: Witten, Moffat, and Bell, Managing Gigabytes , 2nd ed., Morgan Kaufmann, 1999. Melnik et al., Building a Distributed Full-Text Index for the Web . Proc. 10th Int. WWW Conf., 2001. 1 Indexing Documents Basic task:

233 views • 9 slides
Indexing December 12, 2008 Indexing Introduction New tuple is stored without any order next

Indexing December 12, 2008 Indexing Introduction New tuple is stored without any order next

Indexing December 12, 2008 Indexing Introduction New tuple is stored without any order next available space Indexing Introduction New tuple is stored without any order next available space Access will require inspection of every tuple

287 views • 13 slides
Biometric Indexing Yi Wang alice.yi.wang@ieee.org 13/Jan/2017 Outlines Introduction to

Biometric Indexing Yi Wang alice.yi.wang@ieee.org 13/Jan/2017 Outlines Introduction to

Biometric Indexing Yi Wang alice.yi.wang@ieee.org 13/Jan/2017 Outlines Introduction to biometric indexing Accuracy issues: Dealing with low quality query fingerprints Efficiency issues: Search and indexing fingerprints with

969 views • 72 slides
1 Basic Image/Video Features Image Features Color (a). SCD (scalable color descriptor) (b).

1 Basic Image/Video Features Image Features Color (a). SCD (scalable color descriptor) (b).

Problems in Video Indexing and Analysis EE 6882 Statistical Methods for Video Indexing, search, and retrieval for images and videos Image/Video search engine Indexing and Analysis find video clips of basketball going through the

600 views • 20 slides
Audio Indexing and Retrieval IT6902; Semester B, 2004/2005; Leung Audio Indexing and Retrieval

Audio Indexing and Retrieval IT6902; Semester B, 2004/2005; Leung Audio Indexing and Retrieval

Audio Indexing and Retrieval IT6902; Semester B, 2004/2005; Leung Audio Indexing and Retrieval Motivation Main Audio Features Audio Classification Speech Recognition Music Retrieval Using Audio Features for Video

590 views • 30 slides
CSE 132C Database System Implementation Arun Kumar Topic 2: Indexing Chapters 10 and 11 of

CSE 132C Database System Implementation Arun Kumar Topic 2: Indexing Chapters 10 and 11 of

CSE 132C Database System Implementation Arun Kumar Topic 2: Indexing Chapters 10 and 11 of Cow Book Slide ACKs: Jignesh Patel, Paris Koutris 1 Motivation for Indexing Consider the following SQL query: Movies (M) MovieID Name

667 views • 45 slides
Machine-Assisted Indexing Week 12 LBSC 671 Creating Information Infrastructures

Machine-Assisted Indexing Week 12 LBSC 671 Creating Information Infrastructures

Machine-Assisted Indexing Week 12 LBSC 671 Creating Information Infrastructures Machine-Assisted Indexing Goal: Automatically suggest descriptors Better consistency with lower cost Approach: Rule-based expert system Design

685 views • 50 slides
Log-structured Merge Tree (LSM) 1 Big Data Indexing We covered the two-layered global/local

Log-structured Merge Tree (LSM) 1 Big Data Indexing We covered the two-layered global/local

Log-structured Merge Tree (LSM) 1 Big Data Indexing We covered the two-layered global/local indexing scheme Ideal for static data Question: How to update these indexes? HDFS limitation: Random updates are not allowed Nave approach:

208 views • 7 slides
Re-indexing the DFT (n and k) We can investigate the various implementations of the DFT by

Re-indexing the DFT (n and k) We can investigate the various implementations of the DFT by

Re-indexing the DFT (n and k) We can investigate the various implementations of the DFT by looking at ways to re-index values of n and k. Idea: Using properties of addition and multiplication we can rewrite the equation. 1

541 views • 22 slides

More recommend