Collecting Aligned Textual Corpora from the Hidden Web Botjan - PowerPoint PPT Presentation

Aug 04, 2023 •346 likes •424 views

Collecting Aligned Textual Corpora from the Hidden Web Botjan Pajntar bostjan.pajntar@ijs.si ailab.ijs.si Aligned Parallel Corpus Definition (wikipedia): A parallel text is a text placed alongside its translation or translations

Collecting Aligned Textual Corpora from the Hidden Web Boštjan Pajntar bostjan.pajntar@ijs.si ailab.ijs.si
Aligned Parallel Corpus Definition (wikipedia): “ A parallel text is a text placed alongside its translation or translations ” Usage: Translation Memory Machine Translation Natural Language Processing Standards: TMX – Translation Memory eXchange TBX – TermBase eXchange UTX – Universal Terminology eXchange (SRX, GMX-GILT, OLIF, XLIFF, TransWS, ...) ailab.ijs.si
But Where to Get the Data? Non-English professional websites Huge amounts of translated text Generally quality translations We call this the Hidden Web ailab.ijs.si
Problems Translation Memory is hard / expensive to obtain Idea: Automatic harnessing of existing data Data should have very high precision What precision is needed? No standard fully supports automatic: Harnessing of the data Cleaning of the data ailab.ijs.si
Proposed Solution Parallel Corpora Extraction Candidates Filtering Parsing Crawling Extraction Relational List of HTML List of text Parallel Database candidates candidates Corpora WEB Available at : http://kameleon.ijs.si/t4me ailab.ijs.si
Discussion on Standards We build on TMX: Is this the right choice? Source language must be defined! An optional parameter to define the source of each segment Proposals for automatic harnessing of TM: Provide a new standard Build on an existing one Ideas? ailab.ijs.si
Future Work Optimizing Crawling: Two phase crawling Character Encodings Enhanced candidates extraction Optimizing Extraction: Segmentation Language identification Enhanced filtering Web service / Web application Translation Memory distribution Filtering (Web 2.0 style) ailab.ijs.si

Recommend

Collecting and evaluating speech recognition corpora for nine Southern Bantu languages Jaco

Collecting and evaluating speech recognition corpora for nine Southern Bantu languages Jaco Badenhorst, Charl van Heerden, Marelie Davel and Etienne Barnard March 31, 2009 Introduction ASR corpus design Project Lwazi Computational analysis

400 views • 20 slides

Textual Criticism Textual Criticism: Definition Textual criticism is the study of copies of

Textual Criticism Textual Criticism: Definition Textual criticism is the study of copies of any written work of which the autograph (the original) is unknown, with the purpose of ascertaining the original text. J. Harold Greenlee, New

921 views • 60 slides

Towards an Intuitive Sign Language Animation Authoring Environment For the Deaf Alexis Heloir,

Towards an Intuitive Sign Language Animation Authoring Environment For the Deaf Alexis Heloir, Fabrizio Nunnari UVHC, LAMIH - Valenciennes, France DFKI-MMCI, SLSI group - Germany NLP: Parallel Corpora Aligned annotated videos Aligned Glosses

665 views • 18 slides

Morphology and Corpora: Introduction Marco Baroni University of Bologna Granada Morphology

Morphology and Corpora: Introduction Marco Baroni University of Bologna Granada Morphology and Corpora Seminar Outline Corpora General overview Data sparseness and the need for larger corpora Morphology Derivational vs. inflectional

746 views • 63 slides

Natural logic and textual inference Bill MacCartney CS224U 12 May 2014 Textual inference

Natural logic and textual inference Bill MacCartney CS224U 12 May 2014 Textual inference examples P. A Revenue Cutter, the ship was named for Harriet Lane, niece of President James Buchanan, who served as Buchanans White House hostess.

920 views • 62 slides

Design and Realization of the EXCITEMENT Open Platform for Textual Entailment Gnter Neumann,

Design and Realization of the EXCITEMENT Open Platform for Textual Entailment Gnter Neumann, DFKI Sebastian Pado, Universitt Stuttgart Textual Entailment Textual Entailment (TE) A Text (T) entails a Hypothesis (H), if a typical

320 views • 27 slides

Collecting Social Photography Connect to collect, the social dimension of collecting. Bente

Collecting Social Photography Connect to collect, the social dimension of collecting. Bente Jensen , Elisabeth Boogh, Kajsa Hartig and Anni Wallenius - Documenting social processes and movements. 2019, Riksarkivet, Oslo Outline of presentation

482 views • 34 slides

Simulation of Field-Aligned Simulation of Field-Aligned Ideal MHD Flows Around deal MHD Flows

11 th INTERNATIONAL CONFERENCE ON HYPERBOLIC PROBLEMS: THEORY, NUMERICS, APPLICATIONS Simulation of Field-Aligned Simulation of Field-Aligned Ideal MHD Flows Around deal MHD Flows Around Perfectly Conducting Cylinders Using An Artificial

543 views • 37 slides

Dialogue corpora NPFL070 December 11, 2019 (NPFL070) Dialogue corpora December 11, 2019 1 /

Dialogue corpora NPFL070 December 11, 2019 (NPFL070) Dialogue corpora December 11, 2019 1 / 26 Outline 1 Intro 2 Task oriented 3 Chit-chat 4 QA (NPFL070) Dialogue corpora December 11, 2019 2 / 26 What is dialogue Sample conversation

699 views • 26 slides

Collecting Engineering Data Three ways of collecting data on the impacts of factors on a response

ST 370 Probability and Statistics for Engineers Collecting Engineering Data Three ways of collecting data on the impacts of factors on a response in a system: Retrospective Study: Collect relevant data from historical records; Observational

340 views • 12 slides

East Slavic parallel corpora: diachronic and diatopic variaton in Belarusian, Ukrainian, and

East Slavic parallel corpora: diachronic and diatopic variaton in Belarusian, Ukrainian, and Russian Dmitri Sitchinava mitrius@gmail.com Bilingual corpora Bilingual parallel corpora contrastve linguistcs, small typology (English

527 views • 35 slides

Dynamic Embedding on Textual Networks via a Gaussian Process Presenter : Pengyu Cheng Joint work

Dynamic Embedding on Textual Networks via a Gaussian Process Presenter : Pengyu Cheng Joint work with : Yitong Li, Xinyuan Zhang, Liqun Chen, David Carlson, Lawrence Carin Duke University Textual Networks Networks with textual information as

526 views • 15 slides

Collecting Data: New Information Sources November 2019 Outlines Collecting Data Legal

13 TH Session of the Committee of Director-Generals CoDGs of the National Statistics Offices High Level Meeting on Data Governance Tunis, 22 November 2019 Collecting Data: New Information Sources November 2019 Outlines Collecting Data

84 views • 7 slides

Towards Continuous Qvality Control for Spoken Language Corpora Anne Ferger and Hanna Hedeland

Towards Continuous Qvality Control for Spoken Language Corpora Anne Ferger and Hanna Hedeland University of Hamburg INEL HZSK Grammatical Descriptions, Corpora and Language Technology Hamburg Center for Language Corpora for I ndigenous N

488 views • 25 slides

Hidden$Terminals$ Nodes$A$and$C$are$hidden$terminals$when$sending$to$B$

Hidden$Terminals$ Nodes$A$and$C$are$hidden$terminals$when$sending$to$B$ Cant$hear$each$other$(to$coordinate)$yet$collide$at$B$ We$want$to$avoid$the$inefficiency$of$collisions $ CSE$461$University$of$Washington$ 53$

463 views • 31 slides

Textual Entailment Alina Petrova EMCL TUD, HLT FBK February 22, 2012 Alina Petrova EMCL TUD,

Textual Entailment Alina Petrova EMCL TUD, HLT FBK February 22, 2012 Alina Petrova EMCL TUD, HLT FBK Textual Entailment Introduction Textual Entailment (TE): What is it? a notion from classical logic is applied to natural language

676 views • 23 slides

S Sanctification Under the OT tifi ti U d th OT Theocracy eoc acy Charles Clough What

S Sanctification Under the OT tifi ti U d th OT Theocracy eoc acy Charles Clough What Methodology? gy OT OT NT NT textual data textual data textual data Retro projection?? Or Go With the Flow? Position Definition Proponents

470 views • 20 slides

Byte-Aligned Codes Indexing, session 6 CS6200: Information Retrieval Slides by: Jesse Anderton

Byte-Aligned Codes Indexing, session 6 CS6200: Information Retrieval Slides by: Jesse Anderton Byte-Aligned Codes Weve looked at ways to encode integers with bit-aligned codes. These are very compact, but somewhat inconvenient. Processors

300 views • 7 slides

Adversarial Robustness for Aligned AI Ian Goodfellow, Sta ff Research NIPS 2017 Workshop on

Adversarial Robustness for Aligned AI Ian Goodfellow, Sta ff Research NIPS 2017 Workshop on Aligned Artificial Intelligence Many thanks to Catherine Olsson for feedback on drafts The Alignment Problem (This is now fixed. Dont try it!)

626 views • 30 slides

Hidden Markov Models Pratik Lahiri Introduction A hidden Markov model (HMM) is a

Hidden Markov Models Pratik Lahiri Introduction A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states. We call the observed event

741 views • 13 slides

Textual, executable, translatable UML Gergely Dvai, Gbor Ferenc Kovcs, dm Ancsin

Textual, executable, translatable UML Gergely Dvai, Gbor Ferenc Kovcs, dm Ancsin Etvs Lornd University, Budapest, Hungary OCL and textual modeling workshop, 2014-09-30, Valencia Executable UML All aspects of the software, from

481 views • 12 slides

Textual Analytics for Accounting and Auditing Thanks to Ingrid Fisher (SUNY Albany)

Textual Analytics for Accounting and Auditing Thanks to Ingrid Fisher (SUNY Albany) Research interests - textual analysis in accounting, design science in accounting and financial accounting standards/documents Jordan Seebach

968 views • 72 slides

An integrated framework in R for textual sentiment time series aggregation and prediction Ardia,

An integrated framework in R for textual sentiment time series aggregation and prediction Ardia, D. , Bluteau, K., Borms, S . and Boudt, K. (2017). The R Package sentometrics to Compute, Aggregate and Predict with Textual Sentiment. Available

778 views • 15 slides

STANDARDS IN SPOKEN CORPORA OUTLINE (1) Case study: Spoken

7/21/15 Thomas Schmidt, Programmbereich Mndliche Korpora STANDARDS IN SPOKEN CORPORA OUTLINE (1) Case study: Spoken corpora at the SFB 538 (2)

351 views • 22 slides