Mixture of Training Data Xinyu Wang, Yong Jiang, Kewei Tu School of - PowerPoint PPT Presentation

Aug 19, 2023 •178 likes •340 views

Enhanced Universal Dependency Parsing with Second-Order Inference and Mixture of Training Data Xinyu Wang, Yong Jiang, Kewei Tu School of Information Science and Technology, ShanghaiTech University DAMO Academy, Alibaba Group Our Parser A

Enhanced Universal Dependency Parsing with Second-Order Inference and Mixture of Training Data Xinyu Wang, Yong Jiang, Kewei Tu School of Information Science and Technology, ShanghaiTech University DAMO Academy, Alibaba Group
Our Parser • A second-order semantic dependency parser based on Wang et al. (2019) • Equip the parser with state-of-the-art contextual multilingual embeddings: XLM-R (Conneau et al., 2019) • Improve the accuracy for the low-resource language (Tamil) through mixing the training set with another language (English/Czech) • Our Parser performs 0.6 ELAS better than the best parser in official results after fixing the graph connectivity issues [1]: Xinyu Wang, Jingxian Huang, and Kewei Tu. 2019. Second-order semantic dependency parsing with end-to-end neural networks. [2]: Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek , Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale.
Preprocessing: Empty Nodes
Preprocessing: Repeated Edges
Preprocessing • Tokenization: Stanza (Qi et al., 2020) • Multiple Treebanks: concatenate the datasets • Splitting the development sets into halves as validation and test sets [1]: Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D Manning. 2020. Stanza: A python natural language processing toolkit for many human languages.
Approach (Wang et al., 2019)
Mixture of Training Data For Tamil • Problem: low-resource • Only 400 training sentences for Tamil • Solution: utilizing rich-resource language corpus • Multilingual Embedding: XLM-R • Rich-Resource languages: English (12k sents) or Czech (100k sents) • Remove the label of dependency edges in rich-resource training data • New training data: Upsampled Tamil training data + rich-resource training data • Additional language-specific embeddings: Flair (Akbik et al., 2018) and fastText (Bojanowski et al., 2017) [1]: Alan Akbik, Duncan Blythe, and Roland Vollgraf.2018. Contextual string embeddings for sequence labeling. [2]: Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information
Graph Connection • Original submission: • Non-connected graphs (all potential edges with probability > 0.5) • New solution: • Tree algorithms: Maximum Spanning Tree (MST) or Eisner’s Algorithm • First use MST or Eisner’s algorithm to keep connectivity of graphs and then add potential edges with probabilities larger than 0.5
Results
Results
Mixture of Data Comparison
First-Order vs. Second-Order and Concatenating Other Embeddings *: We use labeled F1 score here, which is the metric for SDP
Comparisons of Graph Connection Approaches (Treebank Level)
Comparisons of Graph Connection Approaches (Language Level)
Thank you • Paper: https://arxiv.org/abs/2006.01414

Recommend

Classification of High Dimensional Data By Two-way Mixture Models Jia Li Statistics Department

Classification of High Dimensional Data By Two-way Mixture Models Jia Li Statistics Department The Pennsylvania State University 1 Outline Goals Two-way mixture model approach Background: mixture discriminant analysis Model

418 views • 23 slides

Mixture models of truncated data for estimating the number of species. Li-Thiao-T e S

Context Mixture models Estimation Application to Metagenomics Mixture models of truncated data for estimating the number of species. Li-Thiao-T e S ebastien , Jean-Jacques Daudin, St ephane Robin Equipe Statistique et G enome, UMR

146 views • 13 slides

Lecture 20 Lecture 20 Nov 12 th 2008 Clustering with Mixture of Gaussians Clustering with Mixture

Lecture 20 Lecture 20 Nov 12 th 2008 Clustering with Mixture of Gaussians Clustering with Mixture of Gaussians Underlying model for data: assumes a generative process: Each cluster follows a Gaussian distribution First choose which

302 views • 19 slides

Motivation We are given a data set, and are told that it was generated from a mixture of

Dirichlet Process Mixtures A gentle tutorial Graphical Models 10708 Khalid El-Arini Carnegie Mellon University November 6 th , 2006 1 Motivation We are given a data set, and are told that it was generated from a mixture of Gaussians.

503 views • 13 slides

Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The

DataCamp Mixture Models in R MIXTURE MODELS IN R Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The handwritten digits dataset DataCamp Mixture Models in R Continuous versus discrete variables

440 views • 41 slides

15-388/688 - Practical Data Science: Anomaly detection and mixture of Gaussians J. Zico Kolter

15-388/688 - Practical Data Science: Anomaly detection and mixture of Gaussians J. Zico Kolter Carnegie Mellon University Spring 2018 1 Outline Anomalies and outliers Multivariate Gaussian Mixture of Gaussians 2 Outline Anomalies and

511 views • 34 slides

Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative

Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative Sequence Training Chao Zhang and Phil Woodland March 8, 2017 Cambridge University Engineering Department Introduction Tandem Systems as Mixture

926 views • 17 slides

Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R

DataCamp Mixture Models in R MIXTURE MODELS IN R Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R Description of mixture models 1. Which is the suitable probability distribution? Get familiar with

1.07k views • 36 slides

Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine

Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine Learning Prof. Claudia Czado Ozge Sahin <ozge.sahin@tum.de> Bernoulli-IMS One World Symposium August 2020 Finite mixture models k

467 views • 14 slides

EM Algorithm and Mixture Models Guojun Zhang University of Waterloo Unsupervised learning and

EM Algorithm and Mixture Models Guojun Zhang University of Waterloo Unsupervised learning and clustering Learn the intrinsic representation of unlabeled data Other examples: density estimation, novelty detection Mixture model

218 views • 9 slides

Mixture Models Bhiksha Raj 27 Oct 2016 11755/18797 1 Learning Distributions for Data

Machine Learning for Signal Processing Expectation Maximization Mixture Models Bhiksha Raj 27 Oct 2016 11755/18797 1 Learning Distributions for Data Problem: Given a collection of examples from some data, estimate its distribution

1.05k views • 79 slides

A Nonparametric Finite Mixture Approach to Difference-in-Difference Estimation, with an

A Nonparametric Finite Mixture Approach to Difference-in-Difference Estimation, with an Application to Professional Training and Wages Oliver Cassagneau-Francis 1 Robert Gary-Bobo 2 Julie Pernaudet 3 Jean-Marc Robin 4 1 Sciences Po, 2 CREST,

832 views • 50 slides

Mixture Distribution and Its Applications on P&C Insurance Data Luyang Fu, Ph.D., FCAS, MAAA

Mixture Distribution and Its Applications on P&C Insurance Data Luyang Fu, Ph.D., FCAS, MAAA Doug Pirtle, FCAS May 2011 Auto Home Business STATEAUTO.COM Antitrust Notice The Casualty Actuarial Society is committed to adhering

600 views • 31 slides

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Mixture Models: definition Mixture models: Linear supper-position of mixtures or components | =

1.04k views • 39 slides

Mixture of Heavy-Tailed distributions for Bivariate Precipitation Data

Mixture of Heavy-Tailed distributions for Bivariate Precipitation Data julie.carreau@univ-montp2.fr & Philippe Naveau Julie Carreau & Malaak Kallache philippe.naveau@lsce.ipsl.fr mk@climpact.com HydroSciences Montpellier,

947 views • 58 slides

The Stata module CUB for fitting mixture models for ordinal data Christopher F. BAUM 1 , Giovanni

The statistical framework Empirical evidence The Stata module for CUB The Stata module CUB for fitting mixture models for ordinal data Christopher F. BAUM 1 , Giovanni CERULLI 2 , Francesca DI IORIO 3 , Domenico PICCOLO 3 , Rosaria SIMONE 3 1

739 views • 38 slides

Color orado A o ALTO P Project Data Training Data Training Objectives Understand the

Color orado A o ALTO P Project Data Training Data Training Objectives Understand the alarming statistics/background Discuss the population of interest and timeframe Review naming report conventions; drugs monitored in the program;

204 views • 17 slides

MIXTURE DENSITY NETWORKS MIXTURE DENSITY NETWORKS Charles Martin SO FAR; RNNS THAT MODEL

MIXTURE DENSITY NETWORKS MIXTURE DENSITY NETWORKS Charles Martin SO FAR; RNNS THAT MODEL CATEGORICAL DATA SO FAR; RNNS THAT MODEL CATEGORICAL DATA SO FAR; RNNS THAT MODEL CATEGORICAL DATA SO FAR; RNNS THAT MODEL CATEGORICAL DATA Remember that

1.26k views • 99 slides

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP Missing Data In classification we frequently see to classify objects using vectors of measured features. Sometimes these features are missing: [ 0

478 views • 10 slides

Solutions Unit 6 1 Solutions Homogenous Mixture (Solution) two or more substances mixed

Solutions Unit 6 1 Solutions Homogenous Mixture (Solution) two or more substances mixed together to have a uniform composition, its components are not distinguishable from one another Heterogenous Mixture Homogenous Mixture (Not a

523 views • 29 slides

CSC321 Lecture 18: Mixture Modeling Roger Grosse Roger Grosse CSC321 Lecture 18: Mixture

CSC321 Lecture 18: Mixture Modeling Roger Grosse Roger Grosse CSC321 Lecture 18: Mixture Modeling 1 / 27 Overview Some examples of situations where youd use unupservised learning You want to understand how a scientific field has changed

613 views • 27 slides

Mixture Selection, Mechanism Design, and Signaling Ho Yee Cheung Shaddin Dughmi Yu Cheng Ehsan

Mixture Selection, Mechanism Design, and Signaling Ho Yee Cheung Shaddin Dughmi Yu Cheng Ehsan Emamjomeh-Zadeh Li Han Shang-Hua Teng University of Southern California Yu Cheng (USC) 1 / 14 Mixture selection Mixture Selection Optimization

655 views • 40 slides

MLE part 2 18 / 70 Gaussian Mixture Model Suppose data is drawn from k Gaussians, meaning Y =

MLE part 2 18 / 70 Gaussian Mixture Model Suppose data is drawn from k Gaussians, meaning Y = j Discrete ( 1 , . . . , k ) , X = x | Y = j N ( j , j ) , and the parameters are = (( 1 , 1 , 1 ) , . . . , ( k ,

1.67k views • 138 slides

Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum

0. Co-Training Based on Combining Labeled and Unlabeled Data with Co-Training by A. Blum & T. Mitchell, 1998 1. Problem: Learning to classify data (ex: web pages) when the description of each example can be partitioned in 2

560 views • 9 slides