a latent variable model of synchronous parsing for
play

A Latent Variable Model of Synchronous Parsing for Syntactic and - PowerPoint PPT Presentation

Sep 19, 2022 •112 likes •596 views

A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies James Henderson 1 Paola Merlo 2 Gabriele Musillo 1 2

  1. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies James Henderson 1 Paola Merlo 2 Gabriele Musillo 1 2 Ivan Titov 3 1 Dept Computer Science, Univ Geneva 2 Dept Linguistics, Univ Geneva 3 Dept Computer Science, Univ Illinois at U-C CoNLL 2008 university-logo

  2. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Outline A Latent Variable Model of Synchronous Parsing 1 Probability Model 2 Machine Learning Method 3 Evaluation 4 university-logo

  3. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Motivation for synchronous parsing Syntax and semantics are separate structures , with different generalisations Obj Sub John broke the vase. A0 A1 Sub The vase broke. A1 Syntax and semantics are highly correlated , and therefore should be learned jointly Synchronous parsing provides a single joint model of two separate structures university-logo

  4. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Motivation for latent variables The correlations between syntax and semantics are partly lexical , and independence assumptions are hard to specify a priori The dataset is new, and there was little time for feature engineering Latent variables provide a powerful mechanism for discovering correlations both within and between the structures university-logo

  5. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Outline A Latent Variable Model of Synchronous Parsing 1 Probability Model 2 Machine Learning Method 3 Evaluation 4 university-logo

  6. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Outline A Latent Variable Model of Synchronous Parsing 1 Probability Model 2 Machine Learning Method 3 Evaluation 4 university-logo

  7. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation The Probability Model A generative, history-based model of the joint probability of syntactic and semantic synchronous derivations synchronised at each word . university-logo

  8. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Syntactic and semantic dependencies example ROOT Hope seems doomed to failure P ( T d , T s ) university-logo

  9. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Syntactic and semantic derivations Define two separate derivations , one for the syntactic structure and one for the semantic structure. P ( T d , T s ) = P ( D 1 d , ..., D m d d , D 1 s , ..., D m s s ) Actions of an incremental shift-reduce style parser similar to MALT [Nivre et al., 2006] Semantic derivations are less constrained, because their structures are less constrained Assumes each dependency structure is individually planar (“projective”) university-logo

  10. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Synchronisation granularity Use an intermediate synchronisation granularity, between full predications and individual actions. b t e t d , shift t , D b t s , ..., D e t C t = D d , ..., D d d s s , shift t s d , ..., D m d P ( D 1 d , D 1 s , ..., D m s s ) = P ( C 1 , . . . , C n ) Synchronisation at each word prediction Results in one shared input queue Allows two separate stacks university-logo

  11. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Synchronous parsing example ROOT Hope P ( C 1 ) university-logo

  12. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Synchronous parsing example ROOT Hope seems P ( C 1 ) P ( C 2 | C 1 ) university-logo

  13. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Synchronous parsing example ROOT Hope seems doomed P ( C 1 ) P ( C 2 | C 1 ) P ( C 3 | C 1 , C 2 ) university-logo

  14. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Synchronous parsing example ROOT Hope seems doomed to P ( C 1 ) P ( C 2 | C 1 ) P ( C 3 | C 1 , C 2 ) P ( C 4 | C 1 , C 2 , C 3 ) university-logo

  15. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Synchronous parsing example ROOT Hope seems doomed to failure P ( C 1 ) P ( C 2 | C 1 ) P ( C 3 | C 1 , C 2 ) P ( C 4 | C 1 , C 2 , C 3 ) P ( C 5 | C 1 , C 2 , C 3 , C 4 ) university-logo

  16. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope university-logo

  17. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems university-logo

  18. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems university-logo

  19. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems university-logo

  20. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems university-logo

  21. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed university-logo

  22. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed university-logo

  23. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed university-logo

  24. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed university-logo

  25. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed to university-logo

  26. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed to university-logo

  27. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed to university-logo

  28. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed to failure university-logo

  29. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed to failure university-logo

  30. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Derivation example ROOT Hope seems doomed to failure university-logo

  31. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Projectivisation Allows crossing links between syntax and semantics Use the HEAD method [Nivre et al., 2006] to projectivise syntax Use syntactic dependencies to projectivise semantic dependencies university-logo

  32. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Projectivising semantic dependencies C w1 w2 w3 w4 w5 A B C w1 w2 w3 w4 w5 B A/C university-logo

  33. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Outline A Latent Variable Model of Synchronous Parsing 1 Probability Model 2 Machine Learning Method 3 Evaluation 4 university-logo

  34. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation The Machine Learning Method Synchronous derivations are modeled with an Incremental Sigmoid Belief Network ( ISBN ). ISBNs are Dynamic Bayesian Networks for modeling structures , with vectors of latent variables annotating derivation states that represent features of the derivation history . Use the neural network approximation of ISBNs [Titov and Henderson, ACL 2007] (“Simple Synchrony Netowrks”) university-logo

  35. A Latent Variable Model of Synchronous Parsing Probability Model Machine Learning Method Evaluation Statistical dependencies in the ISBN Connections between latent states reflect locality in the syntactic or semantic structure , thereby specifying the domain of locality for conditioning decisions Explicit conditioning features of the history are also specified t−c t t−1 S S S t−c t−1 t D D D university-logo

Recommend

Paraphrase Generation from Latent-Variable PCFGs for Semantic Parsing Shashi Narayan, Siva Reddy,

Paraphrase Generation from Latent-Variable PCFGs for Semantic Parsing Shashi Narayan, Siva Reddy,

Paraphrase Generation from Latent-Variable PCFGs for Semantic Parsing Shashi Narayan, Siva Reddy, Shay B. Cohen School of Informatics, University of Edinburgh INLG, September 2016 1 / 51 Semantic Parsing for Question Answering Semantically

939 views • 74 slides
1 Latent variable models In the next section we will discuss latent variable models for

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides
Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models

Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models

2/4/2020 Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models Expectation Maximization Algorithm (EM) Factor Analysis Probabilistic Principal Component Analysis Model Formulation Maximum

655 views • 18 slides
Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

2/4/2020 CS 3750 Advanced Machine Learning Latent Variable Generative Models II Ahmad Diab AHD23@cs.pitt.edu Feb 4, 2020 Based on slides of Professor Milos Hauskrecht Outline Latent Variable Generative Models Cooperative Vector

435 views • 20 slides
Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330 Student Presentation Table of Contents Motivation & problem Method overview Experiments Takeaways Discussion (strengths

653 views • 24 slides
Mokken Scale Analysis Alternative names: Unidimensional Latent Variable Model (e.g., Holland &

Mokken Scale Analysis Alternative names: Unidimensional Latent Variable Model (e.g., Holland &

Monotone Homogeneity Model (MHM) Notation: X 1 , ..., X j , ..., X J : item scores; : latent trait MHM (Mokken, 1971) : General IRT model for P( X j = x |) Mokken Scale Analysis Alternative names: Unidimensional Latent Variable Model

485 views • 4 slides
Bayesian Latent Variable Modelling of Longitudinal Family Data for Genetic Pleiotropy Studies

Bayesian Latent Variable Modelling of Longitudinal Family Data for Genetic Pleiotropy Studies

Pleiotropy Latent Variable Model Bayesian Model Simulations Bayesian Latent Variable Modelling of Longitudinal Family Data for Genetic Pleiotropy Studies Radu Craiu Department of Statistics University of Toronto Joint with Lei Sun and

782 views • 38 slides
Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges in Unsupervised Learning Learn a latent variable model without labeled examples. E.g. topic models, hidden Markov models, Gaussian mixtures,

763 views • 61 slides
An Efficient Posterior Regularized Latent Variable Model for Interactive Sound Source

An Efficient Posterior Regularized Latent Variable Model for Interactive Sound Source

An Efficient Posterior Regularized Latent Variable Model for Interactive Sound Source Separation Nicholas J. Bryan, Stanford University Gautham J. Mysore, Adobe Research ICML 2013 Sound Check 1 Motivation I Real world

465 views • 21 slides
Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine Joint work with Majid Janzamin Rong Ge UC Irvine Microsoft Research Latent Variable Probabilistic Models Latent (hidden) variable h R k ,

769 views • 47 slides
Model selection and estimation for latent variable models Presented by Emi Tanaka School of

Model selection and estimation for latent variable models Presented by Emi Tanaka School of

Model selection and estimation for latent variable models Presented by Emi Tanaka School of Mathematics and Statisitcs dr.emi.tanaka@gmail.com @statsgen Download pdf of these slides June 26th 2019 @ EcoSta2019 1 / 18 here. Motivation Many

894 views • 18 slides
A Discriminative Latent Variable Model for Online Clustering Rajhans Samdani, Kai-Wei Chang , Dan

A Discriminative Latent Variable Model for Online Clustering Rajhans Samdani, Kai-Wei Chang , Dan

A Discriminative Latent Variable Model for Online Clustering Rajhans Samdani, Kai-Wei Chang , Dan Roth Department of Computer Science University of Illinois at Urbana-Champaign Motivating Example: Coreference n Coreference resolution: cluster

510 views • 22 slides
FARMS: a probabilistic latent variable model for summarizing Affymetrix array data at probe level

FARMS: a probabilistic latent variable model for summarizing Affymetrix array data at probe level

FARMS: a probabilistic latent variable model for summarizing Affymetrix array data at probe level Djork-Arn Clevert, Sepp Hochreiter Institute of Bioinformatics, Johannes Kepler University Linz Willem Talloen, An De Bond, Hinrich Ghlmann

470 views • 25 slides
MLG Spotlight Talks August 20th, 2018 Growing Better Graphs with Latent-Variable Probabilistic

MLG Spotlight Talks August 20th, 2018 Growing Better Graphs with Latent-Variable Probabilistic

MLG Spotlight Talks August 20th, 2018 Growing Better Graphs with Latent-Variable Probabilistic Graph Xinyi Wang, Salvador Aguinaga, Tim Weninger, David Chiang Background and Problems Solution: latent variable HRG Hyperedge Replacement

475 views • 21 slides
Pengtao Xie Joint work with Yuntian Deng and Eric Xing Carnegie Mellon University 1 Latent

Pengtao Xie Joint work with Yuntian Deng and Eric Xing Carnegie Mellon University 1 Latent

Mutual Angular Regularization of Latent Variable Models: Theory, Algorithm and Applications Pengtao Xie Joint work with Yuntian Deng and Eric Xing Carnegie Mellon University 1 Latent Variable Models (LVMs) Machine Learning Latent Variable

1.04k views • 47 slides
Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine Joint work with Anima Anandkumar Rong Ge UC Irvine Microsoft Research Latent Variable Modeling Goal: Discover hidden effects from observed

1.27k views • 115 slides
Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent

Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent

Spectral Algorithms for Latent Variable Models Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent Variable Models, Edinburgh, UK Joint work with Mariya Ishteva, Ankur Parikh, Eric Xing, Byron Boots , Geoff

548 views • 40 slides
Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models Yarin Gal Mark van der Wilk Carl E. Rasmussen yg279@cam.ac.uk June 25th, 2014 Outline Gaussian process regression and latent variable

1.18k views • 54 slides
Maximum Reconstruction Estimation for Generative Latent-Variable Models Yong Cheng joint work

Maximum Reconstruction Estimation for Generative Latent-Variable Models Yong Cheng joint work

Maximum Reconstruction Estimation for Generative Latent-Variable Models Yong Cheng joint work with Yang Liu, Wei Xu 1 Problem Generative latent-variable models are important for natural language processing due to their capability of providing

494 views • 26 slides
Relational Deep Learning: A Deep Latent Variable Model for Link Prediction Hao Wang, Xingjian

Relational Deep Learning: A Deep Latent Variable Model for Link Prediction Hao Wang, Xingjian

Relational Deep Learning: A Deep Latent Variable Model for Link Prediction Hao Wang, Xingjian Shi, Dit-Yan Yeung Motivation Bayesian Deep Learning Relational Deep Learning Parameter Learning Experiments Conclusion

577 views • 40 slides
Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of Maryland furongh@cs.umd.edu ACM SIGMETRICS Tutorial 2018 1 / 75 Tutorial Topic Learning algorithms for latent variable models based on

1.96k views • 180 slides
Latent Variable models for GWAs Oliver Stegle Machine Learning and Computational Biology Research

Latent Variable models for GWAs Oliver Stegle Machine Learning and Computational Biology Research

Latent Variable models for GWAs Oliver Stegle Machine Learning and Computational Biology Research Group Max-Planck-Institutes T ubingen, Germany September 2011 O. Stegle Latent variable models for GWAs T ubingen 1 Motivation Why

1.07k views • 84 slides
Discriminative Regularization for Latent Variable Models with Applications to Electrocardiography

Discriminative Regularization for Latent Variable Models with Applications to Electrocardiography

Discriminative Regularization for Latent Variable Models with Applications to Electrocardiography Poster #53 Andrew C. Miller with Ziad Obermeyer, John P . Cunningham, and Sendhil Mullainathan <latexit

674 views • 9 slides
Rao-Blackwellized Stochastic Gradients for Discrete Distributions Runjing (Bryan) Liu June 11,

Rao-Blackwellized Stochastic Gradients for Discrete Distributions Runjing (Bryan) Liu June 11,

Rao-Blackwellized Stochastic Gradients for Discrete Distributions Runjing (Bryan) Liu June 11, 2019 University of California, Berkeley Objective We fit a discrete latent variable model . Objective We fit a discrete latent variable

525 views • 24 slides

More recommend