Learning Cross-lingual Distributed Logical Representations for - PowerPoint PPT Presentation

Learning Cross-lingual Distributed Logical Representations for Semantic Parsing Yanyan Zou and Wei Lu StatNLP Group Singapore University of Technology and Design July 18, 2018

Outline ✓ Background & Motivation ✓ Method ✓ Experiments & Analysis ✓ Conclusion 2

Background | Method | Experiments & Analysis | Conclusion Semantic Parsing Goal: Map natural languages into semantic representations. 3

Background | Method | Experiments & Analysis | Conclusion Semantic Parsing Goal: Map natural languages into semantic representations. 3 3

Background | Method | Experiments & Analysis | Conclusion Semantic Parsing Goal: Map natural languages into semantic representations. 3

Background | Method | Experiments & Analysis | Conclusion Joint Representations Proposed in previous works: ✓ Synchronous CFG derivation trees Wong and Mooney (2006, 2007) ✓ CCG derivation trees Zettlemoyer and Collins (2005, 2007) ✓ Bayesian tree transducers Jones, Goldwater and Johnson (2012) ✓ Hybrid Trees Lu, Ng, Lee, Zettlemoyer (2008) 4

(Lu et al. ， 2008) Background | Method | Experiments & Analysis | Conclusion Hybrid Tree Input: what states have no bordering states? 5

(Lu et al. ， 2008) Background | Method | Experiments & Analysis | Conclusion Hybrid Tree Input: what states have no bordering states? Output: 5

<latexit sha1_base64="4k51BASa7ECt1IbqM6i8+zbGU=">ACX3icbVHLSgMxFM2Mr9pWHXUlboJFaKGUGRF0IxTdFnBPqBTSibNtKFJZkgyQhnmJ90JbvwT0xe1jwuBc8+9+TmJIgZVdp1vy374PDo+CR3mi8Uz84vnMurtoSiUkLRyS3QApwqgLU01I91YEsQDRjrB5G1W73wSqWgkPvQ0Jn2ORoKGFCNtqIHzGZd9jvQ4CFOeVdQZBX4An2V8EG64saZT8U8wYiljSwtr7ura41Klu2VHG+oD5ySW3PnAXeBtwQlsIzmwPnyhxFOBEaM6RUz3Nj3U+R1BQzkuX9RJEY4QkakZ6BAnGi+uncnwzeG2YIw0iaIzScs/8nUsSVmvLAdM5WVNu1Gbmv1kt0+NxPqYgTQReXBQmDOoIzsyGQyoJ1mxqAMKSml0hHiOJsDZfkjcmeNtP3gXth5rn1rz3x1L9dWlHDtyCO1AGHngCdATdACGPxYtlWwitavfWKf286i1baWM9dgI+ybP2y+uIg=</latexit> <latexit sha1_base64="4k51BASa7ECt1IbqM6i8+zbGU=">ACX3icbVHLSgMxFM2Mr9pWHXUlboJFaKGUGRF0IxTdFnBPqBTSibNtKFJZkgyQhnmJ90JbvwT0xe1jwuBc8+9+TmJIgZVdp1vy374PDo+CR3mi8Uz84vnMurtoSiUkLRyS3QApwqgLU01I91YEsQDRjrB5G1W73wSqWgkPvQ0Jn2ORoKGFCNtqIHzGZd9jvQ4CFOeVdQZBX4An2V8EG64saZT8U8wYiljSwtr7ura41Klu2VHG+oD5ySW3PnAXeBtwQlsIzmwPnyhxFOBEaM6RUz3Nj3U+R1BQzkuX9RJEY4QkakZ6BAnGi+uncnwzeG2YIw0iaIzScs/8nUsSVmvLAdM5WVNu1Gbmv1kt0+NxPqYgTQReXBQmDOoIzsyGQyoJ1mxqAMKSml0hHiOJsDZfkjcmeNtP3gXth5rn1rz3x1L9dWlHDtyCO1AGHngCdATdACGPxYtlWwitavfWKf286i1baWM9dgI+ybP2y+uIg=</latexit> <latexit sha1_base64="4k51BASa7ECt1IbqM6i8+zbGU=">ACX3icbVHLSgMxFM2Mr9pWHXUlboJFaKGUGRF0IxTdFnBPqBTSibNtKFJZkgyQhnmJ90JbvwT0xe1jwuBc8+9+TmJIgZVdp1vy374PDo+CR3mi8Uz84vnMurtoSiUkLRyS3QApwqgLU01I91YEsQDRjrB5G1W73wSqWgkPvQ0Jn2ORoKGFCNtqIHzGZd9jvQ4CFOeVdQZBX4An2V8EG64saZT8U8wYiljSwtr7ura41Klu2VHG+oD5ySW3PnAXeBtwQlsIzmwPnyhxFOBEaM6RUz3Nj3U+R1BQzkuX9RJEY4QkakZ6BAnGi+uncnwzeG2YIw0iaIzScs/8nUsSVmvLAdM5WVNu1Gbmv1kt0+NxPqYgTQReXBQmDOoIzsyGQyoJ1mxqAMKSml0hHiOJsDZfkjcmeNtP3gXth5rn1rz3x1L9dWlHDtyCO1AGHngCdATdACGPxYtlWwitavfWKf286i1baWM9dgI+ybP2y+uIg=</latexit> <latexit sha1_base64="4k51BASa7ECt1IbqM6i8+zbGU=">ACX3icbVHLSgMxFM2Mr9pWHXUlboJFaKGUGRF0IxTdFnBPqBTSibNtKFJZkgyQhnmJ90JbvwT0xe1jwuBc8+9+TmJIgZVdp1vy374PDo+CR3mi8Uz84vnMurtoSiUkLRyS3QApwqgLU01I91YEsQDRjrB5G1W73wSqWgkPvQ0Jn2ORoKGFCNtqIHzGZd9jvQ4CFOeVdQZBX4An2V8EG64saZT8U8wYiljSwtr7ura41Klu2VHG+oD5ySW3PnAXeBtwQlsIzmwPnyhxFOBEaM6RUz3Nj3U+R1BQzkuX9RJEY4QkakZ6BAnGi+uncnwzeG2YIw0iaIzScs/8nUsSVmvLAdM5WVNu1Gbmv1kt0+NxPqYgTQReXBQmDOoIzsyGQyoJ1mxqAMKSml0hHiOJsDZfkjcmeNtP3gXth5rn1rz3x1L9dWlHDtyCO1AGHngCdATdACGPxYtlWwitavfWKf286i1baWM9dgI+ybP2y+uIg=</latexit> (Lu et al. ， 2008) Background | Method | Experiments & Analysis | Conclusion Generative Hybrid Tree Input: what states have no bordering states? X p ( m , n ) = p ( m , h , n ) h ∈ H ( n , m ) 6

<latexit sha1_base64="de/koeRKw20Rik/raIbKLUGRxQ=">ACX3icbVFdS8MwFE3rx+acWvVJfAkOYMxWhH0RB92eMEp8I6SpqlW1iSliQVRu2f9E3wxX9i2k02nRcC57z83NSZgwqrTrflj2xubWdqW6U9ut7+0fOIdHTypOJSZ9HLNYvoRIEUYF6WuqGXlJE8ZOQ5nN4X9edXIhWNxaOeJWTI0VjQiGKkDRU4r0nT50hPwij+dsPFHkL3kBfpTzIfrhJ7lNRJhixrJtnzWV3ezmjlerI9tL+er0wGm4HbcMuA68BWiARfQC590fxTjlRGjMkFIDz030MENSU8xIXvNTRKEp2hMBgYKxIkaZqU/OTw3zAhGsTRHaFiyq4oMcaVmPDSdxYrqb60g/6sNUh1dDzMqklQTgecXRSmDOoaF2XBEJcGazQxAWFKzK8QTJBHW5ktqxgTv75PXwdNFx3M73sNl4/ZuYUcVnIz0AQeuAK3oAt6oA8w+LRsa9eqW192xd63nXmrbS0x+BX2Cfg425KA=</latexit> <latexit sha1_base64="de/koeRKw20Rik/raIbKLUGRxQ=">ACX3icbVFdS8MwFE3rx+acWvVJfAkOYMxWhH0RB92eMEp8I6SpqlW1iSliQVRu2f9E3wxX9i2k02nRcC57z83NSZgwqrTrflj2xubWdqW6U9ut7+0fOIdHTypOJSZ9HLNYvoRIEUYF6WuqGXlJE8ZOQ5nN4X9edXIhWNxaOeJWTI0VjQiGKkDRU4r0nT50hPwij+dsPFHkL3kBfpTzIfrhJ7lNRJhixrJtnzWV3ezmjlerI9tL+er0wGm4HbcMuA68BWiARfQC590fxTjlRGjMkFIDz030MENSU8xIXvNTRKEp2hMBgYKxIkaZqU/OTw3zAhGsTRHaFiyq4oMcaVmPDSdxYrqb60g/6sNUh1dDzMqklQTgecXRSmDOoaF2XBEJcGazQxAWFKzK8QTJBHW5ktqxgTv75PXwdNFx3M73sNl4/ZuYUcVnIz0AQeuAK3oAt6oA8w+LRsa9eqW192xd63nXmrbS0x+BX2Cfg425KA=</latexit> <latexit sha1_base64="de/koeRKw20Rik/raIbKLUGRxQ=">ACX3icbVFdS8MwFE3rx+acWvVJfAkOYMxWhH0RB92eMEp8I6SpqlW1iSliQVRu2f9E3wxX9i2k02nRcC57z83NSZgwqrTrflj2xubWdqW6U9ut7+0fOIdHTypOJSZ9HLNYvoRIEUYF6WuqGXlJE8ZOQ5nN4X9edXIhWNxaOeJWTI0VjQiGKkDRU4r0nT50hPwij+dsPFHkL3kBfpTzIfrhJ7lNRJhixrJtnzWV3ezmjlerI9tL+er0wGm4HbcMuA68BWiARfQC590fxTjlRGjMkFIDz030MENSU8xIXvNTRKEp2hMBgYKxIkaZqU/OTw3zAhGsTRHaFiyq4oMcaVmPDSdxYrqb60g/6sNUh1dDzMqklQTgecXRSmDOoaF2XBEJcGazQxAWFKzK8QTJBHW5ktqxgTv75PXwdNFx3M73sNl4/ZuYUcVnIz0AQeuAK3oAt6oA8w+LRsa9eqW192xd63nXmrbS0x+BX2Cfg425KA=</latexit> <latexit sha1_base64="de/koeRKw20Rik/raIbKLUGRxQ=">ACX3icbVFdS8MwFE3rx+acWvVJfAkOYMxWhH0RB92eMEp8I6SpqlW1iSliQVRu2f9E3wxX9i2k02nRcC57z83NSZgwqrTrflj2xubWdqW6U9ut7+0fOIdHTypOJSZ9HLNYvoRIEUYF6WuqGXlJE8ZOQ5nN4X9edXIhWNxaOeJWTI0VjQiGKkDRU4r0nT50hPwij+dsPFHkL3kBfpTzIfrhJ7lNRJhixrJtnzWV3ezmjlerI9tL+er0wGm4HbcMuA68BWiARfQC590fxTjlRGjMkFIDz030MENSU8xIXvNTRKEp2hMBgYKxIkaZqU/OTw3zAhGsTRHaFiyq4oMcaVmPDSdxYrqb60g/6sNUh1dDzMqklQTgecXRSmDOoaF2XBEJcGazQxAWFKzK8QTJBHW5ktqxgTv75PXwdNFx3M73sNl4/ZuYUcVnIz0AQeuAK3oAt6oA8w+LRsa9eqW192xd63nXmrbS0x+BX2Cfg425KA=</latexit> (Lu ， 2014, 2015) Background | Method | Experiments & Analysis | Conclusion Discriminative Hybrid Tree Input: what states have no bordering states? X p ( m | n ) = p ( m , h | n ) h ∈ H ( n , m ) 7

Background | Method | Experiments & Analysis | Conclusion (Susanto and Lu, 2017) Neural Hybrid Tree Input: what states have no bordering states? • Neural hybrid tree is an extension of discriminative hybrid tree. 8

(Susanto and Lu, 2017) Background | Method | Experiments & Analysis | Conclusion Neural Hybrid Tree Input: what states have no bordering states? 8

Background | Method | Experiments & Analysis | Conclusion (Susanto and Lu, 2017) Neural Hybrid Tree Score vector Output layer Hidden layer Input layer Discriminative hybrid tree 9

Background | Method | Experiments & Analysis | Conclusion (Susanto and Lu, 2017) Neural Hybrid Tree Score vector Output layer Hidden layer Input layer Discriminative hybrid tree Word window in size of (2J+1) 9

Background | Method | Experiments & Analysis | Conclusion What do we have? English Sentences Semantic Parser For English Semantic Trees 10

Background | Method | Experiments & Analysis | Conclusion What do we have? English Sentences Semantic Parser For English Semantic Trees German Sentences Indonesian Sentences Chinese Sentences … 10

Background | Method | Experiments & Analysis | Conclusion What do we have? English Sentences Semantic Parser For English Semantic Trees German Sentences Can we leverage multi-lingual Indonesian Sentences resources to improve the performance of a monolingual Chinese Sentences semantic parser? … 10

Background | Method | Experiments & Analysis | Conclusion What do we have? English Sentences Semantic Parser For English Semantic Trees German Sentences Can we leverage multi-lingual Indonesian Sentences resources to improve the performance of a monolingual Chinese Sentences semantic parser? … The answer is Yes!!! 10

Background | Method | Experiments & Analysis | Conclusion Setup Target Language (E.g., English) Semantic Parser For English Semantic Trees Auxiliary Languages German Indonesian Chinese … 11

Background | Method | Experiments & Analysis | Conclusion Setup Target Language (E.g., English) Semantic Parser For English Semantic Trees Cross-lingual Auxiliary Languages information German Indonesian Chinese … 11

Background | Method | Experiments & Analysis | Conclusion Setups Target Language (E.g., English) Semantic Parser For English Semantic Trees We learn distributed Auxiliary Languages representations of semantic German units where such Indonesian cross-lingual information is Chinese captured. … 11

Learning Cross-lingual Distributed Logical Representations for - PowerPoint PPT Presentation

Learning Cross-lingual Distributed Logical Representations for Semantic Parsing Yanyan Zou and Wei Lu StatNLP Group Singapore University of Technology and Design July 18, 2018 Outline Background & Motivation Method Experiments

Cross-Lingual Part-of-Speech Tagging through Ambiguous Learning Guillaume Wisniewski Nicolas

Cross-lingual NLP Sara Stymne Uppsala University Department of Linguistics and Philology

Cross-lingual Information Retrieval Pavel Pecina Institute of Formal and Applied Linguistics

WMT 2016 Shared Task on Cross-lingual Pronoun Prediction . Liane Guillou, Christian Hardmeier,

Cr Cros oss-lin lingual al lan languag age mod model pr pretraini ning ng Alexis Conneau

Cross-lingual Predicate Cluster Acquisition to Improve Bilingual Event Extraction by Inductive

Cross-Lingual Cross-Document Coreference with Entity Linking Sean Monahan, John Lehmann, Timothy

Cross-Lingual Cross-Document Coreference with Entity Linking Sean Monahan, John Lehmann, Timothy

Bleaching Text: Abstract Features for Cross-lingual Gender Prediction. Rob van der Goot, Nikola

From Dictionaries to Cross-lingual Lexical Resources Guadalupe Aguado-de-Cea, Elena

Cross-lingual similarity calculation for plagiarism detection and more Tools and resources

Cross-lingual and temporal Wikipedia analysis G ob ol os-Szab o Julianna MTA SZTAKI

The Multilingual and Cross- lingual Web PD Dr. Gnter Neumann LT lab German Research Center

Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity

Building a Cross-lingual Relatedness Thesaurus using a Graph Similarity Measure Lukas

External and Intrinsic Plagiarism Detection using a Cross-Lingual Retrieval and Segmentation

Pronunciation Extraction Through Cross-Lingual Word-to-Phoneme Alignment Felix Stahlberg, Tim

A New Universal Morphological Feature Schema for Rich Morphological Annotation and Cross-Lingual

PD3: Better Cross-Lingual Transfer By Combining Direct Transfer and Annotation Projection Steffen

Cross-lingual Cold-Start Knowledge Base Construction M. Al-Badrashiny, J. Bolton5, A. T.

linking, cross-lingual entity linking) TAC 2011 Summarization Track Guided Summarization task

W ITH the widespread use of hands-free electronic gad- are mapped to a multilingual set using a

Cross-lingual Dependency Parsing of Related Languages with Rich Morphosyntactic Tagsets eljko

Automatic Machine Translation Evaluation using Source Language Inputs and Cross-lingual Language