References Towards finer-grained tagging of discourse connectives Yannick Versley SFB 833 / Univ. Tübingen AG Beyond Semantics, DGfS 2011 Yannick Versley Towards finer-grained tagging of discourse connectives
References Example ◮ I regret this , but the vote has already been taken and [ ❛r❣✶ the decision is made ] [ ❝♦♥♥ so] [ ❛r❣✷ let us leave the matter there ] . Ich bedauere das , aber die Abstimmung ist durchgeführt worden , [ ❛r❣✶ die Entscheidung ist gefallen ] , [ ❝♦♥♥ also] [ ❛r❣✷ lassen wir die Dinge ] . [’Contingency.Cause.Result’] Yannick Versley Towards finer-grained tagging of discourse connectives
References Finding discourse relations Discourse connectives ( explicit relations ) ◮ co-/subordinating conjunctions ( but , after ) ◮ discourse adverbials ( however , then ) ◮ Discontinuous units ( if . . . then , either . . . or ) ◮ (Some) can be modified 3 weeks after he left, mainly because she likes strawberries Syntactic structure ◮ purpose clauses ◮ embedded V1 Unmarked relations ( implicit relations ) Yannick Versley Towards finer-grained tagging of discourse connectives
References Explicit vs. Implicit ◮ Many connectives are unambiguous (e.g. because , although ) ◮ About 46% of discourse relations are implicit ◮ Low performance on implicit relations (Pitler et al., 2009; Lin et al., 2009) ◮ Unsupervised learning on unambiguous connectives does not help classification of implicits (Marcu and Echihabi, 2002; Sporleder and Lascarides, 2008) ⇒ What about ambiguous connectives? Yannick Versley Towards finer-grained tagging of discourse connectives
References Previous work ◮ Use connectives, syntactic configuration to predict lower levels of the discourse structure (Soricut and Marcu, 2003). ◮ Some connectives are ambiguous, but tense is a good disambiguator (Miltsakaki et al., 2005). ◮ Some connectives are ambiguous, but shallow features help (Haddow, 2005). ◮ Most connectives are not ambiguous, and syntax/structure helps with the rest (Pitler and Nenkova, 2009). Yannick Versley Towards finer-grained tagging of discourse connectives
References Corpora with Discourse Structure RST ◮ Relations between arbitrary segments ◮ Relation inventory: 119 Relations (core inventory of ≈ 20) PDTB I ◮ Only explicit relations Inter-sentential links only for discourse adverbials ◮ No relation labels PDTB II ◮ Taxonomic schema of discourse relations Level 1: Contingency, Comparison, Expansion, Temporal Level 2: 12 relations (Contrast/Concession, Cause/Consequence) Level 3: ≈ 30 relations ◮ implicit relations, AltLex , NoRel , EntRel Yannick Versley Towards finer-grained tagging of discourse connectives
References Experiment on ambiguous connectives Take explicit relations, with features: ◮ The connective ◮ Connective at start of sentence? ◮ Structural information (Pitler and Nenkova, 2009) ◮ parent, sibling nodes with semantic function labels (-TMP, -PRP) ◮ VP in right sibling ◮ trace in right sibling ◮ Tense information on Arg2 (Miltsakaki et al., 2005) ◮ be -form ◮ have -form ◮ main verb form Yannick Versley Towards finer-grained tagging of discourse connectives
References Problematic connectives (1): coarse c+syn B c+syn A connective frequency conn c+first c+verb(arg1) since 154 0.571 0.571 0.675 0.935 0.909 finally 30 0.633 0.933 0.867 0.867 0.933 in turn 27 0.704 0.704 0.704 0.704 0.704 even as 11 0.727 0.636 0.364 0.455 0.636 while 652 0.729 0.727 0.729 0.839 0.805 as 588 0.786 0.786 0.781 0.810 0.781 as long as 20 0.800 0.786 0.750 0.700 0.750 Yannick Versley Towards finer-grained tagging of discourse connectives
References Problematic connectives (2): fine c+syn B c+syn A c+syn B +arg1 connective freq conn c+first c+arg1 in fact 70 0.300 0.386 0.286 0.300 0.343 0.429 although 277 0.498 0.588 0.520 0.549 0.592 0.606 still 156 0.500 0.429 0.462 0.506 0.417 0.449 since 154 0.571 0.571 0.669 0.929 0.903 0.896 though 187 0.588 0.652 0.540 0.551 0.652 0.652 while 652 0.598 0.598 0.604 0.718 0.667 0.672 indeed 86 0.605 0.593 0.593 0.593 0.570 0.558 when 837 0.611 0.608 0.609 0.609 0.596 0.588 yet 88 0.648 0.648 0.523 0.432 0.523 0.545 as 588 0.745 0.745 0.736 0.767 0.743 0.745 but 2767 0.790 0.790 0.789 0.788 0.789 0.785 meanwhile 160 0.800 0.800 0.800 0.775 0.794 0.794 Yannick Versley Towards finer-grained tagging of discourse connectives
References Is there a problem? ◮ Many connectives are unambiguous on the coarse level, but ambiguous at the finer levels ◮ The features that automatic parsing does not give us (semantic function labels, traces) would be useful ◮ Tense information helps (for some connectives) Can we improve the classification part? Yannick Versley Towards finer-grained tagging of discourse connectives
References Hierarchical classification Idea: use information on different levels of the taxonomy Flat classification: ◮ Only consider lowest level of the taxonomy Greedy classification: ◮ First classify coarse relation ◮ Then classify finer relations Hierarchical classification ◮ Used by Ciaramita et al. (2003) for WSD Coarse level: supersenses Fine level: actual synsets for words ◮ Here: relations on different levels (Contingency, Contingency.cause, C.cause.result) Yannick Versley Towards finer-grained tagging of discourse connectives
References How well does it work? (1) flat conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.954 0.954 0.954 0.953 0.952 0.952 d=2 0.847 0.847 0.845 0.845 d=3 0.796 0.798 hierarchical conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.954 0.953 0.954 0.953 0.952 0.952 d=2 0.847 0.847 0.845 0.845 d=3 0.796 0.798 Yannick Versley Towards finer-grained tagging of discourse connectives
References How well does it work? (2) flat conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.954 0.954 0.954 0.953 0.952 0.952 d=2 0.847 0.847 0.845 0.845 d=3 0.796 0.798 greedy conn+syntax conn+verb(arg1) evaluated 1 2 3 1 2 3 d=1 0.955 0.954 0.955 0.953 0.953 0.953 d=2 0.847 0.847 0.845 0.845 d=3 0.798 0.800 Yannick Versley Towards finer-grained tagging of discourse connectives
References Better features? An Apples-vs-Oranges comparison: while während always Contrast 0.662 0.711 tense+. . . 0.619 0.622 +wordpairs+productions 0.638 0.784 (Accuracy on first relation) Yannick Versley Towards finer-grained tagging of discourse connectives
References Summary ◮ More connectives ambiguous on lower taxonomy levels ◮ Tense and syntactic structure address different problems ◮ Reconstruct semantic functions, traces? ◮ Limited improvements from fancier classification Yannick Versley Towards finer-grained tagging of discourse connectives
References Thanks for listening! THE END Yannick Versley Towards finer-grained tagging of discourse connectives
References Ciaramita, M., Hofmann, T., and Johnson, M. (2003). Hierarchical semantic classification: Word sense disambiguation with world knowledge. In 18th International Joint Conference on Artificial Intelligence (IJCAI 2003) . Haddow, B. (2005). Acquiring a disambiguation model for discourse connectives. Master’s thesis, School of Informatics, University of Edinburgh. Lin, Z., Kan, M.-Y., and Ng, H. T. (2009). Recognizing implicit discourse relations in the Penn Discourse Treebank. In EMNLP 2009 . Marcu, D. and Echihabi, A. (2002). An unsupervised approach to recognizing discourse relations. In ACL 2002 . Miltsakaki, E., Dinesh, N., Prasad, R., Joshi, A., and Webber, B. (2005). Experiments on sense annotations and sense disambiguation of discourse connectives. In TLT 2005 . Yannick Versley Towards finer-grained tagging of discourse connectives
References Pitler, E., Louis, A., and Nenkova, A. (2009). Automatic sense prediction for implicit discourse relations in text. In ACL-IJCNLP 2009 . Pitler, E. and Nenkova, A. (2009). Using syntax to disambiguate explicit discourse connectives in text. In ACL 2009 short papers . Soricut, R. and Marcu, D. (2003). Sentence level discourse parsing using syntactic and lexical information. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference (HLT/NAACL-2003) . Sporleder, C. and Lascarides, A. (2008). Using automatically labelled examples to classify rhetorical relations: An assessment. Natural Language Engineering , 14(3):369–416. Yannick Versley Towards finer-grained tagging of discourse connectives
Recommend
More recommend
