Predicting word sense annotation agreement

Hector Martinez Alonso; Anders Trærup Johannsen; Oier Lopez de Lacalle; Eneko Agirre

Predicting word sense annotation agreement

Hector Martinez Alonso, Anders Trærup Johannsen, Oier Lopez de Lacalle, Eneko Agirre

Centre for Language Technology

Abstract

High agreement is a common objective when annotating data for word senses. However, a number of factors make perfect agreement impossible, e.g. the limitations of sense inventories, the difficulty of the examples or the interpretation preferences of the annotators. Estimating potential agreement is thus a relevant task to supplement the evaluation of sense annotations. In this article we propose two methods to predict agreement on word-annotation instances. We experiment with a continuous representation and a three-way discretization of observed agreement. In spite of the difficulty of the task, we find that different levels of agreement can be identified-in particular, low-agreement examples are easier to identify.

Original language	English
Title of host publication	LSDSem 2015 : Linking Models of Lexical, Sentential and Discourse-level Semantics
Number of pages	6
Publisher	Association for Computational Linguistics
Publication date	2015
Pages	89-94
ISBN (Print)	978-1-941643-32-7
Publication status	Published - 2015

Access to Document

http://aclweb.org/anthology/W15-27

Cite this

Predicting word sense annotation agreement. / Martinez Alonso, Hector; Johannsen, Anders Trærup; Lopez de Lacalle, Oier et al.

LSDSem 2015 : Linking Models of Lexical, Sentential and Discourse-level Semantics. Association for Computational Linguistics, 2015. p. 89-94.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

@inproceedings{15637f3bea2840eab89246c4abb5bfd8,

title = "Predicting word sense annotation agreement",

abstract = "High agreement is a common objective when annotating data for word senses. However, a number of factors make perfect agreement impossible, e.g. the limitations of sense inventories, the difficulty of the examples or the interpretation preferences of the annotators. Estimating potential agreement is thus a relevant task to supplement the evaluation of sense annotations. In this article we propose two methods to predict agreement on word-annotation instances. We experiment with a continuous representation and a three-way discretization of observed agreement. In spite of the difficulty of the task, we find that different levels of agreement can be identified-in particular, low-agreement examples are easier to identify.",

author = "{Martinez Alonso}, Hector and Johannsen, {Anders Tr{\ae}rup} and {Lopez de Lacalle}, Oier and Eneko Agirre",

year = "2015",

language = "English",

isbn = "978-1-941643-32-7",

pages = "89--94",

booktitle = "LSDSem 2015",

publisher = "Association for Computational Linguistics",

}

TY - GEN

T1 - Predicting word sense annotation agreement

AU - Martinez Alonso, Hector

AU - Johannsen, Anders Trærup

AU - Lopez de Lacalle, Oier

AU - Agirre, Eneko

PY - 2015

Y1 - 2015

N2 - High agreement is a common objective when annotating data for word senses. However, a number of factors make perfect agreement impossible, e.g. the limitations of sense inventories, the difficulty of the examples or the interpretation preferences of the annotators. Estimating potential agreement is thus a relevant task to supplement the evaluation of sense annotations. In this article we propose two methods to predict agreement on word-annotation instances. We experiment with a continuous representation and a three-way discretization of observed agreement. In spite of the difficulty of the task, we find that different levels of agreement can be identified-in particular, low-agreement examples are easier to identify.

AB - High agreement is a common objective when annotating data for word senses. However, a number of factors make perfect agreement impossible, e.g. the limitations of sense inventories, the difficulty of the examples or the interpretation preferences of the annotators. Estimating potential agreement is thus a relevant task to supplement the evaluation of sense annotations. In this article we propose two methods to predict agreement on word-annotation instances. We experiment with a continuous representation and a three-way discretization of observed agreement. In spite of the difficulty of the task, we find that different levels of agreement can be identified-in particular, low-agreement examples are easier to identify.

M3 - Article in proceedings

SN - 978-1-941643-32-7

SP - 89

EP - 94

BT - LSDSem 2015

PB - Association for Computational Linguistics

ER -

Predicting word sense annotation agreement

Abstract

Access to Document

Fingerprint

Cite this