Sequence classification with human attention

Maria Jung Barrett; Joachim Bingel; Nora  Hollenstein; Marek Rei; Anders Søgaard

Sequence classification with human attention

Maria Jung Barrett, Joachim Bingel, Nora Hollenstein, Marek Rei, Anders Søgaard

Department of Nordic Studies and Linguistics

22 Citations (Scopus)

1 Downloads (Pure)

Abstract

Learning attention functions requires large
volumes of data, but many NLP tasks simulate
human behavior, and in this paper, we
show that human attention really does provide
a good inductive bias on many attention
functions in NLP. Specifically, we use
estimated human attention derived from eyetracking
corpora to regularize attention functions
in recurrent neural networks. We show
substantial improvements across a range of
tasks, including sentiment analysis, grammatical
error detection, and detection of abusive
language.

Original language	English
Title of host publication	Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018)
Editors	Anna Korhonen , Ivan Titov
Publisher	Association for Computational Linguistics
Publication date	2018
Pages	302–312
ISBN (Print)	978-1-948087-72-8
Publication status	Published - 2018
Event	22nd Conference on Computational Natural Language Learning (CoNLL 2018) - Brussels, Belgium Duration: 31 Oct 2018 → 1 Nov 2018

Conference

Conference	22nd Conference on Computational Natural Language Learning (CoNLL 2018)
Country/Territory	Belgium
City	Brussels
Period	31/10/2018 → 01/11/2018

Access to Document

K18-1030Final published version, 295 KB
Best_paper_award

Cite this

Sequence classification with human attention. / Barrett, Maria Jung; Bingel, Joachim; Hollenstein, Nora et al.

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018). ed. / Anna Korhonen ; Ivan Titov . Association for Computational Linguistics, 2018. p. 302–312.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Barrett, MJ, Bingel, J, Hollenstein, N, Rei, M & Søgaard, A 2018, Sequence classification with human attention. in A Korhonen & I Titov (eds), Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018). Association for Computational Linguistics, pp. 302–312, 22nd Conference on Computational Natural Language Learning (CoNLL 2018), Brussels, Belgium, 31/10/2018.

@inproceedings{f78fb48ee4a04cec9bf7355e0e958cfb,

title = "Sequence classification with human attention",

abstract = "Learning attention functions requires largevolumes of data, but many NLP tasks simulatehuman behavior, and in this paper, weshow that human attention really does providea good inductive bias on many attentionfunctions in NLP. Specifically, we useestimated human attention derived from eyetrackingcorpora to regularize attention functionsin recurrent neural networks. We showsubstantial improvements across a range oftasks, including sentiment analysis, grammaticalerror detection, and detection of abusivelanguage.",

author = "Barrett, {Maria Jung} and Joachim Bingel and Nora Hollenstein and Marek Rei and Anders S{\o}gaard",

year = "2018",

language = "English",

isbn = "978-1-948087-72-8",

pages = "302–312",

editor = "{Korhonen }, Anna and {Titov }, {Ivan }",

booktitle = "Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018)",

publisher = "Association for Computational Linguistics",

note = "22nd Conference on Computational Natural Language Learning (CoNLL 2018) ; Conference date: 31-10-2018 Through 01-11-2018",

}

TY - GEN

T1 - Sequence classification with human attention

AU - Barrett, Maria Jung

AU - Bingel, Joachim

AU - Hollenstein, Nora

AU - Rei, Marek

AU - Søgaard, Anders

PY - 2018

Y1 - 2018

N2 - Learning attention functions requires largevolumes of data, but many NLP tasks simulatehuman behavior, and in this paper, weshow that human attention really does providea good inductive bias on many attentionfunctions in NLP. Specifically, we useestimated human attention derived from eyetrackingcorpora to regularize attention functionsin recurrent neural networks. We showsubstantial improvements across a range oftasks, including sentiment analysis, grammaticalerror detection, and detection of abusivelanguage.

AB - Learning attention functions requires largevolumes of data, but many NLP tasks simulatehuman behavior, and in this paper, weshow that human attention really does providea good inductive bias on many attentionfunctions in NLP. Specifically, we useestimated human attention derived from eyetrackingcorpora to regularize attention functionsin recurrent neural networks. We showsubstantial improvements across a range oftasks, including sentiment analysis, grammaticalerror detection, and detection of abusivelanguage.

M3 - Article in proceedings

SN - 978-1-948087-72-8

SP - 302

EP - 312

BT - Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018)

A2 - Korhonen , Anna

A2 - Titov , Ivan

PB - Association for Computational Linguistics

T2 - 22nd Conference on Computational Natural Language Learning (CoNLL 2018)

Y2 - 31 October 2018 through 1 November 2018

ER -

Sequence classification with human attention

Abstract

Conference

Access to Document

Fingerprint

Cite this