Sequence classification with human attention

Maria Jung Barrett; Joachim Bingel; Nora  Hollenstein; Marek Rei; Anders Søgaard

Sequence classification with human attention

Maria Jung Barrett, Joachim Bingel, Nora Hollenstein, Marek Rei, Anders Søgaard

Institut for Nordiske Studier og Sprogvidenskab

22 Citationer (Scopus)

1 Downloads (Pure)

Abstract

Learning attention functions requires large
volumes of data, but many NLP tasks simulate
human behavior, and in this paper, we
show that human attention really does provide
a good inductive bias on many attention
functions in NLP. Specifically, we use
estimated human attention derived from eyetracking
corpora to regularize attention functions
in recurrent neural networks. We show
substantial improvements across a range of
tasks, including sentiment analysis, grammatical
error detection, and detection of abusive
language.

Originalsprog	Engelsk
Titel	Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018)
Redaktører	Anna Korhonen , Ivan Titov
Forlag	Association for Computational Linguistics
Publikationsdato	2018
Sider	302–312
ISBN (Trykt)	978-1-948087-72-8
Status	Udgivet - 2018
Begivenhed	22nd Conference on Computational Natural Language Learning (CoNLL 2018) - Brussels, Belgien Varighed: 31 okt. 2018 → 1 nov. 2018

Konference

Konference	22nd Conference on Computational Natural Language Learning (CoNLL 2018)
Land/Område	Belgien
By	Brussels
Periode	31/10/2018 → 01/11/2018

Adgang til dokumentet

K18-1030Forlagets udgivne version, 295 KB
Best_paper_award

Citationsformater

Sequence classification with human attention. / Barrett, Maria Jung; Bingel, Joachim; Hollenstein, Nora et al.

Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018). red. / Anna Korhonen ; Ivan Titov . Association for Computational Linguistics, 2018. s. 302–312.

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

Barrett, MJ, Bingel, J, Hollenstein, N, Rei, M & Søgaard, A 2018, Sequence classification with human attention. i A Korhonen & I Titov (red), Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018). Association for Computational Linguistics, s. 302–312, 22nd Conference on Computational Natural Language Learning (CoNLL 2018), Brussels, Belgien, 31/10/2018.

@inproceedings{f78fb48ee4a04cec9bf7355e0e958cfb,

title = "Sequence classification with human attention",

abstract = "Learning attention functions requires largevolumes of data, but many NLP tasks simulatehuman behavior, and in this paper, weshow that human attention really does providea good inductive bias on many attentionfunctions in NLP. Specifically, we useestimated human attention derived from eyetrackingcorpora to regularize attention functionsin recurrent neural networks. We showsubstantial improvements across a range oftasks, including sentiment analysis, grammaticalerror detection, and detection of abusivelanguage.",

author = "Barrett, {Maria Jung} and Joachim Bingel and Nora Hollenstein and Marek Rei and Anders S{\o}gaard",

year = "2018",

language = "English",

isbn = "978-1-948087-72-8",

pages = "302–312",

editor = "{Korhonen }, Anna and {Titov }, {Ivan }",

booktitle = "Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018)",

publisher = "Association for Computational Linguistics",

note = "22nd Conference on Computational Natural Language Learning (CoNLL 2018) ; Conference date: 31-10-2018 Through 01-11-2018",

}

TY - GEN

T1 - Sequence classification with human attention

AU - Barrett, Maria Jung

AU - Bingel, Joachim

AU - Hollenstein, Nora

AU - Rei, Marek

AU - Søgaard, Anders

PY - 2018

Y1 - 2018

N2 - Learning attention functions requires largevolumes of data, but many NLP tasks simulatehuman behavior, and in this paper, weshow that human attention really does providea good inductive bias on many attentionfunctions in NLP. Specifically, we useestimated human attention derived from eyetrackingcorpora to regularize attention functionsin recurrent neural networks. We showsubstantial improvements across a range oftasks, including sentiment analysis, grammaticalerror detection, and detection of abusivelanguage.

AB - Learning attention functions requires largevolumes of data, but many NLP tasks simulatehuman behavior, and in this paper, weshow that human attention really does providea good inductive bias on many attentionfunctions in NLP. Specifically, we useestimated human attention derived from eyetrackingcorpora to regularize attention functionsin recurrent neural networks. We showsubstantial improvements across a range oftasks, including sentiment analysis, grammaticalerror detection, and detection of abusivelanguage.

M3 - Article in proceedings

SN - 978-1-948087-72-8

SP - 302

EP - 312

BT - Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018)

A2 - Korhonen , Anna

A2 - Titov , Ivan

PB - Association for Computational Linguistics

T2 - 22nd Conference on Computational Natural Language Learning (CoNLL 2018)

Y2 - 31 October 2018 through 1 November 2018

ER -

Sequence classification with human attention

Abstract

Konference

Adgang til dokumentet

Fingeraftryk

Citationsformater