Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens

Marek Rei; Anders Søgaard

Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens

16 Citationer (Scopus)

Abstract

Can attention-or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels? We construct a neural network architecture based on soft attention, train it as a binary sentence classifier and evaluate against tokenlevel annotation on four different datasets. Inferring token labels from a network provides a method for quantitatively evaluating what the model is learning, along with generating useful feedback in assistance systems. Our results indicate that attention-based methods are able to predict token-level labels more accurately, compared to gradient-based methods, sometimes even rivaling the supervised oracle network.

Originalsprog	Engelsk
Titel	Proceedings, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies : (Long Ppaers)
Vol/bind	1
Forlag	Association for Computational Linguistics
Publikationsdato	2018
Sider	293–302
Status	Udgivet - 2018
Begivenhed	16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - New Orleans, USA Varighed: 1 jun. 2018 → 6 jun. 2018

Konference

Konference	16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Land/Område	USA
By	New Orleans
Periode	01/06/2018 → 06/06/2018

Citationsformater

Zero-shot Sequence Labeling : Transferring Knowledge from Sentences to Tokens. / Rei, Marek; Søgaard, Anders.

Proceedings, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies : (Long Ppaers). Bind 1 Association for Computational Linguistics, 2018. s. 293–302.

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

Rei, M & Søgaard, A 2018, Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens. i Proceedings, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies : (Long Ppaers). bind 1, Association for Computational Linguistics, s. 293–302, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, USA, 01/06/2018.

@inproceedings{b9ea626955e6424ab7abe7ed350417e2,

title = "Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens",

abstract = "Can attention-or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels? We construct a neural network architecture based on soft attention, train it as a binary sentence classifier and evaluate against tokenlevel annotation on four different datasets. Inferring token labels from a network provides a method for quantitatively evaluating what the model is learning, along with generating useful feedback in assistance systems. Our results indicate that attention-based methods are able to predict token-level labels more accurately, compared to gradient-based methods, sometimes even rivaling the supervised oracle network.",

author = "Marek Rei and Anders S{\o}gaard",

year = "2018",

language = "English",

volume = "1",

pages = "293–302",

booktitle = "Proceedings, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",

publisher = "Association for Computational Linguistics",

note = "16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2018 ; Conference date: 01-06-2018 Through 06-06-2018",

}

TY - GEN

T1 - Zero-shot Sequence Labeling

T2 - 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

AU - Rei, Marek

AU - Søgaard, Anders

PY - 2018

Y1 - 2018

N2 - Can attention-or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels? We construct a neural network architecture based on soft attention, train it as a binary sentence classifier and evaluate against tokenlevel annotation on four different datasets. Inferring token labels from a network provides a method for quantitatively evaluating what the model is learning, along with generating useful feedback in assistance systems. Our results indicate that attention-based methods are able to predict token-level labels more accurately, compared to gradient-based methods, sometimes even rivaling the supervised oracle network.

AB - Can attention-or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels? We construct a neural network architecture based on soft attention, train it as a binary sentence classifier and evaluate against tokenlevel annotation on four different datasets. Inferring token labels from a network provides a method for quantitatively evaluating what the model is learning, along with generating useful feedback in assistance systems. Our results indicate that attention-based methods are able to predict token-level labels more accurately, compared to gradient-based methods, sometimes even rivaling the supervised oracle network.

M3 - Article in proceedings

VL - 1

SP - 293

EP - 302

BT - Proceedings, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

PB - Association for Computational Linguistics

Y2 - 1 June 2018 through 6 June 2018

ER -

Zero-shot Sequence Labeling: Transferring Knowledge from Sentences to Tokens

Abstract

Konference

Fingeraftryk

Citationsformater