Multi-task learning for historical text normalization: Size matters

Marc Marcel Bollmann; Anders Søgaard; Joachim Bingel

Multi-task learning for historical text normalization: Size matters

Marc Marcel Bollmann, Anders Søgaard, Joachim Bingel

Abstract

Historical text normalization suffers fromsmall datasets that exhibit high variance,and previous work has shown that multitasklearning can be used to leverage datafrom related problems in order to obtainmore robust models. Previous work hasbeen limited to datasets from a specific languageand a specific historical period, andit is not clear whether results generalize. Ittherefore remains an open problem, whenhistorical text normalization benefits frommulti-task learning. We explore the benefitsof multi-task learning across 10 differentdatasets, representing different languagesand periods. Our main finding—contrary to what has been observed forother NLP tasks—is that multi-task learningmainly works when target task data isvery scarce.

Originalsprog	Engelsk
Titel	Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP
Forlag	Association for Computational Linguistics
Publikationsdato	2018
Sider	19–24
Status	Udgivet - 2018
Begivenhed	Workshop on Deep Learning Approaches for Low-Resource NLP - Melbourne, Australien Varighed: 19 jul. 2018 → 19 jul. 2018

Workshop

Workshop	Workshop on Deep Learning Approaches for Low-Resource NLP
Land/Område	Australien
By	Melbourne
Periode	19/07/2018 → 19/07/2018

Citationsformater

Multi-task learning for historical text normalization: Size matters. / Bollmann, Marc Marcel; Søgaard, Anders; Bingel, Joachim.
Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP. Association for Computational Linguistics, 2018. s. 19–24.

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

@inproceedings{e0c40b1c1f9d44279d28902cceaad388,

title = "Multi-task learning for historical text normalization: Size matters",

abstract = "Historical text normalization suffers fromsmall datasets that exhibit high variance,and previous work has shown that multitasklearning can be used to leverage datafrom related problems in order to obtainmore robust models. Previous work hasbeen limited to datasets from a specific languageand a specific historical period, andit is not clear whether results generalize. Ittherefore remains an open problem, whenhistorical text normalization benefits frommulti-task learning. We explore the benefitsof multi-task learning across 10 differentdatasets, representing different languagesand periods. Our main finding—contrary to what has been observed forother NLP tasks—is that multi-task learningmainly works when target task data isvery scarce.",

author = "Bollmann, {Marc Marcel} and Anders S{\o}gaard and Joachim Bingel",

year = "2018",

language = "English",

pages = "19–24",

booktitle = "Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP",

publisher = "Association for Computational Linguistics",

note = "Workshop on Deep Learning Approaches for Low-Resource NLP ; Conference date: 19-07-2018 Through 19-07-2018",

}

TY - GEN

T1 - Multi-task learning for historical text normalization

T2 - Workshop on Deep Learning Approaches for Low-Resource NLP

AU - Bollmann, Marc Marcel

AU - Søgaard, Anders

AU - Bingel, Joachim

PY - 2018

Y1 - 2018

N2 - Historical text normalization suffers fromsmall datasets that exhibit high variance,and previous work has shown that multitasklearning can be used to leverage datafrom related problems in order to obtainmore robust models. Previous work hasbeen limited to datasets from a specific languageand a specific historical period, andit is not clear whether results generalize. Ittherefore remains an open problem, whenhistorical text normalization benefits frommulti-task learning. We explore the benefitsof multi-task learning across 10 differentdatasets, representing different languagesand periods. Our main finding—contrary to what has been observed forother NLP tasks—is that multi-task learningmainly works when target task data isvery scarce.

AB - Historical text normalization suffers fromsmall datasets that exhibit high variance,and previous work has shown that multitasklearning can be used to leverage datafrom related problems in order to obtainmore robust models. Previous work hasbeen limited to datasets from a specific languageand a specific historical period, andit is not clear whether results generalize. Ittherefore remains an open problem, whenhistorical text normalization benefits frommulti-task learning. We explore the benefitsof multi-task learning across 10 differentdatasets, representing different languagesand periods. Our main finding—contrary to what has been observed forother NLP tasks—is that multi-task learningmainly works when target task data isvery scarce.

M3 - Article in proceedings

SP - 19

EP - 24

BT - Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP

PB - Association for Computational Linguistics

Y2 - 19 July 2018 through 19 July 2018

ER -