Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

Marcel Bollman; Anders Søgaard

Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

Marcel Bollman, Anders Søgaard

Department of Computer Science

19 Citations (Scopus)

59 Downloads (Pure)

Abstract

Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.

Original language	English
Title of host publication	The 26th International Conference on Computational Linguistics : proceedings of COLING 2016: technical Papers
Number of pages	9
Publication date	2016
Pages	131-139
ISBN (Electronic)	978-4-87974-702-0
Publication status	Published - 2016
Event	The 26th International Conference on Computational Linguistics - Osaka, Japan Duration: 11 Dec 2016 → 16 Dec 2016 Conference number: 26

Conference

Conference	The 26th International Conference on Computational Linguistics
Number	26
Country/Territory	Japan
City	Osaka
Period	11/12/2016 → 16/12/2016

Access to Document

Bollmann_2016_Improving_historicalFinal published version, 481 KBLicence: CC BY

http://coling2016.anlp.jp/doc/main.pdfLicence: CC BY

Cite this

Improving historical spelling normalization with bi-directional LSTMs and multi-task learning. / Bollman, Marcel; Søgaard, Anders.

The 26th International Conference on Computational Linguistics: proceedings of COLING 2016: technical Papers. 2016. p. 131-139.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Bollman, M & Søgaard, A 2016, Improving historical spelling normalization with bi-directional LSTMs and multi-task learning. in The 26th International Conference on Computational Linguistics: proceedings of COLING 2016: technical Papers. pp. 131-139, The 26th International Conference on Computational Linguistics , Osaka, Japan, 11/12/2016. <http://coling2016.anlp.jp/doc/main.pdf>

@inproceedings{d9aefd5378c6495982ca85951b377b32,

title = "Improving historical spelling normalization with bi-directional LSTMs and multi-task learning",

abstract = "Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.",

author = "Marcel Bollman and Anders S{\o}gaard",

year = "2016",

language = "English",

pages = "131--139",

booktitle = "The 26th International Conference on Computational Linguistics",

note = "The 26th International Conference on Computational Linguistics ; Conference date: 11-12-2016 Through 16-12-2016",

}

TY - GEN

T1 - Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

AU - Bollman, Marcel

AU - Søgaard, Anders

N1 - Conference code: 26

PY - 2016

Y1 - 2016

N2 - Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.

AB - Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.

M3 - Article in proceedings

SP - 131

EP - 139

BT - The 26th International Conference on Computational Linguistics

T2 - The 26th International Conference on Computational Linguistics

Y2 - 11 December 2016 through 16 December 2016

ER -

Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

Abstract

Conference

Access to Document

Fingerprint

Cite this