Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

TY - GEN

T1 - Improving historical spelling normalization with bi-directional LSTMs and multi-task learning

AU - Bollman, Marcel

AU - Søgaard, Anders

N1 - Conference code: 26

PY - 2016

Y1 - 2016

N2 - Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.

AB - Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data. A common approach is to normalize the spelling of historical words to modern forms. We explore the suitability of a deep neural network architecture for this task, particularly a deep bi-LSTM network applied on a character level. Our model compares well to previously established normalization algorithms when evaluated on a diverse set of texts from Early New High German. We show that multi-task learning with additional normalization data can improve our model's performance further.

M3 - Article in proceedings

SP - 131

EP - 139

BT - The 26th International Conference on Computational Linguistics

T2 - The 26th International Conference on Computational Linguistics

Y2 - 11 December 2016 through 16 December 2016

ER -