Lessons learned in multilingual grounded language learning

Ákos Kádár; Desmond Elliott; Marc-Alexandre Côté; Grzegorz Chrupala; Afra Alishahi

Lessons learned in multilingual grounded language learning

Ákos Kádár, Desmond Elliott, Marc-Alexandre Côté, Grzegorz Chrupala, Afra Alishahi

10 Citationer (Scopus)

Abstract

Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language. Here, we investigate in detail which conditions affect the performance of this type of grounded language learning model. We show that multilingual training improves over bilingual training, and that low-resource languages benefit from training with higher-resource languages. We demonstrate that a multilingual model can be trained equally well on either translations or comparable sentence pairs, and that annotating the same set of images in multiple language enables further improvements via an additional caption-caption ranking objective.

Originalsprog	Udefineret/Ukendt
Titel	Conference on Computational Natural Language Learning
Antal sider	11
Publikationsdato	2018
Sider	402-412
Status	Udgivet - 2018

Citationsformater

@inproceedings{937709f484784c409d1d5643104d03de,

title = "Lessons learned in multilingual grounded language learning",

abstract = "Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language. Here, we investigate in detail which conditions affect the performance of this type of grounded language learning model. We show that multilingual training improves over bilingual training, and that low-resource languages benefit from training with higher-resource languages. We demonstrate that a multilingual model can be trained equally well on either translations or comparable sentence pairs, and that annotating the same set of images in multiple language enables further improvements via an additional caption-caption ranking objective.",

author = "{\'A}kos K{\'a}d{\'a}r and Desmond Elliott and Marc-Alexandre C{\^o}t{\'e} and Grzegorz Chrupala and Afra Alishahi",

year = "2018",

language = "Udefineret/Ukendt",

pages = "402--412",

booktitle = "Conference on Computational Natural Language Learning",

}

TY - GEN

T1 - Lessons learned in multilingual grounded language learning

AU - Kádár, Ákos

AU - Elliott, Desmond

AU - Côté, Marc-Alexandre

AU - Chrupala, Grzegorz

AU - Alishahi, Afra

PY - 2018

Y1 - 2018

N2 - Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language. Here, we investigate in detail which conditions affect the performance of this type of grounded language learning model. We show that multilingual training improves over bilingual training, and that low-resource languages benefit from training with higher-resource languages. We demonstrate that a multilingual model can be trained equally well on either translations or comparable sentence pairs, and that annotating the same set of images in multiple language enables further improvements via an additional caption-caption ranking objective.

AB - Recent work has shown how to learn better visual-semantic embeddings by leveraging image descriptions in more than one language. Here, we investigate in detail which conditions affect the performance of this type of grounded language learning model. We show that multilingual training improves over bilingual training, and that low-resource languages benefit from training with higher-resource languages. We demonstrate that a multilingual model can be trained equally well on either translations or comparable sentence pairs, and that annotating the same set of images in multiple language enables further improvements via an additional caption-caption ranking objective.

M3 - Konferencebidrag i proceedings

SP - 402

EP - 412

BT - Conference on Computational Natural Language Learning

ER -