Learning when to point: A data-driven approach

TY - GEN

T1 - Learning when to point: A data-driven approach

AU - Gatt, Albert

AU - Paggio, Patrizia

PY - 2014

Y1 - 2014

N2 - The relationship between how people describe objects and when they choose to point is complex and likely to be influenced by factors related to both perceptual and discourse context. In this paper, we explore these interactions using machine-learning on a dialogue corpus, to identify multimodal referential strategies that can be used in automatic multimodal generation. We show that the decision to use a pointing gesture depends on features of the accompanying description (especially whether it contains spatial information), and on visual properties, especially distance or separation of a referent from its previous referent.

AB - The relationship between how people describe objects and when they choose to point is complex and likely to be influenced by factors related to both perceptual and discourse context. In this paper, we explore these interactions using machine-learning on a dialogue corpus, to identify multimodal referential strategies that can be used in automatic multimodal generation. We show that the decision to use a pointing gesture depends on features of the accompanying description (especially whether it contains spatial information), and on visual properties, especially distance or separation of a referent from its previous referent.

M3 - Article in proceedings

SP - 2007

EP - 2017

BT - Proceedings of the 25th International Conference on Computational Linguistics (COLING '14)

PB - Association for Computational Linguistics

CY - Dublin, Ireland

T2 - Coling 2014

Y2 - 23 August 2014 through 29 August 2014

ER -