Learning when to point: A data-driven approach

Albert Gatt; Patrizia Paggio

Learning when to point: A data-driven approach

Centre for Language Technology

5 Citations (Scopus)

Abstract

The relationship between how people describe objects and when they choose to point is complex and likely to be influenced by factors related to both perceptual and discourse context. In this paper, we explore these interactions using machine-learning on a dialogue corpus, to identify multimodal referential strategies that can be used in automatic multimodal generation. We show that the decision to use a pointing gesture depends on features of the accompanying description (especially whether it contains spatial information), and on visual properties, especially distance or separation of a referent from its previous referent.

Original language	English
Title of host publication	Proceedings of the 25th International Conference on Computational Linguistics (COLING '14)
Number of pages	10
Place of Publication	Dublin, Ireland
Publisher	Association for Computational Linguistics
Publication date	2014
Pages	2007-2017
Publication status	Published - 2014
Event	Coling 2014 - Dublin, Ireland Duration: 23 Aug 2014 → 29 Aug 2014

Conference

Conference	Coling 2014
Country/Territory	Ireland
City	Dublin
Period	23/08/2014 → 29/08/2014

Cite this

Learning when to point: A data-driven approach. / Gatt, Albert; Paggio, Patrizia.

Proceedings of the 25th International Conference on Computational Linguistics (COLING '14). Dublin, Ireland : Association for Computational Linguistics, 2014. p. 2007-2017.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

@inproceedings{a4cb3002e22644848750a6067dfd1cdb,

title = "Learning when to point: A data-driven approach",

abstract = "The relationship between how people describe objects and when they choose to point is complex and likely to be influenced by factors related to both perceptual and discourse context. In this paper, we explore these interactions using machine-learning on a dialogue corpus, to identify multimodal referential strategies that can be used in automatic multimodal generation. We show that the decision to use a pointing gesture depends on features of the accompanying description (especially whether it contains spatial information), and on visual properties, especially distance or separation of a referent from its previous referent.",

author = "Albert Gatt and Patrizia Paggio",

year = "2014",

language = "English",

pages = "2007--2017",

booktitle = "Proceedings of the 25th International Conference on Computational Linguistics (COLING '14)",

publisher = "Association for Computational Linguistics",

note = "Coling 2014 ; Conference date: 23-08-2014 Through 29-08-2014",

}

TY - GEN

T1 - Learning when to point: A data-driven approach

AU - Gatt, Albert

AU - Paggio, Patrizia

PY - 2014

Y1 - 2014

N2 - The relationship between how people describe objects and when they choose to point is complex and likely to be influenced by factors related to both perceptual and discourse context. In this paper, we explore these interactions using machine-learning on a dialogue corpus, to identify multimodal referential strategies that can be used in automatic multimodal generation. We show that the decision to use a pointing gesture depends on features of the accompanying description (especially whether it contains spatial information), and on visual properties, especially distance or separation of a referent from its previous referent.

AB - The relationship between how people describe objects and when they choose to point is complex and likely to be influenced by factors related to both perceptual and discourse context. In this paper, we explore these interactions using machine-learning on a dialogue corpus, to identify multimodal referential strategies that can be used in automatic multimodal generation. We show that the decision to use a pointing gesture depends on features of the accompanying description (especially whether it contains spatial information), and on visual properties, especially distance or separation of a referent from its previous referent.

M3 - Article in proceedings

SP - 2007

EP - 2017

BT - Proceedings of the 25th International Conference on Computational Linguistics (COLING '14)

PB - Association for Computational Linguistics

CY - Dublin, Ireland

T2 - Coling 2014

Y2 - 23 August 2014 through 29 August 2014

ER -

Learning when to point: A data-driven approach

Abstract

Conference

Fingerprint

Cite this