Mobile Screen Size Limits Multimodal Synergy

Frans van der Sluis; Egon L. van den Broek; Annemiek van Drunen; John G. Beerends

doi:10.1145/3232078.3232101

Mobile Screen Size Limits Multimodal Synergy

Frans van der Sluis, Egon L. van den Broek, Annemiek van Drunen, John G. Beerends

1 Citation (Scopus)

Abstract

Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

Original language	English
Title of host publication	ECCE'18 : Proceedings of the 36th European Conference on Cognitive Ergonomics
Number of pages	4
Volume	36
Place of Publication	New York, NY, USA
Publisher	Association for Computing Machinery, Inc
Publication date	5 Sept 2018
Article number	16
ISBN (Electronic)	978-1-4503-6449-2
DOIs	https://doi.org/10.1145/3232078.3232101
Publication status	Published - 5 Sept 2018
Event	European Conference on Cognitive Ergonomics - Utrecht, Netherlands Duration: 5 Sept 2018 → 7 Sept 2018 Conference number: 36

Conference

Conference	European Conference on Cognitive Ergonomics
Number	36
Country/Territory	Netherlands
City	Utrecht
Period	05/09/2018 → 07/09/2018

Access to Document

10.1145/3232078.3232101

Cite this

Mobile Screen Size Limits Multimodal Synergy. / van der Sluis, Frans; van den Broek, Egon L.; van Drunen, Annemiek et al.
ECCE'18: Proceedings of the 36th European Conference on Cognitive Ergonomics . Vol. 36 New York, NY, USA: Association for Computing Machinery, Inc, 2018. 16.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

van der Sluis, F, van den Broek, EL, van Drunen, A & Beerends, JG 2018, Mobile Screen Size Limits Multimodal Synergy. in ECCE'18: Proceedings of the 36th European Conference on Cognitive Ergonomics . vol. 36, 16, Association for Computing Machinery, Inc, New York, NY, USA, European Conference on Cognitive Ergonomics, Utrecht, Netherlands, 05/09/2018. https://doi.org/10.1145/3232078.3232101

@inproceedings{802dbc9bed7f424690343624157f3dc7,

title = "Mobile Screen Size Limits Multimodal Synergy",

abstract = "Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.",

author = "{van der Sluis}, Frans and {van den Broek}, {Egon L.} and {van Drunen}, Annemiek and Beerends, {John G.}",

year = "2018",

month = sep,

day = "5",

doi = "10.1145/3232078.3232101",

language = "English",

volume = "36",

booktitle = "ECCE'18",

publisher = "Association for Computing Machinery, Inc",

note = "European Conference on Cognitive Ergonomics, ECCE2018 ; Conference date: 05-09-2018 Through 07-09-2018",

}

TY - GEN

T1 - Mobile Screen Size Limits Multimodal Synergy

AU - van der Sluis, Frans

AU - van den Broek, Egon L.

AU - van Drunen, Annemiek

AU - Beerends, John G.

N1 - Conference code: 36

PY - 2018/9/5

Y1 - 2018/9/5

N2 - Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

AB - Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

U2 - 10.1145/3232078.3232101

DO - 10.1145/3232078.3232101

M3 - Article in proceedings

VL - 36

BT - ECCE'18

PB - Association for Computing Machinery, Inc

CY - New York, NY, USA

T2 - European Conference on Cognitive Ergonomics

Y2 - 5 September 2018 through 7 September 2018

ER -

Mobile Screen Size Limits Multimodal Synergy

Abstract

Conference

Access to Document

Fingerprint

Cite this