What Do Usability Test Moderators Say? ‘Mm hm’, ‘Uh-huh’, and Beyond

TY - GEN

T1 - What Do Usability Test Moderators Say? ‘Mm hm’, ‘Uh-huh’, and Beyond

AU - Hertzum, Morten

AU - Kristoffersen, Kristina Bonde

PY - 2018/9/29

Y1 - 2018/9/29

N2 - Moderators in usability tests wrestle with the conflicting goals of obtaining relevant information from the users while at the same time avoiding to influence the users in ways that change how they use and feel about the tested system. In this study we investigate what moderators say by categorizing the moderator verbalizations from 12 test sessions. During the test tasks affirmations (38%) were the most common moderator verbalizations, followed by task instructions (32%) and prompts for reflection (16%). In addition, more of the moderator verbalizations during the tasks were closed (31%) than open (14%) and many more were positive (16%) than negative (1%). The moderators verbalized at a lower rate during the tasks than in the part of the sessions before the first task and after the last task. Still, they talked quite a lot. We discuss the content of their verbalizations and the implications of our findings.

AB - Moderators in usability tests wrestle with the conflicting goals of obtaining relevant information from the users while at the same time avoiding to influence the users in ways that change how they use and feel about the tested system. In this study we investigate what moderators say by categorizing the moderator verbalizations from 12 test sessions. During the test tasks affirmations (38%) were the most common moderator verbalizations, followed by task instructions (32%) and prompts for reflection (16%). In addition, more of the moderator verbalizations during the tasks were closed (31%) than open (14%) and many more were positive (16%) than negative (1%). The moderators verbalized at a lower rate during the tasks than in the part of the sessions before the first task and after the last task. Still, they talked quite a lot. We discuss the content of their verbalizations and the implications of our findings.

KW - Faculty of Humanities

KW - Usability evaluation method

KW - Usability testing

KW - Test moderation

KW - Test instructions

KW - Verbal reports

KW - Thinking aloud

U2 - 10.1145/3240167.3240181

DO - 10.1145/3240167.3240181

M3 - Article in proceedings

SP - 364

EP - 375

BT - NordiCHI2018: Proceedings of the 10th Nordic Conference on Human-Computer Interaction

PB - Association for Computing Machinery

CY - New York

ER -