TY - BOOK
T1 - An Investigation on the Cognitive Effects of Emoji Usage in Text
AU - Ousterhout, Thomas Kenneth
PY - 2017/9
Y1 - 2017/9
N2 - Face-to-face communication is multimodal involving at least the auditory (speech) and the visual (gestures such as head movements, facial expressions and hand gestures) modalities. While multimodal signals are produced naturally in face-to-face communication, they are not so easily provided in written computer-mediated communication, and especially in instant messaging. The visual nonverbal cues are not available and there is a great potential for miscommunication. The growing use of emojis, pictures or short videos of facial expressions and symbols of various types, are a means to replace non-verbal cues. Preceding studies have shown that emojis contribute to the semantics of the message, but their effect on reading and their potential uses as e.g. reading aids, are not thoroughly studied.The purpose of this dissertation is to investigate some cognitive effects of emojis in text, in order to determine how these stimuli complement the written text in a way that facilitates reading ability and/orcomprehension. The method comprised electroencephalography, button press response times and accuracy. The main results of the experiments are the following. It is possible to reproduce the results of a medical EEG equipment with a cheap commercial EEG device. Simple words and emojis produce semantic priming despite being different channels within the same modality and seem to corroborate with how we behave in face-to-face communication. Merging words with emojis also produced semantic congruity effects. When mixing emojis into sentences, the results of the experiments show that if the sentences are kept simple, they produce semantic integration and that facial emojis were processed more quickly than hand gesture emojis.
AB - Face-to-face communication is multimodal involving at least the auditory (speech) and the visual (gestures such as head movements, facial expressions and hand gestures) modalities. While multimodal signals are produced naturally in face-to-face communication, they are not so easily provided in written computer-mediated communication, and especially in instant messaging. The visual nonverbal cues are not available and there is a great potential for miscommunication. The growing use of emojis, pictures or short videos of facial expressions and symbols of various types, are a means to replace non-verbal cues. Preceding studies have shown that emojis contribute to the semantics of the message, but their effect on reading and their potential uses as e.g. reading aids, are not thoroughly studied.The purpose of this dissertation is to investigate some cognitive effects of emojis in text, in order to determine how these stimuli complement the written text in a way that facilitates reading ability and/orcomprehension. The method comprised electroencephalography, button press response times and accuracy. The main results of the experiments are the following. It is possible to reproduce the results of a medical EEG equipment with a cheap commercial EEG device. Simple words and emojis produce semantic priming despite being different channels within the same modality and seem to corroborate with how we behave in face-to-face communication. Merging words with emojis also produced semantic congruity effects. When mixing emojis into sentences, the results of the experiments show that if the sentences are kept simple, they produce semantic integration and that facial emojis were processed more quickly than hand gesture emojis.
M3 - Ph.D. thesis
BT - An Investigation on the Cognitive Effects of Emoji Usage in Text
PB - Det Humanistiske Fakultet, Københavns Universitet
ER -
