Classifying head movements in video-recorded conversations based on movement velocity, acceleration and jerk

Bart Jongejan; Patrizia Paggio; Costanza Navarretta

Classifying head movements in video-recorded conversations based on movement velocity, acceleration and jerk

Bart Jongejan, Patrizia Paggio, Costanza Navarretta

Department of Nordic Studies and Linguistics

Abstract

This paper is about the automatic annotation of head movements in videos of face-to-face conversations. Manual annotation of gestures is resource consuming, and modelling gesture behaviours in different types of communicative settings requires many types of annotated data. Therefore, developing methods for automatic annotation is crucial. We present an approach where an SVM classifier learns to classify head movements based on measurements of velocity, acceleration, and the third derivative of position with respect to time, jerk. Consequently, annotations of head movements are added to new video data. The results of the automatic annotation are evaluated against manual annotations in the same data and show an accuracy of 73.47% with respect to these. The results also show that using jerk improves accuracy.

Original language	English
Article number	003
Journal	Linköping Electronic Conference Proceedings
Issue number	141
Pages (from-to)	10-17
ISSN	1650-3740
Publication status	Published - 2017
Event	Nordic and European Symposium on Multimodal Communication : 7th Nordic and 4th European Symposium on Multimodal Communication - University of Copenhagen, Copenhagen, Denmark Duration: 29 Sept 2016 → 30 Sept 2016 Conference number: 4th, 7th http://mmsym.org/?page_id=412

Conference

Conference	Nordic and European Symposium on Multimodal Communication
Number	4th, 7th
Location	University of Copenhagen
Country/Territory	Denmark
City	Copenhagen
Period	29/09/2016 → 30/09/2016
Internet address	http://mmsym.org/?page_id=412

Access to Document

http://www.ep.liu.se/ecp/article.asp?issue=141&article=003&volume=

Cite this

@inproceedings{95b4568c4af54849b787a694047842f3,

title = "Classifying head movements in video-recorded conversations based on movement velocity, acceleration and jerk",

abstract = "This paper is about the automatic annotation of head movements in videos of face-to-face conversations. Manual annotation of gestures is resource consuming, and modelling gesture behaviours in different types of communicative settings requires many types of annotated data. Therefore, developing methods for automatic annotation is crucial. We present an approach where an SVM classifier learns to classify head movements based on measurements of velocity, acceleration, and the third derivative of position with respect to time, jerk. Consequently, annotations of head movements are added to new video data. The results of the automatic annotation are evaluated against manual annotations in the same data and show an accuracy of 73.47% with respect to these. The results also show that using jerk improves accuracy.",

author = "Bart Jongejan and Patrizia Paggio and Costanza Navarretta",

year = "2017",

language = "English",

pages = "10--17",

journal = "Link{\"o}ping Electronic Conference Proceedings",

issn = "1650-3740",

number = "141",

note = "Nordic and European Symposium on Multimodal Communication : 7th Nordic and 4th European Symposium on Multimodal Communication , MMSYM2017 ; Conference date: 29-09-2016 Through 30-09-2016",

url = "http://mmsym.org/?page_id=412",

}

TY - GEN

T1 - Classifying head movements in video-recorded conversations based on movement velocity, acceleration and jerk

AU - Jongejan, Bart

AU - Paggio, Patrizia

AU - Navarretta, Costanza

N1 - Conference code: 4th, 7th

PY - 2017

Y1 - 2017

N2 - This paper is about the automatic annotation of head movements in videos of face-to-face conversations. Manual annotation of gestures is resource consuming, and modelling gesture behaviours in different types of communicative settings requires many types of annotated data. Therefore, developing methods for automatic annotation is crucial. We present an approach where an SVM classifier learns to classify head movements based on measurements of velocity, acceleration, and the third derivative of position with respect to time, jerk. Consequently, annotations of head movements are added to new video data. The results of the automatic annotation are evaluated against manual annotations in the same data and show an accuracy of 73.47% with respect to these. The results also show that using jerk improves accuracy.

AB - This paper is about the automatic annotation of head movements in videos of face-to-face conversations. Manual annotation of gestures is resource consuming, and modelling gesture behaviours in different types of communicative settings requires many types of annotated data. Therefore, developing methods for automatic annotation is crucial. We present an approach where an SVM classifier learns to classify head movements based on measurements of velocity, acceleration, and the third derivative of position with respect to time, jerk. Consequently, annotations of head movements are added to new video data. The results of the automatic annotation are evaluated against manual annotations in the same data and show an accuracy of 73.47% with respect to these. The results also show that using jerk improves accuracy.

M3 - Conference article

SN - 1650-3740

SP - 10

EP - 17

JO - Linköping Electronic Conference Proceedings

JF - Linköping Electronic Conference Proceedings

IS - 141

M1 - 003

T2 - Nordic and European Symposium on Multimodal Communication

Y2 - 29 September 2016 through 30 September 2016

ER -

Classifying head movements in video-recorded conversations based on movement velocity, acceleration and jerk

Abstract

Conference

Access to Document

Fingerprint

Cite this