Multi-view Consensus CNN for 3D Facial Landmark Placement

Rasmus R. Paulsen; Kristine Aavild Juhl; Thilde Marie Haspang; Thomas Hansen; Melanie Ganz; Gudmundur Einarsson

doi:10.1007/978-3-030-20887-5_44

Multi-view Consensus CNN for 3D Facial Landmark Placement

Rasmus R. Paulsen, Kristine Aavild Juhl, Thilde Marie Haspang, Thomas Hansen, Melanie Ganz, Gudmundur Einarsson

Department of Computer Science

6 Citations (Scopus)

Abstract

The rapid increase in the availability of accurate 3D scanning devices has moved facial recognition and analysis into the 3D domain. 3D facial landmarks are often used as a simple measure of anatomy and it is crucial to have accurate algorithms for automatic landmark placement. The current state-of-the-art approaches have yet to gain from the dramatic increase in performance reported in human pose tracking and 2D facial landmark placement due to the use of deep convolutional neural networks (CNN). Development of deep learning approaches for 3D meshes has given rise to the new subfield called geometric deep learning, where one topic is the adaptation of meshes for the use of deep CNNs. In this work, we demonstrate how methods derived from geometric deep learning, namely multi-view CNNs, can be combined with recent advances in human pose tracking. The method finds 2D landmark estimates and propagates this information to 3D space, where a consensus method determines the accurate 3D face landmark position. We utilise the method on a standard 3D face dataset and show that it outperforms current methods by a large margin. Further, we demonstrate how models trained on 3D range scans can be used to accurately place anatomical landmarks in magnetic resonance images.

Original language	English
Title of host publication	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number of pages	14
Publisher	Springer
Publication date	2019
Pages	706-719
ISBN (Print)	9783030208868
DOIs	https://doi.org/10.1007/978-3-030-20887-5_44
Publication status	Published - 2019

Series	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11361 LNCS

Keywords

3D facial landmarks
Geometric deep learning
Multi-view CNN

Access to Document

10.1007/978-3-030-20887-5_44

Cite this

Multi-view Consensus CNN for 3D Facial Landmark Placement. / Paulsen, Rasmus R.; Juhl, Kristine Aavild; Haspang, Thilde Marie et al.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, 2019. p. 706-719 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 11361 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Book chapter › Research › peer-review

Paulsen, RR, Juhl, KA, Haspang, TM, Hansen, T, Ganz, M & Einarsson, G 2019, Multi-view Consensus CNN for 3D Facial Landmark Placement. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11361 LNCS, pp. 706-719. https://doi.org/10.1007/978-3-030-20887-5_44

Paulsen RR, Juhl KA, Haspang TM, Hansen T, Ganz M, Einarsson G. Multi-view Consensus CNN for 3D Facial Landmark Placement. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer. 2019. p. 706-719. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 11361 LNCS). doi: 10.1007/978-3-030-20887-5_44

Paulsen, Rasmus R. ; Juhl, Kristine Aavild ; Haspang, Thilde Marie et al. / Multi-view Consensus CNN for 3D Facial Landmark Placement. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, 2019. pp. 706-719 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 11361 LNCS).

@inbook{6dd733d006d9466b970b1da1672ba231,

title = "Multi-view Consensus CNN for 3D Facial Landmark Placement",

abstract = "The rapid increase in the availability of accurate 3D scanning devices has moved facial recognition and analysis into the 3D domain. 3D facial landmarks are often used as a simple measure of anatomy and it is crucial to have accurate algorithms for automatic landmark placement. The current state-of-the-art approaches have yet to gain from the dramatic increase in performance reported in human pose tracking and 2D facial landmark placement due to the use of deep convolutional neural networks (CNN). Development of deep learning approaches for 3D meshes has given rise to the new subfield called geometric deep learning, where one topic is the adaptation of meshes for the use of deep CNNs. In this work, we demonstrate how methods derived from geometric deep learning, namely multi-view CNNs, can be combined with recent advances in human pose tracking. The method finds 2D landmark estimates and propagates this information to 3D space, where a consensus method determines the accurate 3D face landmark position. We utilise the method on a standard 3D face dataset and show that it outperforms current methods by a large margin. Further, we demonstrate how models trained on 3D range scans can be used to accurately place anatomical landmarks in magnetic resonance images.",

keywords = "3D facial landmarks, Geometric deep learning, Multi-view CNN",

author = "Paulsen, {Rasmus R.} and Juhl, {Kristine Aavild} and Haspang, {Thilde Marie} and Thomas Hansen and Melanie Ganz and Gudmundur Einarsson",

year = "2019",

doi = "10.1007/978-3-030-20887-5_44",

language = "English",

isbn = "9783030208868",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "706--719",

booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - CHAP

T1 - Multi-view Consensus CNN for 3D Facial Landmark Placement

AU - Paulsen, Rasmus R.

AU - Juhl, Kristine Aavild

AU - Haspang, Thilde Marie

AU - Hansen, Thomas

AU - Ganz, Melanie

AU - Einarsson, Gudmundur

PY - 2019

Y1 - 2019

N2 - The rapid increase in the availability of accurate 3D scanning devices has moved facial recognition and analysis into the 3D domain. 3D facial landmarks are often used as a simple measure of anatomy and it is crucial to have accurate algorithms for automatic landmark placement. The current state-of-the-art approaches have yet to gain from the dramatic increase in performance reported in human pose tracking and 2D facial landmark placement due to the use of deep convolutional neural networks (CNN). Development of deep learning approaches for 3D meshes has given rise to the new subfield called geometric deep learning, where one topic is the adaptation of meshes for the use of deep CNNs. In this work, we demonstrate how methods derived from geometric deep learning, namely multi-view CNNs, can be combined with recent advances in human pose tracking. The method finds 2D landmark estimates and propagates this information to 3D space, where a consensus method determines the accurate 3D face landmark position. We utilise the method on a standard 3D face dataset and show that it outperforms current methods by a large margin. Further, we demonstrate how models trained on 3D range scans can be used to accurately place anatomical landmarks in magnetic resonance images.

AB - The rapid increase in the availability of accurate 3D scanning devices has moved facial recognition and analysis into the 3D domain. 3D facial landmarks are often used as a simple measure of anatomy and it is crucial to have accurate algorithms for automatic landmark placement. The current state-of-the-art approaches have yet to gain from the dramatic increase in performance reported in human pose tracking and 2D facial landmark placement due to the use of deep convolutional neural networks (CNN). Development of deep learning approaches for 3D meshes has given rise to the new subfield called geometric deep learning, where one topic is the adaptation of meshes for the use of deep CNNs. In this work, we demonstrate how methods derived from geometric deep learning, namely multi-view CNNs, can be combined with recent advances in human pose tracking. The method finds 2D landmark estimates and propagates this information to 3D space, where a consensus method determines the accurate 3D face landmark position. We utilise the method on a standard 3D face dataset and show that it outperforms current methods by a large margin. Further, we demonstrate how models trained on 3D range scans can be used to accurately place anatomical landmarks in magnetic resonance images.

KW - 3D facial landmarks

KW - Geometric deep learning

KW - Multi-view CNN

U2 - 10.1007/978-3-030-20887-5_44

DO - 10.1007/978-3-030-20887-5_44

M3 - Book chapter

SN - 9783030208868

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 706

EP - 719

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

PB - Springer

ER -

Multi-view Consensus CNN for 3D Facial Landmark Placement

Abstract

Keywords

Access to Document

Fingerprint

Cite this