Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge

K. Murphy; B. van Ginneken; J. Reinhardt; S. Kabus; K. Ding; X. Deng; K. Cao; K. Du; G. Christensen; V. Garcia; T. Vercauteren; N. Ayache; O. Commowick; G. Malandain; B. Glocker; N. Paragios; N. Navab; Vladlena Gorbunova; Jon Sporring; Marleen de Bruijne; X. Han; M. Heinrich; J. Schnabel; M. Jenkinson; C. Lorenz; M. Modat; J. McClelland; S. Ourselin; S. Muenzing; M. Viergever; D. De Nigris; D. Collins; T. Arbel; M. Peroni; R. Li; G. Sharp; A. Schmidt-Richberg; J. Ehrhardt; R. Werner; D. Smeets; D. Loeckx; G. Song; N. Tustison; B. Avants; J. Gee; M. Staring; S. Klein; B. Stoel; M. Urschler; M. Werlberger; J. Vandemeulebroucke; S. Rit; D. Sarrut; J. Pluim

doi:10.1109/TMI.2011.2158349

Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge

K. Murphy, B. van Ginneken, J. Reinhardt, S. Kabus, K. Ding, X. Deng, K. Cao, K. Du, G. Christensen, V. Garcia, T. Vercauteren, N. Ayache, O. Commowick, G. Malandain, B. Glocker, N. Paragios, N. Navab, Vladlena Gorbunova, Jon Sporring, Marleen de BruijneX. Han, M. Heinrich, J. Schnabel, M. Jenkinson, C. Lorenz, M. Modat, J. McClelland, S. Ourselin, S. Muenzing, M. Viergever, D. De Nigris, D. Collins, T. Arbel, M. Peroni, R. Li, G. Sharp, A. Schmidt-Richberg, J. Ehrhardt, R. Werner, D. Smeets, D. Loeckx, G. Song, N. Tustison, B. Avants, J. Gee, M. Staring, S. Klein, B. Stoel, M. Urschler, M. Werlberger, J. Vandemeulebroucke, S. Rit, D. Sarrut, J. Pluim

Department of Computer Science

292 Citations (Scopus)

3429 Downloads (Pure)

Abstract

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

Original language	English
Journal	IEEE Transactions on Medical Imaging
Volume	30
Issue number	11
Pages (from-to)	1901-1920
Number of pages	20
ISSN	1558-254X
DOIs	https://doi.org/10.1109/TMI.2011.2158349
Publication status	Published - Nov 2011

Access to Document

10.1109/TMI.2011.2158349

Murphy_2011_Evaluation_of_RegistrationFinal published version, 1.8 MB

Cite this

Murphy, K., van Ginneken, B., Reinhardt, J., Kabus, S., Ding, K., Deng, X., Cao, K., Du, K., Christensen, G., Garcia, V., Vercauteren, T., Ayache, N., Commowick, O., Malandain, G., Glocker, B., Paragios, N., Navab, N., Gorbunova, V., Sporring, J., ... Pluim, J. (2011). Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge. IEEE Transactions on Medical Imaging, 30(11), 1901-1920. https://doi.org/10.1109/TMI.2011.2158349

Murphy, K, van Ginneken, B, Reinhardt, J, Kabus, S, Ding, K, Deng, X, Cao, K, Du, K, Christensen, G, Garcia, V, Vercauteren, T, Ayache, N, Commowick, O, Malandain, G, Glocker, B, Paragios, N, Navab, N, Gorbunova, V, Sporring, J , de Bruijne, M, Han, X, Heinrich, M, Schnabel, J, Jenkinson, M, Lorenz, C, Modat, M, McClelland, J, Ourselin, S, Muenzing, S, Viergever, M, De Nigris, D, Collins, D, Arbel, T, Peroni, M, Li, R, Sharp, G, Schmidt-Richberg, A, Ehrhardt, J, Werner, R, Smeets, D, Loeckx, D, Song, G, Tustison, N, Avants, B, Gee, J, Staring, M, Klein, S, Stoel, B, Urschler, M, Werlberger, M, Vandemeulebroucke, J, Rit, S, Sarrut, D & Pluim, J 2011, 'Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge', IEEE Transactions on Medical Imaging, vol. 30, no. 11, pp. 1901-1920. https://doi.org/10.1109/TMI.2011.2158349

@article{c64e27e5cbc2423b872e08012a7c1a16,

title = "Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge",

abstract = "EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.",

author = "K. Murphy and {van Ginneken}, B. and J. Reinhardt and S. Kabus and K. Ding and X. Deng and K. Cao and K. Du and G. Christensen and V. Garcia and T. Vercauteren and N. Ayache and O. Commowick and G. Malandain and B. Glocker and N. Paragios and N. Navab and Vladlena Gorbunova and Jon Sporring and {de Bruijne}, Marleen and X. Han and M. Heinrich and J. Schnabel and M. Jenkinson and C. Lorenz and M. Modat and J. McClelland and S. Ourselin and S. Muenzing and M. Viergever and {De Nigris}, D. and D. Collins and T. Arbel and M. Peroni and R. Li and G. Sharp and A. Schmidt-Richberg and J. Ehrhardt and R. Werner and D. Smeets and D. Loeckx and G. Song and N. Tustison and B. Avants and J. Gee and M. Staring and S. Klein and B. Stoel and M. Urschler and M. Werlberger and J. Vandemeulebroucke and S. Rit and D. Sarrut and J. Pluim",

year = "2011",

month = nov,

doi = "10.1109/TMI.2011.2158349",

language = "English",

volume = "30",

pages = "1901--1920",

journal = "I E E E Transactions on Medical Imaging",

issn = "0278-0062",

publisher = "Institute of Electrical and Electronics Engineers",

number = "11",

}

TY - JOUR

T1 - Evaluation of registration methods on thoracic CT

T2 - the EMPIRE10 challenge

AU - Murphy, K.

AU - van Ginneken, B.

AU - Reinhardt, J.

AU - Kabus, S.

AU - Ding, K.

AU - Deng, X.

AU - Cao, K.

AU - Du, K.

AU - Christensen, G.

AU - Garcia, V.

AU - Vercauteren, T.

AU - Ayache, N.

AU - Commowick, O.

AU - Malandain, G.

AU - Glocker, B.

AU - Paragios, N.

AU - Navab, N.

AU - Gorbunova, Vladlena

AU - Sporring, Jon

AU - de Bruijne, Marleen

AU - Han, X.

AU - Heinrich, M.

AU - Schnabel, J.

AU - Jenkinson, M.

AU - Lorenz, C.

AU - Modat, M.

AU - McClelland, J.

AU - Ourselin, S.

AU - Muenzing, S.

AU - Viergever, M.

AU - De Nigris, D.

AU - Collins, D.

AU - Arbel, T.

AU - Peroni, M.

AU - Li, R.

AU - Sharp, G.

AU - Schmidt-Richberg, A.

AU - Ehrhardt, J.

AU - Werner, R.

AU - Smeets, D.

AU - Loeckx, D.

AU - Song, G.

AU - Tustison, N.

AU - Avants, B.

AU - Gee, J.

AU - Staring, M.

AU - Klein, S.

AU - Stoel, B.

AU - Urschler, M.

AU - Werlberger, M.

AU - Vandemeulebroucke, J.

AU - Rit, S.

AU - Sarrut, D.

AU - Pluim, J.

PY - 2011/11

Y1 - 2011/11

N2 - EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

AB - EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

U2 - 10.1109/TMI.2011.2158349

DO - 10.1109/TMI.2011.2158349

M3 - Journal article

SN - 0278-0062

VL - 30

SP - 1901

EP - 1920

JO - I E E E Transactions on Medical Imaging

JF - I E E E Transactions on Medical Imaging

IS - 11

ER -

Evaluation of registration methods on thoracic CT: the EMPIRE10 challenge

Abstract

Access to Document

Fingerprint

Cite this