Crowdsourced emphysema assessment

Silas Nyboe Ørting; Veronika Cheplygina; Jens Petersen; Laura H. Thomsen; Mathilde M. W. Wille; Marleen de Bruijne

doi:10.1007/978-3-319-67534-3_14

Crowdsourced emphysema assessment

Silas Nyboe Ørting^*, Veronika Cheplygina, Jens Petersen, Laura H. Thomsen, Mathilde M. W. Wille, Marleen de Bruijne

^*Corresponding author for this work

Department of Computer Science

2 Citations (Scopus)

Abstract

Classification of emphysema patterns is believed to be useful for improved diagnosis and prognosis of chronic obstructive pulmonary disease. Emphysema patterns can be assessed visually on lung CT scans. Visual assessment is a complex and time-consuming task performed by experts, making it unsuitable for obtaining large amounts of labeled data. We investigate if visual assessment of emphysema can be framed as an image similarity task that does not require expert. Substituting untrained annotators for experts makes it possible to label data sets much faster and at a lower cost. We use crowd annotators to gather similarity triplets and use t-distributed stochastic triplet embedding to learn an embedding. The quality of the embedding is evaluated by predicting expert assessed emphysema patterns. We find that although performance varies due to low quality triplets and randomness in the embedding, we still achieve a median F₁ score of 0.58 for prediction of four patterns.

Original language	English
Title of host publication	Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis : 6th Joint International Workshops, CVII-STENT 2017 and Second International Workshop, LABELS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10–14, 2017, Proceedings
Editors	M. Jorge Cardoso, Tal Arbel, Su-Lin Lee, Veronika Cheplygina, Simone Balocco, Diana Mateus, Guillaume Zahnd, Lena Maier-Hein, Stefanie Dermirci, Eric Granger, Luc Duong, Marc-André Carbonneau, Shadi Albarquoni, Gustaco Carneiro
Number of pages	10
Publisher	Springer
Publication date	2017
Pages	126-135
ISBN (Print)	978-3-319-67533-6
ISBN (Electronic)	978-3-319-67534-3
DOIs	https://doi.org/10.1007/978-3-319-67534-3_14
Publication status	Published - 2017
Event	2nd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis - Québec City, Canada Duration: 10 Sept 2017 → 14 Sept 2017 Conference number: 2

Workshop

Workshop	2nd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis
Number	2
Country/Territory	Canada
City	Québec City
Period	10/09/2017 → 14/09/2017

Series	Lecture notes in computer science
Volume	10552
ISSN	0302-9743

Keywords

Crowdsourcing
Emphysema
Similarity learning

Access to Document

10.1007/978-3-319-67534-3_14

Cite this

Ørting, S. N., Cheplygina, V., Petersen, J., Thomsen, L. H., Wille, M. M. W., & de Bruijne, M. (2017). Crowdsourced emphysema assessment. In M. J. Cardoso, T. Arbel, S.-L. Lee, V. Cheplygina, S. Balocco, D. Mateus, G. Zahnd, L. Maier-Hein, S. Dermirci, E. Granger, L. Duong, M.-A. Carbonneau, S. Albarquoni, & G. Carneiro (Eds.), Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 6th Joint International Workshops, CVII-STENT 2017 and Second International Workshop, LABELS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10–14, 2017, Proceedings (pp. 126-135). Springer. https://doi.org/10.1007/978-3-319-67534-3_14

Crowdsourced emphysema assessment. / Ørting, Silas Nyboe; Cheplygina, Veronika; Petersen, Jens et al.
Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 6th Joint International Workshops, CVII-STENT 2017 and Second International Workshop, LABELS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10–14, 2017, Proceedings. ed. / M. Jorge Cardoso; Tal Arbel; Su-Lin Lee; Veronika Cheplygina; Simone Balocco; Diana Mateus; Guillaume Zahnd; Lena Maier-Hein; Stefanie Dermirci; Eric Granger; Luc Duong; Marc-André Carbonneau; Shadi Albarquoni; Gustaco Carneiro. Springer, 2017. p. 126-135 (Lecture notes in computer science, Vol. 10552).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Ørting, SN, Cheplygina, V, Petersen, J, Thomsen, LH, Wille, MMW & de Bruijne, M 2017, Crowdsourced emphysema assessment. in MJ Cardoso, T Arbel, S-L Lee, V Cheplygina, S Balocco, D Mateus, G Zahnd, L Maier-Hein, S Dermirci, E Granger, L Duong, M-A Carbonneau, S Albarquoni & G Carneiro (eds), Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 6th Joint International Workshops, CVII-STENT 2017 and Second International Workshop, LABELS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10–14, 2017, Proceedings. Springer, Lecture notes in computer science, vol. 10552, pp. 126-135, 2nd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, Québec City, Canada, 10/09/2017. https://doi.org/10.1007/978-3-319-67534-3_14

Ørting SN, Cheplygina V, Petersen J, Thomsen LH, Wille MMW, de Bruijne M. Crowdsourced emphysema assessment. In Cardoso MJ, Arbel T, Lee SL, Cheplygina V, Balocco S, Mateus D, Zahnd G, Maier-Hein L, Dermirci S, Granger E, Duong L, Carbonneau MA, Albarquoni S, Carneiro G, editors, Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 6th Joint International Workshops, CVII-STENT 2017 and Second International Workshop, LABELS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10–14, 2017, Proceedings. Springer. 2017. p. 126-135. (Lecture notes in computer science, Vol. 10552). doi: 10.1007/978-3-319-67534-3_14

Ørting, Silas Nyboe ; Cheplygina, Veronika ; Petersen, Jens et al. / Crowdsourced emphysema assessment. Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis: 6th Joint International Workshops, CVII-STENT 2017 and Second International Workshop, LABELS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 10–14, 2017, Proceedings. editor / M. Jorge Cardoso ; Tal Arbel ; Su-Lin Lee ; Veronika Cheplygina ; Simone Balocco ; Diana Mateus ; Guillaume Zahnd ; Lena Maier-Hein ; Stefanie Dermirci ; Eric Granger ; Luc Duong ; Marc-André Carbonneau ; Shadi Albarquoni ; Gustaco Carneiro. Springer, 2017. pp. 126-135 (Lecture notes in computer science, Vol. 10552).

@inproceedings{4b5bba72f1c740cc98f89698e2f97341,

title = "Crowdsourced emphysema assessment",

abstract = "Classification of emphysema patterns is believed to be useful for improved diagnosis and prognosis of chronic obstructive pulmonary disease. Emphysema patterns can be assessed visually on lung CT scans. Visual assessment is a complex and time-consuming task performed by experts, making it unsuitable for obtaining large amounts of labeled data. We investigate if visual assessment of emphysema can be framed as an image similarity task that does not require expert. Substituting untrained annotators for experts makes it possible to label data sets much faster and at a lower cost. We use crowd annotators to gather similarity triplets and use t-distributed stochastic triplet embedding to learn an embedding. The quality of the embedding is evaluated by predicting expert assessed emphysema patterns. We find that although performance varies due to low quality triplets and randomness in the embedding, we still achieve a median F1 score of 0.58 for prediction of four patterns.",

keywords = "Crowdsourcing, Emphysema, Similarity learning",

author = "{\O}rting, {Silas Nyboe} and Veronika Cheplygina and Jens Petersen and Thomsen, {Laura H.} and Wille, {Mathilde M. W.} and {de Bruijne}, Marleen",

year = "2017",

doi = "10.1007/978-3-319-67534-3_14",

language = "English",

isbn = "978-3-319-67533-6",

series = "Lecture notes in computer science",

publisher = "Springer",

pages = "126--135",

editor = "Cardoso, {M. Jorge} and Tal Arbel and Su-Lin Lee and Veronika Cheplygina and Simone Balocco and Diana Mateus and Guillaume Zahnd and Lena Maier-Hein and Stefanie Dermirci and Eric Granger and Luc Duong and Marc-Andr{\'e} Carbonneau and Shadi Albarquoni and Gustaco Carneiro",

booktitle = "Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis",

note = "2nd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis ; Conference date: 10-09-2017 Through 14-09-2017",

}

TY - GEN

T1 - Crowdsourced emphysema assessment

AU - Ørting, Silas Nyboe

AU - Cheplygina, Veronika

AU - Petersen, Jens

AU - Thomsen, Laura H.

AU - Wille, Mathilde M. W.

AU - de Bruijne, Marleen

N1 - Conference code: 2

PY - 2017

Y1 - 2017

N2 - Classification of emphysema patterns is believed to be useful for improved diagnosis and prognosis of chronic obstructive pulmonary disease. Emphysema patterns can be assessed visually on lung CT scans. Visual assessment is a complex and time-consuming task performed by experts, making it unsuitable for obtaining large amounts of labeled data. We investigate if visual assessment of emphysema can be framed as an image similarity task that does not require expert. Substituting untrained annotators for experts makes it possible to label data sets much faster and at a lower cost. We use crowd annotators to gather similarity triplets and use t-distributed stochastic triplet embedding to learn an embedding. The quality of the embedding is evaluated by predicting expert assessed emphysema patterns. We find that although performance varies due to low quality triplets and randomness in the embedding, we still achieve a median F1 score of 0.58 for prediction of four patterns.

AB - Classification of emphysema patterns is believed to be useful for improved diagnosis and prognosis of chronic obstructive pulmonary disease. Emphysema patterns can be assessed visually on lung CT scans. Visual assessment is a complex and time-consuming task performed by experts, making it unsuitable for obtaining large amounts of labeled data. We investigate if visual assessment of emphysema can be framed as an image similarity task that does not require expert. Substituting untrained annotators for experts makes it possible to label data sets much faster and at a lower cost. We use crowd annotators to gather similarity triplets and use t-distributed stochastic triplet embedding to learn an embedding. The quality of the embedding is evaluated by predicting expert assessed emphysema patterns. We find that although performance varies due to low quality triplets and randomness in the embedding, we still achieve a median F1 score of 0.58 for prediction of four patterns.

KW - Crowdsourcing

KW - Emphysema

KW - Similarity learning

UR - http://www.scopus.com/inward/record.url?scp=85029803937&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-67534-3_14

DO - 10.1007/978-3-319-67534-3_14

M3 - Article in proceedings

AN - SCOPUS:85029803937

SN - 978-3-319-67533-6

T3 - Lecture notes in computer science

SP - 126

EP - 135

BT - Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

A2 - Cardoso, M. Jorge

A2 - Arbel, Tal

A2 - Lee, Su-Lin

A2 - Cheplygina, Veronika

A2 - Balocco, Simone

A2 - Mateus, Diana

A2 - Zahnd, Guillaume

A2 - Maier-Hein, Lena

A2 - Dermirci, Stefanie

A2 - Granger, Eric

A2 - Duong, Luc

A2 - Carbonneau, Marc-André

A2 - Albarquoni, Shadi

A2 - Carneiro, Gustaco

PB - Springer

T2 - 2nd International Workshop on Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

Y2 - 10 September 2017 through 14 September 2017

ER -

Crowdsourced emphysema assessment

Abstract

Workshop

Keywords

Access to Document

Other files and links

Fingerprint

Cite this