A community resource benchmarking predictions of peptide binding to MHC-I molecules

Bjoern Peters; Huynh-Hoa Bui; Sune Frankild; Morten Nielson; Claus Lundegaard; Emrah Kostem; Derek Basch; Kasper Lamberth; Mikkel Harndahl; Ward Fleri; Stephen S Wilson; John Sidney; Ole Lund; Søren Buus; Alessandro Sette

doi:10.1371/journal.pcbi.0020065

A community resource benchmarking predictions of peptide binding to MHC-I molecules

Bjoern Peters, Huynh-Hoa Bui, Sune Frankild, Morten Nielson, Claus Lundegaard, Emrah Kostem, Derek Basch, Kasper Lamberth, Mikkel Harndahl, Ward Fleri, Stephen S Wilson, John Sidney, Ole Lund, Søren Buus, Alessandro Sette

Department of Immunology and Microbiology

234 Citations (Scopus)

Abstract

Recognition of peptides bound to major histocompatibility complex (MHC) class I molecules by T lymphocytes is an essential part of immune surveillance. Each MHC allele has a characteristic peptide binding preference, which can be captured in prediction algorithms, allowing for the rapid scan of entire pathogen proteomes for peptide likely to bind MHC. Here we make public a large set of 48,828 quantitative peptide-binding affinity measurements relating to 48 different mouse, human, macaque, and chimpanzee MHC class I alleles. We use this data to establish a set of benchmark predictions with one neural network method and two matrix-based prediction methods extensively utilized in our groups. In general, the neural network outperforms the matrix-based predictions mainly due to its ability to generalize even on a small amount of data. We also retrieved predictions from tools publicly available on the internet. While differences in the data used to generate these predictions hamper direct comparisons, we do conclude that tools based on combinatorial peptide libraries perform remarkably well. The transparent prediction evaluation on this dataset provides tool developers with a benchmark for comparison of newly developed prediction methods. In addition, to generate and evaluate our own prediction methods, we have established an easily extensible web-based prediction framework that allows automated side-by-side comparisons of prediction methods implemented by experts. This is an advance over the current practice of tool developers having to generate reference predictions themselves, which can lead to underestimating the performance of prediction methods they are not as familiar with as their own. The overall goal of this effort is to provide a transparent prediction evaluation allowing bioinformaticians to identify promising features of prediction methods and providing guidance to immunologists regarding the reliability of prediction tools.

Original language	English
Journal	PLoS Computational Biology
Volume	2
Issue number	6
Pages (from-to)	e65
DOIs	https://doi.org/10.1371/journal.pcbi.0020065
Publication status	Published - 2006

Access to Document

10.1371/journal.pcbi.0020065

Cite this

@article{af4f3320ebc911ddbf70000ea68e967b,

title = "A community resource benchmarking predictions of peptide binding to MHC-I molecules",

abstract = "Recognition of peptides bound to major histocompatibility complex (MHC) class I molecules by T lymphocytes is an essential part of immune surveillance. Each MHC allele has a characteristic peptide binding preference, which can be captured in prediction algorithms, allowing for the rapid scan of entire pathogen proteomes for peptide likely to bind MHC. Here we make public a large set of 48,828 quantitative peptide-binding affinity measurements relating to 48 different mouse, human, macaque, and chimpanzee MHC class I alleles. We use this data to establish a set of benchmark predictions with one neural network method and two matrix-based prediction methods extensively utilized in our groups. In general, the neural network outperforms the matrix-based predictions mainly due to its ability to generalize even on a small amount of data. We also retrieved predictions from tools publicly available on the internet. While differences in the data used to generate these predictions hamper direct comparisons, we do conclude that tools based on combinatorial peptide libraries perform remarkably well. The transparent prediction evaluation on this dataset provides tool developers with a benchmark for comparison of newly developed prediction methods. In addition, to generate and evaluate our own prediction methods, we have established an easily extensible web-based prediction framework that allows automated side-by-side comparisons of prediction methods implemented by experts. This is an advance over the current practice of tool developers having to generate reference predictions themselves, which can lead to underestimating the performance of prediction methods they are not as familiar with as their own. The overall goal of this effort is to provide a transparent prediction evaluation allowing bioinformaticians to identify promising features of prediction methods and providing guidance to immunologists regarding the reliability of prediction tools.",

author = "Bjoern Peters and Huynh-Hoa Bui and Sune Frankild and Morten Nielson and Claus Lundegaard and Emrah Kostem and Derek Basch and Kasper Lamberth and Mikkel Harndahl and Ward Fleri and Wilson, {Stephen S} and John Sidney and Ole Lund and S{\o}ren Buus and Alessandro Sette",

note = "Keywords: Animals; Databases, Factual; HLA Antigens; Histocompatibility Antigens Class I; Humans; Inhibitory Concentration 50; Macaca; Mice; Neural Networks (Computer); Pan troglodytes; Peptides; ROC Curve; Software",

year = "2006",

doi = "10.1371/journal.pcbi.0020065",

language = "English",

volume = "2",

pages = "e65",

journal = "P L o S Computational Biology (Online)",

issn = "1553-734X",

publisher = "Public Library of Science",

number = "6",

}

TY - JOUR

T1 - A community resource benchmarking predictions of peptide binding to MHC-I molecules

AU - Peters, Bjoern

AU - Bui, Huynh-Hoa

AU - Frankild, Sune

AU - Nielson, Morten

AU - Lundegaard, Claus

AU - Kostem, Emrah

AU - Basch, Derek

AU - Lamberth, Kasper

AU - Harndahl, Mikkel

AU - Fleri, Ward

AU - Wilson, Stephen S

AU - Sidney, John

AU - Lund, Ole

AU - Buus, Søren

AU - Sette, Alessandro

N1 - Keywords: Animals; Databases, Factual; HLA Antigens; Histocompatibility Antigens Class I; Humans; Inhibitory Concentration 50; Macaca; Mice; Neural Networks (Computer); Pan troglodytes; Peptides; ROC Curve; Software

PY - 2006

Y1 - 2006

N2 - Recognition of peptides bound to major histocompatibility complex (MHC) class I molecules by T lymphocytes is an essential part of immune surveillance. Each MHC allele has a characteristic peptide binding preference, which can be captured in prediction algorithms, allowing for the rapid scan of entire pathogen proteomes for peptide likely to bind MHC. Here we make public a large set of 48,828 quantitative peptide-binding affinity measurements relating to 48 different mouse, human, macaque, and chimpanzee MHC class I alleles. We use this data to establish a set of benchmark predictions with one neural network method and two matrix-based prediction methods extensively utilized in our groups. In general, the neural network outperforms the matrix-based predictions mainly due to its ability to generalize even on a small amount of data. We also retrieved predictions from tools publicly available on the internet. While differences in the data used to generate these predictions hamper direct comparisons, we do conclude that tools based on combinatorial peptide libraries perform remarkably well. The transparent prediction evaluation on this dataset provides tool developers with a benchmark for comparison of newly developed prediction methods. In addition, to generate and evaluate our own prediction methods, we have established an easily extensible web-based prediction framework that allows automated side-by-side comparisons of prediction methods implemented by experts. This is an advance over the current practice of tool developers having to generate reference predictions themselves, which can lead to underestimating the performance of prediction methods they are not as familiar with as their own. The overall goal of this effort is to provide a transparent prediction evaluation allowing bioinformaticians to identify promising features of prediction methods and providing guidance to immunologists regarding the reliability of prediction tools.

AB - Recognition of peptides bound to major histocompatibility complex (MHC) class I molecules by T lymphocytes is an essential part of immune surveillance. Each MHC allele has a characteristic peptide binding preference, which can be captured in prediction algorithms, allowing for the rapid scan of entire pathogen proteomes for peptide likely to bind MHC. Here we make public a large set of 48,828 quantitative peptide-binding affinity measurements relating to 48 different mouse, human, macaque, and chimpanzee MHC class I alleles. We use this data to establish a set of benchmark predictions with one neural network method and two matrix-based prediction methods extensively utilized in our groups. In general, the neural network outperforms the matrix-based predictions mainly due to its ability to generalize even on a small amount of data. We also retrieved predictions from tools publicly available on the internet. While differences in the data used to generate these predictions hamper direct comparisons, we do conclude that tools based on combinatorial peptide libraries perform remarkably well. The transparent prediction evaluation on this dataset provides tool developers with a benchmark for comparison of newly developed prediction methods. In addition, to generate and evaluate our own prediction methods, we have established an easily extensible web-based prediction framework that allows automated side-by-side comparisons of prediction methods implemented by experts. This is an advance over the current practice of tool developers having to generate reference predictions themselves, which can lead to underestimating the performance of prediction methods they are not as familiar with as their own. The overall goal of this effort is to provide a transparent prediction evaluation allowing bioinformaticians to identify promising features of prediction methods and providing guidance to immunologists regarding the reliability of prediction tools.

U2 - 10.1371/journal.pcbi.0020065

DO - 10.1371/journal.pcbi.0020065

M3 - Journal article

C2 - 16789818

SN - 1553-734X

VL - 2

SP - e65

JO - P L o S Computational Biology (Online)

JF - P L o S Computational Biology (Online)

IS - 6

ER -

A community resource benchmarking predictions of peptide binding to MHC-I molecules

Abstract

Access to Document

Fingerprint

Cite this