Sequential rank agreement methods for comparison of ranked lists

TY - JOUR

T1 - Sequential rank agreement methods for comparison of ranked lists

AU - Ekstrøm, Claus Thorn

AU - Gerds, Thomas Alexander

AU - Jensen, Andreas Kryger

PY - 2019/10/1

Y1 - 2019/10/1

N2 - The comparison of alternative rankings of a set of items is a general and common task in applied statistics. Predictor variables are ranked according to magnitude of association with an outcome, prediction models rank subjects according to the personalized risk of an event, and genetic studies rank genes according to their difference in gene expression levels. We propose a sequential rank agreement measure to quantify the rank agreement among two or more ordered lists. This measure has an intuitive interpretation, it can be applied to any number of lists even if some are partially incomplete, and it provides information about the agreement along the lists. The sequential rank agreement can be evaluated analytically or be compared graphically to a permutation based reference set in order to identify changes in the list agreements. The usefulness of this measure is illustrated using gene rankings, and using data from two Danish ovarian cancer studies where we assess the within and between agreement of different statistical classification methods.

AB - The comparison of alternative rankings of a set of items is a general and common task in applied statistics. Predictor variables are ranked according to magnitude of association with an outcome, prediction models rank subjects according to the personalized risk of an event, and genetic studies rank genes according to their difference in gene expression levels. We propose a sequential rank agreement measure to quantify the rank agreement among two or more ordered lists. This measure has an intuitive interpretation, it can be applied to any number of lists even if some are partially incomplete, and it provides information about the agreement along the lists. The sequential rank agreement can be evaluated analytically or be compared graphically to a permutation based reference set in order to identify changes in the list agreements. The usefulness of this measure is illustrated using gene rankings, and using data from two Danish ovarian cancer studies where we assess the within and between agreement of different statistical classification methods.

U2 - 10.1093/biostatistics/kxy017

DO - 10.1093/biostatistics/kxy017

M3 - Journal article

C2 - 29868883

SN - 1465-4644

VL - 20

SP - 582

EP - 598

JO - Biostatistics

JF - Biostatistics

IS - 4

ER -