The precision: recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases

TY - JOUR

T1 - The precision

T2 - recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases

AU - Ozenne, Brice

AU - Subtil, Fabien

AU - Maucort-Boulch, Delphine

N1 - Copyright © 2015 Elsevier Inc. All rights reserved.

PY - 2015/8/1

Y1 - 2015/8/1

N2 - Objectives Compare the area under the receiver operating characteristic curve (AUC) vs. the area under the precision-recall curve (AUPRC) in summarizing the performance of a diagnostic biomarker according to the disease prevalence. Study Design and Setting A simulation study was performed considering different sizes of diseased and nondiseased groups. Values of a biomarker were sampled with various variances and differences in mean values between the two groups. The AUCs and the AUPRCs were examined regarding their agreement and vs. the positive predictive value (PPV) and the negative predictive value (NPV) of the biomarker. Results With a disease prevalence of 50%, the AUC and the AUPRC showed high correlations with the PPV and the NPV (ρ > 0.95). With a prevalence of 1%, small PPV and AUPRC values (<0.2) but high AUC values (>0.9) were found. The AUPRC reflected better than the AUC the discriminant ability of the biomarker; it had a higher correlation with the PPV (ρ = 0.995 vs. 0.724; P < 0.001). Conclusion In uncommon and rare diseases, the AUPRC should be preferred to the AUC because it summarizes better the performance of a biomarker.

AB - Objectives Compare the area under the receiver operating characteristic curve (AUC) vs. the area under the precision-recall curve (AUPRC) in summarizing the performance of a diagnostic biomarker according to the disease prevalence. Study Design and Setting A simulation study was performed considering different sizes of diseased and nondiseased groups. Values of a biomarker were sampled with various variances and differences in mean values between the two groups. The AUCs and the AUPRCs were examined regarding their agreement and vs. the positive predictive value (PPV) and the negative predictive value (NPV) of the biomarker. Results With a disease prevalence of 50%, the AUC and the AUPRC showed high correlations with the PPV and the NPV (ρ > 0.95). With a prevalence of 1%, small PPV and AUPRC values (<0.2) but high AUC values (>0.9) were found. The AUPRC reflected better than the AUC the discriminant ability of the biomarker; it had a higher correlation with the PPV (ρ = 0.995 vs. 0.724; P < 0.001). Conclusion In uncommon and rare diseases, the AUPRC should be preferred to the AUC because it summarizes better the performance of a biomarker.

KW - Area Under Curve

KW - Biomarkers

KW - Humans

KW - Predictive Value of Tests

KW - Prevalence

KW - ROC Curve

KW - Rare Diseases

KW - Sensitivity and Specificity

KW - Journal Article

U2 - 10.1016/j.jclinepi.2015.02.010

DO - 10.1016/j.jclinepi.2015.02.010

M3 - Journal article

C2 - 25881487

SN - 0895-4356

VL - 68

SP - 855

EP - 859

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

IS - 8

ER -