Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review

Emmanuelle Kempf; Jennifer A. de Beyer; Jonathan Cook; Jane Holmes; Seid Mohammed; Tri Long Nguyên; Iveta Simera; Marialena Trivella; Douglas G. Altman; Sally Hopewell; Karel G.M. Moons; Raphael Porcher; Johannes B. Reitsma; Willi Sauerbrei; Gary S. Collins

doi:10.1038/s41416-018-0305-5

Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review

Emmanuelle Kempf, Jennifer A. de Beyer, Jonathan Cook, Jane Holmes, Seid Mohammed, Tri Long Nguyên, Iveta Simera, Marialena Trivella, Douglas G. Altman, Sally Hopewell, Karel G.M. Moons, Raphael Porcher, Johannes B. Reitsma, Willi Sauerbrei, Gary S. Collins^*

^*Corresponding author for this work

5 Citations (Scopus)

Abstract

Background: Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology. Methods: This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty). Results: Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1–Q3, 1–7). The prognostic factors’ effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions. Conclusions: Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.

Original language	English
Journal	British Journal of Cancer
Volume	119
Issue number	10
Pages (from-to)	1288-1296
Number of pages	9
ISSN	0007-0920
DOIs	https://doi.org/10.1038/s41416-018-0305-5
Publication status	Published - 2018
Externally published	Yes

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1038/s41416-018-0305-5

s41416-018-0305-5.pdfFinal published version, 386 KBLicence: CC BY

Cite this

Kempf, E., de Beyer, J. A., Cook, J., Holmes, J., Mohammed, S., Nguyên, T. L., Simera, I., Trivella, M., Altman, D. G., Hopewell, S., Moons, K. G. M., Porcher, R., Reitsma, J. B., Sauerbrei, W., & Collins, G. S. (2018). Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review. British Journal of Cancer, 119(10), 1288-1296. https://doi.org/10.1038/s41416-018-0305-5

Kempf, E, de Beyer, JA, Cook, J, Holmes, J, Mohammed, S, Nguyên, TL, Simera, I, Trivella, M, Altman, DG, Hopewell, S, Moons, KGM, Porcher, R, Reitsma, JB, Sauerbrei, W & Collins, GS 2018, 'Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review', British Journal of Cancer, vol. 119, no. 10, pp. 1288-1296. https://doi.org/10.1038/s41416-018-0305-5

@article{3f9b258d41624067bfe8c25f745ea2be,

title = "Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review",

abstract = "Background: Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology. Methods: This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty). Results: Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1–Q3, 1–7). The prognostic factors{\textquoteright} effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions. Conclusions: Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.",

author = "Emmanuelle Kempf and {de Beyer}, {Jennifer A.} and Jonathan Cook and Jane Holmes and Seid Mohammed and Nguy{\^e}n, {Tri Long} and Iveta Simera and Marialena Trivella and Altman, {Douglas G.} and Sally Hopewell and Moons, {Karel G.M.} and Raphael Porcher and Reitsma, {Johannes B.} and Willi Sauerbrei and Collins, {Gary S.}",

year = "2018",

doi = "10.1038/s41416-018-0305-5",

language = "English",

volume = "119",

pages = "1288--1296",

journal = "The British journal of cancer. Supplement",

issn = "0007-0920",

publisher = "nature publishing group",

number = "10",

}

TY - JOUR

T1 - Overinterpretation and misreporting of prognostic factor studies in oncology

T2 - a systematic review

AU - Kempf, Emmanuelle

AU - de Beyer, Jennifer A.

AU - Cook, Jonathan

AU - Holmes, Jane

AU - Mohammed, Seid

AU - Nguyên, Tri Long

AU - Simera, Iveta

AU - Trivella, Marialena

AU - Altman, Douglas G.

AU - Hopewell, Sally

AU - Moons, Karel G.M.

AU - Porcher, Raphael

AU - Reitsma, Johannes B.

AU - Sauerbrei, Willi

AU - Collins, Gary S.

PY - 2018

Y1 - 2018

N2 - Background: Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology. Methods: This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty). Results: Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1–Q3, 1–7). The prognostic factors’ effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions. Conclusions: Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.

AB - Background: Cancer prognostic biomarkers have shown disappointing clinical applicability. The objective of this study was to classify and estimate how study results are overinterpreted and misreported in prognostic factor studies in oncology. Methods: This systematic review focused on 17 oncology journals with an impact factor above 7. PubMed was searched for primary clinical studies published in 2015, evaluating prognostic factors. We developed a classification system, focusing on three domains: misleading reporting (selective, incomplete reporting, misreporting), misleading interpretation (unreliable statistical analysis, spin) and misleading extrapolation of the results (claiming irrelevant clinical applicability, ignoring uncertainty). Results: Our search identified 10,844 articles. The 98 studies included investigated a median of two prognostic factors (Q1–Q3, 1–7). The prognostic factors’ effects were selectively and incompletely reported in 35/98 and 24/98 full texts, respectively. Twenty-nine articles used linguistic spin in the form of strong statements. Linguistic spin rejecting non-significant results was found in 34 full-text results and 15 abstract results sections. One in five articles had discussion and/or abstract conclusions that were inconsistent with the study findings. Sixteen reports had discrepancies between their full-text and abstract conclusions. Conclusions: Our study provides evidence of frequent overinterpretation of findings of prognostic factor assessment in high-impact medical oncology journals.

U2 - 10.1038/s41416-018-0305-5

DO - 10.1038/s41416-018-0305-5

M3 - Journal article

C2 - 30353050

AN - SCOPUS:85055491223

SN - 0007-0920

VL - 119

SP - 1288

EP - 1296

JO - The British journal of cancer. Supplement

JF - The British journal of cancer. Supplement

IS - 10

ER -

Overinterpretation and misreporting of prognostic factor studies in oncology: a systematic review

Abstract

UN SDGs

Access to Document

Fingerprint

Cite this