Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?

TY - JOUR

T1 - Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?

AU - Thorlund, Kristian

AU - Devereaux, P J

AU - Wetterslev, Jørn

AU - Guyatt, Gordon

AU - Ioannidis, John P A

AU - Thabane, Lehana

AU - Gluud, Lise-Lotte

AU - Als-Nielsen, Bodil

AU - Gluud, Christian

N1 - Keywords: Data Interpretation, Statistical; False Positive Reactions; Humans; Meta-Analysis as Topic; Randomized Controlled Trials as Topic; Research Design; Treatment Outcome

PY - 2008

Y1 - 2008

N2 - BACKGROUND: Results from apparently conclusive meta-analyses may be false. A limited number of events from a few small trials and the associated random error may be under-recognized sources of spurious findings. The information size (IS, i.e. number of participants) required for a reliable and conclusive meta-analysis should be no less rigorous than the sample size of a single, optimally powered randomized clinical trial. If a meta-analysis is conducted before a sufficient IS is reached, it should be evaluated in a manner that accounts for the increased risk that the result might represent a chance finding (i.e. applying trial sequential monitoring boundaries). METHODS: We analysed 33 meta-analyses with a sufficient IS to detect a treatment effect of 15% relative risk reduction (RRR). We successively monitored the results of the meta-analyses by generating interim cumulative meta-analyses after each included trial and evaluated their results using a conventional statistical criterion (alpha = 0.05) and two-sided Lan-DeMets monitoring boundaries. We examined the proportion of false positive results and important inaccuracies in estimates of treatment effects that resulted from the two approaches. RESULTS: Using the random-effects model and final data, 12 of the meta-analyses yielded P > alpha = 0.05, and 21 yielded P alpha = 0.05. The monitoring boundaries eliminated all false positives. Important inaccuracies in estimates were observed in 6 out of 21 meta-analyses using the conventional P

AB - BACKGROUND: Results from apparently conclusive meta-analyses may be false. A limited number of events from a few small trials and the associated random error may be under-recognized sources of spurious findings. The information size (IS, i.e. number of participants) required for a reliable and conclusive meta-analysis should be no less rigorous than the sample size of a single, optimally powered randomized clinical trial. If a meta-analysis is conducted before a sufficient IS is reached, it should be evaluated in a manner that accounts for the increased risk that the result might represent a chance finding (i.e. applying trial sequential monitoring boundaries). METHODS: We analysed 33 meta-analyses with a sufficient IS to detect a treatment effect of 15% relative risk reduction (RRR). We successively monitored the results of the meta-analyses by generating interim cumulative meta-analyses after each included trial and evaluated their results using a conventional statistical criterion (alpha = 0.05) and two-sided Lan-DeMets monitoring boundaries. We examined the proportion of false positive results and important inaccuracies in estimates of treatment effects that resulted from the two approaches. RESULTS: Using the random-effects model and final data, 12 of the meta-analyses yielded P > alpha = 0.05, and 21 yielded P alpha = 0.05. The monitoring boundaries eliminated all false positives. Important inaccuracies in estimates were observed in 6 out of 21 meta-analyses using the conventional P

U2 - 10.1093/ije/dyn179

DO - 10.1093/ije/dyn179

M3 - Journal article

C2 - 18824467

SN - 0300-5771

VL - 38

SP - 276

EP - 286

JO - International Journal of Epidemiology

JF - International Journal of Epidemiology

IS - 1

ER -