Simple randomization did not protect against bias in smaller trials

TY - JOUR

T1 - Simple randomization did not protect against bias in smaller trials

AU - Nguyen, Tri Long

AU - Collins, Gary S.

AU - Lamy, André

AU - Devereaux, Philip J.

AU - Daurès, Jean Pierre

AU - Landais, Paul

AU - Le Manach, Yannick

PY - 2017

Y1 - 2017

N2 - Objectives By removing systematic differences across treatment groups, simple randomization is assumed to protect against bias. However, random differences may remain if the sample size is insufficiently large. We sought to determine the minimal sample size required to eliminate random differences, thereby allowing an unbiased estimation of the treatment effect. Study Design and Setting We reanalyzed two published multicenter, large, and simple trials: the International Stroke Trial (IST) and the Coronary Artery Bypass Grafting (CABG) Off- or On-Pump Revascularization Study (CORONARY). We reiterated 1,000 times the analysis originally reported by the investigators in random samples of varying size. We measured the covariates balance across the treatment arms. We estimated the effect of aspirin and heparin on death or dependency at 30 days after stroke (IST), and the effect of off-pump CABG on a composite primary outcome of death, nonfatal stroke, nonfatal myocardial infarction, or new renal failure requiring dialysis at 30 days (CORONARY). In addition, we conducted a series of Monte Carlo simulations of randomized trials to supplement these analyses. Results Randomization removes random differences between treatment groups when including at least 1,000 participants, thereby resulting in minimal bias in effects estimation. Later, substantial bias is observed. In a short review, we show such an enrollment is achieved in 41.5% of phase 3 trials published in the highest impact medical journals. Conclusions Conclusions drawn from completely randomized trials enrolling a few participants may not be reliable. In these circumstances, alternatives such as minimization or blocking should be considered for allocating the treatment.

AB - Objectives By removing systematic differences across treatment groups, simple randomization is assumed to protect against bias. However, random differences may remain if the sample size is insufficiently large. We sought to determine the minimal sample size required to eliminate random differences, thereby allowing an unbiased estimation of the treatment effect. Study Design and Setting We reanalyzed two published multicenter, large, and simple trials: the International Stroke Trial (IST) and the Coronary Artery Bypass Grafting (CABG) Off- or On-Pump Revascularization Study (CORONARY). We reiterated 1,000 times the analysis originally reported by the investigators in random samples of varying size. We measured the covariates balance across the treatment arms. We estimated the effect of aspirin and heparin on death or dependency at 30 days after stroke (IST), and the effect of off-pump CABG on a composite primary outcome of death, nonfatal stroke, nonfatal myocardial infarction, or new renal failure requiring dialysis at 30 days (CORONARY). In addition, we conducted a series of Monte Carlo simulations of randomized trials to supplement these analyses. Results Randomization removes random differences between treatment groups when including at least 1,000 participants, thereby resulting in minimal bias in effects estimation. Later, substantial bias is observed. In a short review, we show such an enrollment is achieved in 41.5% of phase 3 trials published in the highest impact medical journals. Conclusions Conclusions drawn from completely randomized trials enrolling a few participants may not be reliable. In these circumstances, alternatives such as minimization or blocking should be considered for allocating the treatment.

KW - Bias

KW - Causal inference

KW - Clinical trial

KW - Covariate balance

KW - Randomization

KW - Sample size

U2 - 10.1016/j.jclinepi.2017.02.010

DO - 10.1016/j.jclinepi.2017.02.010

M3 - Journal article

C2 - 28257927

AN - SCOPUS:85015992437

SN - 0895-4356

VL - 84

SP - 105

EP - 113

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

ER -