No Need to Turn Bayesian in Multilevel Analysis with Few Clusters: How Frequentist Methods Provide Unbiased Estimates and Accurate Inference

TY - UNPB

T1 - No Need to Turn Bayesian in Multilevel Analysis with Few Clusters

T2 - How Frequentist Methods Provide Unbiased Estimates and Accurate Inference

AU - Elff, Martin

AU - Heisig, Jan Paul

AU - Schaeffer, Merlin

AU - Shikano, Susumu

PY - 2016

Y1 - 2016

N2 - Comparative political science has long worried about the performance of multilevel models when the number of upper-level units is small. Exacerbating these concerns, an influential Monte Carlo study by Stegmueller (2013) suggests that frequentist methods yield biased estimates and severely anti-conservative inference with small upper-level samples. Stegmueller recommends Bayesian techniques, which he claims to be superior in terms of both bias and inferential accuracy. In this paper, we reassess and refute these results. First, we formally prove that frequentist maximum likelihood estimators of coefficients are unbiased. The apparent bias found by Stegmueller is simply a manifestation of Monte Carlo Error. Second, we show how inferential problems can be overcome by using restricted maximum likelihood estimators for variance parameters and a t-distribution with appropriate degrees of freedom for statistical inference. Thus, accurate multilevel analysis is possible without turning to Bayesian methods, even if the number of upper-level units is small.

AB - Comparative political science has long worried about the performance of multilevel models when the number of upper-level units is small. Exacerbating these concerns, an influential Monte Carlo study by Stegmueller (2013) suggests that frequentist methods yield biased estimates and severely anti-conservative inference with small upper-level samples. Stegmueller recommends Bayesian techniques, which he claims to be superior in terms of both bias and inferential accuracy. In this paper, we reassess and refute these results. First, we formally prove that frequentist maximum likelihood estimators of coefficients are unbiased. The apparent bias found by Stegmueller is simply a manifestation of Monte Carlo Error. Second, we show how inferential problems can be overcome by using restricted maximum likelihood estimators for variance parameters and a t-distribution with appropriate degrees of freedom for statistical inference. Thus, accurate multilevel analysis is possible without turning to Bayesian methods, even if the number of upper-level units is small.

U2 - 10.17605/OSF.IO/Z65S4

DO - 10.17605/OSF.IO/Z65S4

M3 - Working paper

BT - No Need to Turn Bayesian in Multilevel Analysis with Few Clusters

PB - SocArXiv

ER -