A Comparison of Model-Based and Design-Based Impact Evaluations of Interventions in Developing Countries

Henrik Hansen; Ninja Ritter Klejnstrup; Ole Winckler Andersen

doi:10.1177/1098214013476915

A Comparison of Model-Based and Design-Based Impact Evaluations of Interventions in Developing Countries

Henrik Hansen, Ninja Ritter Klejnstrup, Ole Winckler Andersen

6 Citations (Scopus)

Abstract

There is a long-standing debate as to whether nonexperimental estimators of causal effects of social programs can overcome selection bias. Most existing reviews either are inconclusive or point to significant selection biases in nonexperimental studies. However, many of the reviews, the so-called between-studies, do not make direct comparisons of the estimates. We survey four impact studies, all using data from development interventions that directly compare experimental and nonexperimental impact estimates. Our review illustrates that when the program participation process is well understood, and correctly modeled, then the nonexperimental estimators can overcome the selection bias to the same degree as randomized controlled trials. Hence, we suggest that evaluators of development programs aim to be careful and precise in the formulation of the statistical model for the assignment into the program and also to use the assignment information for model-based systematic sampling.

Original language	English
Journal	American Journal of Evaluation
Volume	34
Issue number	3
Pages (from-to)	320-338
Number of pages	19
ISSN	1098-2140
DOIs	https://doi.org/10.1177/1098214013476915
Publication status	Published - Sept 2013

Access to Document

10.1177/1098214013476915

http://aje.sagepub.com/content/34/3/320.abstract

Cite this

@article{ac0c93c5714c42f98a3c3200fe14636d,

title = "A Comparison of Model-Based and Design-Based Impact Evaluations of Interventions in Developing Countries",

abstract = "There is a long-standing debate as to whether nonexperimental estimators of causal effects of social programs can overcome selection bias. Most existing reviews either are inconclusive or point to significant selection biases in nonexperimental studies. However, many of the reviews, the so-called between-studies, do not make direct comparisons of the estimates. We survey four impact studies, all using data from development interventions that directly compare experimental and nonexperimental impact estimates. Our review illustrates that when the program participation process is well understood, and correctly modeled, then the nonexperimental estimators can overcome the selection bias to the same degree as randomized controlled trials. Hence, we suggest that evaluators of development programs aim to be careful and precise in the formulation of the statistical model for the assignment into the program and also to use the assignment information for model-based systematic sampling.",

author = "Henrik Hansen and Klejnstrup, {Ninja Ritter} and Andersen, {Ole Winckler}",

year = "2013",

month = sep,

doi = "10.1177/1098214013476915",

language = "English",

volume = "34",

pages = "320--338",

journal = "American Journal of Evaluation",

issn = "1098-2140",

publisher = "SAGE Publications",

number = "3",

}

TY - JOUR

T1 - A Comparison of Model-Based and Design-Based Impact Evaluations of Interventions in Developing Countries

AU - Hansen, Henrik

AU - Klejnstrup, Ninja Ritter

AU - Andersen, Ole Winckler

PY - 2013/9

Y1 - 2013/9

N2 - There is a long-standing debate as to whether nonexperimental estimators of causal effects of social programs can overcome selection bias. Most existing reviews either are inconclusive or point to significant selection biases in nonexperimental studies. However, many of the reviews, the so-called between-studies, do not make direct comparisons of the estimates. We survey four impact studies, all using data from development interventions that directly compare experimental and nonexperimental impact estimates. Our review illustrates that when the program participation process is well understood, and correctly modeled, then the nonexperimental estimators can overcome the selection bias to the same degree as randomized controlled trials. Hence, we suggest that evaluators of development programs aim to be careful and precise in the formulation of the statistical model for the assignment into the program and also to use the assignment information for model-based systematic sampling.

AB - There is a long-standing debate as to whether nonexperimental estimators of causal effects of social programs can overcome selection bias. Most existing reviews either are inconclusive or point to significant selection biases in nonexperimental studies. However, many of the reviews, the so-called between-studies, do not make direct comparisons of the estimates. We survey four impact studies, all using data from development interventions that directly compare experimental and nonexperimental impact estimates. Our review illustrates that when the program participation process is well understood, and correctly modeled, then the nonexperimental estimators can overcome the selection bias to the same degree as randomized controlled trials. Hence, we suggest that evaluators of development programs aim to be careful and precise in the formulation of the statistical model for the assignment into the program and also to use the assignment information for model-based systematic sampling.

U2 - 10.1177/1098214013476915

DO - 10.1177/1098214013476915

M3 - Journal article

SN - 1098-2140

VL - 34

SP - 320

EP - 338

JO - American Journal of Evaluation

JF - American Journal of Evaluation

IS - 3

ER -