Analysis of Rater Agreement by Rasch and IRT Models

Jørgen Holm Petersen*

*Corresponding author for this work

Abstract

This chapter first discusses the use of item response theory (IRT) model for the analysis of agreement between raters in a situation where all raters have supplied dichotomous ratings of the same cases in a sample. Next, the chapter describes two approaches to the quantification of the rater variation in the Rasch model. The approaches are fixed-effect approach, and random effects approach. The models allow raters to have different propensities to score a given set of individuals positively or negatively. Further, the chapter presents a study of umbilical artery Doppler velocimetry used by obstetricians to asses the status of a fetus. Meta analyses of the randomized clinical trials have shown a 49% reduction in perinatal mortality on using the technique on high-risk pregnancies. It is clear from these considerations that the variations in effects of using the ultrasound technique may in part be attributed to the considerable rater variation.

Original languageEnglish
Title of host publicationRasch Models in Health
Number of pages16
PublisherWiley
Publication date4 Mar 2013
Pages219-234
ISBN (Print)9781848212220
DOIs
Publication statusPublished - 4 Mar 2013

Keywords

  • Doppler velocimetry
  • Item response theory (IRT)
  • Laboratory measurement
  • Medicine
  • Perinatal mortality
  • Rasch model
  • Rater agreement

Fingerprint

Dive into the research topics of 'Analysis of Rater Agreement by Rasch and IRT Models'. Together they form a unique fingerprint.

Cite this