Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy

Morten Jørgensen; Mona Meral Savran; Christos Christakopoulos; Toke Bek; Jakob Grauslund; Peter Bjerre Toft; Focke Ziemssen; Lars Konge; Torben Lykke Sørensen; Yousif Subhi

doi:10.1111/aos.14065

Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy

Morten Jørgensen, Mona Meral Savran, Christos Christakopoulos, Toke Bek, Jakob Grauslund, Peter Bjerre Toft, Focke Ziemssen, Lars Konge, Torben Lykke Sørensen, Yousif Subhi^*

^*Corresponding author for this work

Department of Clinical Medicine

2 Citations (Scopus)

Abstract

Purpose: Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy. Methods: The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass–fail score and the consequences of its use). Results: The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass–fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification. Conclusion: We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.

Original language	English
Journal	Acta Ophthalmologica
ISSN	1755-375X
DOIs	https://doi.org/10.1111/aos.14065
Publication status	Published - 1 Nov 2019

Access to Document

10.1111/aos.14065

aos.14065Final published version, 529 KB

Cite this

@article{f77d7b8b4df543e0b5f397523198d8d9,

title = "Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy",

abstract = "Purpose: Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy. Methods: The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass–fail score and the consequences of its use). Results: The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass–fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification. Conclusion: We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.",

author = "Morten J{\o}rgensen and Savran, {Mona Meral} and Christos Christakopoulos and Toke Bek and Jakob Grauslund and Toft, {Peter Bjerre} and Focke Ziemssen and Lars Konge and S{\o}rensen, {Torben Lykke} and Yousif Subhi",

year = "2019",

month = nov,

day = "1",

doi = "10.1111/aos.14065",

language = "English",

journal = "Acta Ophthalmologica",

issn = "1755-375X",

publisher = "Wiley-Blackwell",

}

TY - JOUR

T1 - Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy

AU - Jørgensen, Morten

AU - Savran, Mona Meral

AU - Christakopoulos, Christos

AU - Bek, Toke

AU - Grauslund, Jakob

AU - Toft, Peter Bjerre

AU - Ziemssen, Focke

AU - Konge, Lars

AU - Sørensen, Torben Lykke

AU - Subhi, Yousif

PY - 2019/11/1

Y1 - 2019/11/1

N2 - Purpose: Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy. Methods: The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass–fail score and the consequences of its use). Results: The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass–fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification. Conclusion: We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.

AB - Purpose: Direct ophthalmoscopy can reveal systemic, neurologic and ophthalmic conditions, but is poorly mastered among young physicians. A theoretical test is needed to measure effect of educational interventions. We developed and gathered validity evidence for a multiple-choice questionnaire (MCQ)-based theoretical test in direct ophthalmoscopy. Methods: The MCQ was developed by interviewing experts. Then, validity evidence was evaluated using Messick's validity framework. Content was ensured by inviting the experts to contribute in a Delphi-like process. Response process was ensured by piloting and by streamlining all instructions. Then, the test was taken by ophthalmologists and by medical students without experience in direct ophthalmoscopy. Results were used to evaluate internal structure (item quality analysis and internal consistency), relations to other variables (correlation of test scores to experience level) and consequences (establishment of pass–fail score and the consequences of its use). Results: The first phase of the study yielded 100 MCQs. In second phase, we identified that 60 items fulfilled predefined relevance and item quality requirements. These items demonstrated very high internal consistency (Cronbach's alpha = 0.95), significantly discriminated medical students from specialists (p < 0.001, independent samples t-test) and the established pass–fail score of 50 (83%) correct answers resulted in no false positives (students passing) and no false negatives (specialists failing). A Decision study identified that sampling 15 items suffice for certification. Conclusion: We developed and validated an MCQ-based theoretical test in direct ophthalmoscopy that enables an evidence-based approach to measuring, evaluating and certifying the theoretical knowledge necessary for direct ophthalmoscopy.

U2 - 10.1111/aos.14065

DO - 10.1111/aos.14065

M3 - Journal article

C2 - 30816642

SN - 1755-375X

JO - Acta Ophthalmologica

JF - Acta Ophthalmologica

ER -

Development and validation of a multiple-choice questionnaire-based theoretical test in direct ophthalmoscopy

Abstract

Access to Document

Fingerprint

Cite this