Competency-based assessment in surgeon-performed head and neck ultrasonography: A validity study

TY - JOUR

T1 - Competency-based assessment in surgeon-performed head and neck ultrasonography

T2 - A validity study

AU - Todsen, Tobias

AU - Melchiors, Jacob

AU - Charabi, Birgitte

AU - Henriksen, Birthe

AU - Ringsted, Charlotte

AU - Konge, Lars

AU - von Buchwald, Christian

N1 - © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

PY - 2018/6

Y1 - 2018/6

N2 - Objective: Head and neck ultrasonography (HNUS) increasingly is used as a point-of-care diagnostic tool by otolaryngologists. However, ultrasonography (US) is a very operator-dependent image modality. Hence, this study aimed to explore the diagnostic accuracy of surgeon-performed HNUS and to establish validity evidence for an objective structured assessment of ultrasound skills (OSAUS) used for competency-based assessment. Study Design: A prospective experimental study. Methods: Six otolaryngologists and 11 US novices were included in a standardized test setup for which they had to perform focused HNUS of eight patients suspected for different head and neck lesions. Their diagnostic accuracy was calculated based on the US reports, and two blinded raters assessed the video-recorded US performance using the OSAUS scale. Results: The otolaryngologists obtained a high diagnostic accuracy on 88% (range 63%–100%) compared to the US novices on 38% (range 0–63%); P < 0.001. The OSAUS score demonstrated good inter-case reliability (0.85) and inter-rater reliability (0.76), and significant discrimination between otolaryngologist and US novices; P < 0.001. A strong correlation between the OSAUS score and the diagnostic accuracy was found (Spearman's ρ, 0.85; P < P 0.001), and a pass/fail score was established at 2.8. Conclusion: Strong validity evidence supported the use of the OSAUS scale to assess HNUS competence with good reliability, significant discrimination between US competence levels, and a strong correlation of assessment score to diagnostic accuracy. An OSAUS pass/fail score was established and could be used for competence-based assessment in surgeon-performed HNUS. Level of Evidence: NA. Laryngoscope, 128:1346–1352, 2018.

AB - Objective: Head and neck ultrasonography (HNUS) increasingly is used as a point-of-care diagnostic tool by otolaryngologists. However, ultrasonography (US) is a very operator-dependent image modality. Hence, this study aimed to explore the diagnostic accuracy of surgeon-performed HNUS and to establish validity evidence for an objective structured assessment of ultrasound skills (OSAUS) used for competency-based assessment. Study Design: A prospective experimental study. Methods: Six otolaryngologists and 11 US novices were included in a standardized test setup for which they had to perform focused HNUS of eight patients suspected for different head and neck lesions. Their diagnostic accuracy was calculated based on the US reports, and two blinded raters assessed the video-recorded US performance using the OSAUS scale. Results: The otolaryngologists obtained a high diagnostic accuracy on 88% (range 63%–100%) compared to the US novices on 38% (range 0–63%); P < 0.001. The OSAUS score demonstrated good inter-case reliability (0.85) and inter-rater reliability (0.76), and significant discrimination between otolaryngologist and US novices; P < 0.001. A strong correlation between the OSAUS score and the diagnostic accuracy was found (Spearman's ρ, 0.85; P < P 0.001), and a pass/fail score was established at 2.8. Conclusion: Strong validity evidence supported the use of the OSAUS scale to assess HNUS competence with good reliability, significant discrimination between US competence levels, and a strong correlation of assessment score to diagnostic accuracy. An OSAUS pass/fail score was established and could be used for competence-based assessment in surgeon-performed HNUS. Level of Evidence: NA. Laryngoscope, 128:1346–1352, 2018.

KW - Journal Article

U2 - 10.1002/lary.26841

DO - 10.1002/lary.26841

M3 - Journal article

C2 - 28868625

SN - 0023-852X

VL - 128

SP - 1346

EP - 1352

JO - Laryngoscope

JF - Laryngoscope

IS - 6

ER -