Reliable and Valid Assessment of Point-of-care Ultrasonography

Tobias Todsen; Martin Grønnebæk Tolsgaard; Beth Härstedt Olsen; Birthe Merete Henriksen; Jens Georg Hillingsø; Lars Konge; Morten Lind Jensen; Charlotte Ringsted

doi:10.1097/SLA.0000000000000552

Reliable and Valid Assessment of Point-of-care Ultrasonography

Tobias Todsen, Martin Grønnebæk Tolsgaard, Beth Härstedt Olsen, Birthe Merete Henriksen, Jens Georg Hillingsø, Lars Konge, Morten Lind Jensen, Charlotte Ringsted

Department of Clinical Medicine

48 Citations (Scopus)

Abstract

Objective: To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance. Background: POC US is increasingly used by clinicians and is an essential part of the management of acute surgical conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees" ultrasonography competence is needed to ensure patient safety. Methods: Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different surgical patient cases in a controlled set-up. All ultrasound examinations were video-recorded and assessed by 2 blinded radiologists using OSAUS. Reliability was examined using generalizability theory. Construct validity was examined by comparing performance scores between the groups and by correlating physicians" OSAUS scores with diagnostic accuracy. Results: The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores between the novice group (17.0; SD 8.4) and the intermediate group (30.0; SD 10.1), P = 0.007, as well as between the intermediate group and the expert group (72.9; SD 4.4), P = 0.04, and by a high correlation between OSAUS scores and diagnostic accuracy (Spearman correlation coefficient = 0.76; P < 0.001). Conclusions: This study demonstrates high reliability as well as evidence of construct validity of the OSAUS scale for assessment of POC US competence. Hence, the OSAUS scale may be suitable for both in-training as well as endof-training assessment.

Original language	English
Journal	Annals of Surgery
Volume	261
Issue number	2
Pages (from-to)	309-15
Number of pages	7
ISSN	0003-4932
DOIs	https://doi.org/10.1097/SLA.0000000000000552
Publication status	Published - Feb 2015

Access to Document

10.1097/SLA.0000000000000552

Cite this

@article{fbd393ebc6b84a808b1f5ea5a7ec1f14,

title = "Reliable and Valid Assessment of Point-of-care Ultrasonography",

abstract = "Objective: To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance. Background: POC US is increasingly used by clinicians and is an essential part of the management of acute surgical conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees{"} ultrasonography competence is needed to ensure patient safety. Methods: Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different surgical patient cases in a controlled set-up. All ultrasound examinations were video-recorded and assessed by 2 blinded radiologists using OSAUS. Reliability was examined using generalizability theory. Construct validity was examined by comparing performance scores between the groups and by correlating physicians{"} OSAUS scores with diagnostic accuracy. Results: The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores between the novice group (17.0; SD 8.4) and the intermediate group (30.0; SD 10.1), P = 0.007, as well as between the intermediate group and the expert group (72.9; SD 4.4), P = 0.04, and by a high correlation between OSAUS scores and diagnostic accuracy (Spearman correlation coefficient = 0.76; P < 0.001). Conclusions: This study demonstrates high reliability as well as evidence of construct validity of the OSAUS scale for assessment of POC US competence. Hence, the OSAUS scale may be suitable for both in-training as well as endof-training assessment.",

keywords = "Abdomen, Clinical Competence, Humans, Point-of-Care Systems, Reproducibility of Results, Single-Blind Method, Ultrasonography, Video Recording",

author = "Tobias Todsen and Tolsgaard, {Martin Gr{\o}nneb{\ae}k} and Olsen, {Beth H{\"a}rstedt} and Henriksen, {Birthe Merete} and Hillings{\o}, {Jens Georg} and Lars Konge and Jensen, {Morten Lind} and Charlotte Ringsted",

year = "2015",

month = feb,

doi = "10.1097/SLA.0000000000000552",

language = "English",

volume = "261",

pages = "309--15",

journal = "Annals of Surgery",

issn = "0003-4932",

publisher = "Lippincott Williams & Wilkins",

number = "2",

}

TY - JOUR

T1 - Reliable and Valid Assessment of Point-of-care Ultrasonography

AU - Todsen, Tobias

AU - Tolsgaard, Martin Grønnebæk

AU - Olsen, Beth Härstedt

AU - Henriksen, Birthe Merete

AU - Hillingsø, Jens Georg

AU - Konge, Lars

AU - Jensen, Morten Lind

AU - Ringsted, Charlotte

PY - 2015/2

Y1 - 2015/2

N2 - Objective: To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance. Background: POC US is increasingly used by clinicians and is an essential part of the management of acute surgical conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees" ultrasonography competence is needed to ensure patient safety. Methods: Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different surgical patient cases in a controlled set-up. All ultrasound examinations were video-recorded and assessed by 2 blinded radiologists using OSAUS. Reliability was examined using generalizability theory. Construct validity was examined by comparing performance scores between the groups and by correlating physicians" OSAUS scores with diagnostic accuracy. Results: The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores between the novice group (17.0; SD 8.4) and the intermediate group (30.0; SD 10.1), P = 0.007, as well as between the intermediate group and the expert group (72.9; SD 4.4), P = 0.04, and by a high correlation between OSAUS scores and diagnostic accuracy (Spearman correlation coefficient = 0.76; P < 0.001). Conclusions: This study demonstrates high reliability as well as evidence of construct validity of the OSAUS scale for assessment of POC US competence. Hence, the OSAUS scale may be suitable for both in-training as well as endof-training assessment.

AB - Objective: To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance. Background: POC US is increasingly used by clinicians and is an essential part of the management of acute surgical conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees" ultrasonography competence is needed to ensure patient safety. Methods: Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different surgical patient cases in a controlled set-up. All ultrasound examinations were video-recorded and assessed by 2 blinded radiologists using OSAUS. Reliability was examined using generalizability theory. Construct validity was examined by comparing performance scores between the groups and by correlating physicians" OSAUS scores with diagnostic accuracy. Results: The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores between the novice group (17.0; SD 8.4) and the intermediate group (30.0; SD 10.1), P = 0.007, as well as between the intermediate group and the expert group (72.9; SD 4.4), P = 0.04, and by a high correlation between OSAUS scores and diagnostic accuracy (Spearman correlation coefficient = 0.76; P < 0.001). Conclusions: This study demonstrates high reliability as well as evidence of construct validity of the OSAUS scale for assessment of POC US competence. Hence, the OSAUS scale may be suitable for both in-training as well as endof-training assessment.

KW - Abdomen

KW - Clinical Competence

KW - Humans

KW - Point-of-Care Systems

KW - Reproducibility of Results

KW - Single-Blind Method

KW - Ultrasonography

KW - Video Recording

U2 - 10.1097/SLA.0000000000000552

DO - 10.1097/SLA.0000000000000552

M3 - Journal article

C2 - 24509198

SN - 0003-4932

VL - 261

SP - 309

EP - 315

JO - Annals of Surgery

JF - Annals of Surgery

IS - 2

ER -

Reliable and Valid Assessment of Point-of-care Ultrasonography

Abstract

Access to Document

Fingerprint

Cite this