Using virtual-reality simulation to assess performance in endobronchial ultrasound

TY - JOUR

T1 - Using virtual-reality simulation to assess performance in endobronchial ultrasound

AU - Konge, Lars

AU - Annema, Jouke

AU - Clementsen, Paul

AU - Minddal, Valentina

AU - Vilmann, Peter

AU - Ringsted, Charlotte

PY - 2013/7

Y1 - 2013/7

N2 - Background: For optimal treatment of patients with non-small cell lung carcinoma, it is essential to have physicians with competence in endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA). EBUS training and certification requirements are under discussion and the establishment of basic competence should be based on an objective assessment of performance. Objectives: The aims of this study were to design an evidence-based and credible EBUS certification based on a virtual-reality (VR) EBUS simulator test. Methods: Twenty-two respiratory physicians were divided into 3 groups: experienced EBUS operators (group 1, n = 6), untrained novices (group 2, n = 8) and simulator-trained novices (group 3, n = 8). Each physician performed two standardized simulated EBUS-TBNA procedures. Simulator metrics with discriminatory ability were identified and reliability was explored. Finally, the contrasting-groups method was used to establish a pass/fail standard, and the consequences of this standard were explored. Results: Successfully sampled lymph nodes and procedure time were the only simulator metrics that showed statistically significant differences of p = 0.047 and p = 0.002, respectively. The resulting quality score (QS, i.e. sampled lymph nodes per minute) showed an acceptable reliability and a generalizability coefficient of 0.67. Reliability of 0.8 could be obtained by testing in 4 procedures. Median QS was 0.24 (range 0.21-0.26) and 0.098 (range 0.04-0.21) for groups 1 and 2, respectively (p = 0.001). The resulting pass/fail standard was 0.19. Group 3 had a median posttraining QS of 0.11 (range 0-0.17). None of them met the pass/fail standard. Conclusions: With careful design of standardized tests, a credible standard setting and appropriate transfer studies, VR simulators could be an important first line in credentialing before proceeding to supervised performance on patients.

AB - Background: For optimal treatment of patients with non-small cell lung carcinoma, it is essential to have physicians with competence in endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA). EBUS training and certification requirements are under discussion and the establishment of basic competence should be based on an objective assessment of performance. Objectives: The aims of this study were to design an evidence-based and credible EBUS certification based on a virtual-reality (VR) EBUS simulator test. Methods: Twenty-two respiratory physicians were divided into 3 groups: experienced EBUS operators (group 1, n = 6), untrained novices (group 2, n = 8) and simulator-trained novices (group 3, n = 8). Each physician performed two standardized simulated EBUS-TBNA procedures. Simulator metrics with discriminatory ability were identified and reliability was explored. Finally, the contrasting-groups method was used to establish a pass/fail standard, and the consequences of this standard were explored. Results: Successfully sampled lymph nodes and procedure time were the only simulator metrics that showed statistically significant differences of p = 0.047 and p = 0.002, respectively. The resulting quality score (QS, i.e. sampled lymph nodes per minute) showed an acceptable reliability and a generalizability coefficient of 0.67. Reliability of 0.8 could be obtained by testing in 4 procedures. Median QS was 0.24 (range 0.21-0.26) and 0.098 (range 0.04-0.21) for groups 1 and 2, respectively (p = 0.001). The resulting pass/fail standard was 0.19. Group 3 had a median posttraining QS of 0.11 (range 0-0.17). None of them met the pass/fail standard. Conclusions: With careful design of standardized tests, a credible standard setting and appropriate transfer studies, VR simulators could be an important first line in credentialing before proceeding to supervised performance on patients.

U2 - 10.1159/000350428

DO - 10.1159/000350428

M3 - Journal article

C2 - 23712017

SN - 0025-7931

VL - 86

SP - 59

EP - 65

JO - Respiration

JF - Respiration

IS - 1

ER -