Assessing Trainee Surgeons’ Nontechnical Skills: Five Cases are Sufficient for Reliable Assessments

TY - JOUR

T1 - Assessing Trainee Surgeons’ Nontechnical Skills

T2 - Five Cases are Sufficient for Reliable Assessments

AU - Spanager, Lene

AU - Konge, Lars

AU - Dieckmann, Peter

AU - Beier-Holgersen, Randi

AU - Rosenberg, Jacob

AU - Oestergaard, Doris

N1 - Copyright © 2014 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

PY - 2015/1/1

Y1 - 2015/1/1

N2 - BACKGROUND: Trainee surgeons would benefit from regular, formative assessments to ensure they learn the nontechnical aspects of surgical performance. Non-Technical Skills for Surgeons in Denmark (NOTSSdk) is a tool to assess surgeons' nontechnical skills (NTS) during an operation. The aims of this study were to explore which parts of NOTSSdk supervisors use to assess trainee surgeons' NTS, to determine the internal consistency reliability of NOTSSdk, and to estimate how many operations were needed to obtain reliable ratings of a trainee surgeon's NTS.METHODS: A total of 12 supervisors from 2 hospitals assessed 13 trainees in 48 procedures including laparoscopic cholecystectomies, inguinal hernia repairs, and laparoscopic appendectomies.RESULTS: NOTSSdk showed good internal consistency reliability (R(2) = 0.95). Supervisors used both categories and elements of the tool to provide assessments, although 2 elements of leadership ("supporting others" and "coping with pressure") were considered irrelevant in 27% to 31% of the cases. Assessments of 5 procedures were sufficient to gain reliable ratings (Generalizability coefficient > 0.80) of a trainee surgeon's NTS.CONCLUSION: As supervisors used the full tool, the internal consistency reliability was high and assessment of 5 procedures could be achieved in clinical practice, we suggest that NOTSSdk could be implemented as a formative assessment tool to facilitate the training of surgeons' NTS.

AB - BACKGROUND: Trainee surgeons would benefit from regular, formative assessments to ensure they learn the nontechnical aspects of surgical performance. Non-Technical Skills for Surgeons in Denmark (NOTSSdk) is a tool to assess surgeons' nontechnical skills (NTS) during an operation. The aims of this study were to explore which parts of NOTSSdk supervisors use to assess trainee surgeons' NTS, to determine the internal consistency reliability of NOTSSdk, and to estimate how many operations were needed to obtain reliable ratings of a trainee surgeon's NTS.METHODS: A total of 12 supervisors from 2 hospitals assessed 13 trainees in 48 procedures including laparoscopic cholecystectomies, inguinal hernia repairs, and laparoscopic appendectomies.RESULTS: NOTSSdk showed good internal consistency reliability (R(2) = 0.95). Supervisors used both categories and elements of the tool to provide assessments, although 2 elements of leadership ("supporting others" and "coping with pressure") were considered irrelevant in 27% to 31% of the cases. Assessments of 5 procedures were sufficient to gain reliable ratings (Generalizability coefficient > 0.80) of a trainee surgeon's NTS.CONCLUSION: As supervisors used the full tool, the internal consistency reliability was high and assessment of 5 procedures could be achieved in clinical practice, we suggest that NOTSSdk could be implemented as a formative assessment tool to facilitate the training of surgeons' NTS.

U2 - 10.1016/j.jsurg.2014.06.001

DO - 10.1016/j.jsurg.2014.06.001

M3 - Journal article

C2 - 25023468

SN - 1931-7204

VL - 72

SP - 16

EP - 22

JO - Journal of Surgical Education

JF - Journal of Surgical Education

IS - 1

ER -