Introduction: There has been interest in the use of high-fidelity medical simulation to evaluate performance. We hypothesized that technical and nontechnical performance in the simulated environment is related to other various criterion measures, providing evidence to support the validity of the scores from the performance-based assessment.
Methods: Twelve third-year anesthesia residents participated in a series of 6 short 5-minute scenarios and 1 longer 30-minute scenario. The short scenarios measured technical skills, whereas the longer one focused on nontechnical skills. Two independent raters scored subjects using analytic and holistic ratings. Short scenarios involved acute hemorrhage, blocked endotracheal tube, bronchospasm, hyperkalemia, tension pneumothorax, and unstable ventricular tachycardia. The long scenario concerned management of myocardial ischemia/infarction leading to cardiac arrest. Scores from the simulations were correlated with (a) rankings generated from an Internet-based global ranking instrument that categorized residents based on overall clinical ability and (b) residency board scores.
Results: There were moderate correlations between various participant scores from the simulation-based assessment and aggregate rankings based on the global ranking instrument and residency examination scores.
Conclusions: The associations between simulator performance, both for technical and nontechnical skills, and other markers of ability provide some evidence to support the validity of simulation-based assessment scores. Replication studies with larger numbers of residents are warranted.