Use of an Objective Structured Clinical Examination (OSCE) for the assessment of physician performance in the ultrasound evaluation of trauma.

BACKGROUND: A reliable means of assessing physician competency in performing ultrasound (US) is critical for training and credentialing. Objective Structured Clinical Examinations (OSCE) have been used successfully to assess clinical competency in other areas of surgical education but have not been applied previously to trauma ultrasound training. The objectives of this study were to assess physician performance in the focused abdominal sonography in trauma (FAST) examination by using a specifically designed OSCE, and to determine whether the OSCE detects differences in two determinants of competency (knowledge acquisition and clinical interpretation skills).

METHODS: Eighty-two physicians in surgery (n = 49) and emergency medicine (n = 33) at a Level I trauma center were evaluated. All participated in a FAST course consisting of didactic sessions on US physics, indications, and technique, FAST examination videos, and a hands-on session with human models. The OSCE consisted of two parts: written examination that assessed factual knowledge, and videotape of real-time US examinations that assessed interpretation skills. The OSCE was administered before and after the FAST course.

RESULTS: Significant improvements in postcourse OSCE scores were observed for factual knowledge (52.5 +/- 2.0 vs. 87.5 +/- 1.1, p < 0.001) and interpretation skills (27.2 +/- 1.4 vs. 62.9 +/- 1.3, p < 0.007). Scores for US interpretation were significantly lower than those for factual knowledge at both precourse (27.2 +/- 1.4 vs. 52.5 - 2.0, p < 0.001) and postcourse (62.9 +/- 1.3 vs. 87.5 +/- 1.1, p < 0.01). No performance differences were observed between surgeons and emergency medicine physicians and no effect of training level on test scores was observed.

CONCLUSION: Knowledge acquisition and US interpretation skills can be assessed reliably with a specifically designed OSCE. Although both skills improved after participation in a FAST course, US interpretation scores were consistently lower than those for factual knowledge. This study supports the use of the objective structured clinical examination in both the design of ultrasound teaching programs and the assessment of physician competency.