Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents

Jeffrey G Chipman, Constance C Schmitz
Journal of the American College of Surgeons 2009, 209 (3): 364-370.e2

BACKGROUND: In response to new Accreditation Council for Graduate Medical Education requirements about simulation skill laboratories, programs are incorporating simulation into residents' training. Despite substantial research on simulators, few data exist to support the effectiveness of simulation skills curricula. We report on an Objective Structured Assessment of Technical Skills (OSATS) used to assess residents' needs and evaluate a curriculum designed to increase proficiency.

STUDY DESIGN: The five-session (10-week) curriculum covered asepsis, skin preparation, gowning, gloving, knot-tying, suturing, and excision. Performance on a 20-minute OSATS station was measured by unblinded raters using a task-specific checklist and seven global rating scales. Interns' pre-post improvement was assessed using paired t-tests. PGY2 and PGY3 residents were used as nonequivalent controls; their scores set a benchmark for PGY1 residents postcurriculum. Percentage of possible points earned was compared with a 75% "needs" criterion; ANOVA was used to assess group differences at the p < 0.05 level.

RESULTS: Seven PGY2 and 6 PGY3 residents took the OSATS; 24 of 25 PGY1s completed both the baseline and postcurriculum OSATS. At baseline, PGY1 mean percent correct total score was 49%; they performed considerably below PGY2 (68%) and PGY3 (74%) residents. PGY1 scores improved significantly after 10 weeks (p < or = 0.001). When their postcurriculum scores were compared with PGY2 and PGY3 resident benchmarks, there were no significant differences in checklist (p = 0.38), global item (p = 0.29), or total scores (p = 0.45).

CONCLUSIONS: Our results suggest that the simulation curriculum helped PGY1 residents attain basic surgical skills at levels consistent with PGY2 and PGY3 residents as measured by an OSATS. Only PGY3 residents performed at the 75% criterion.

Full Text Links

Find Full Text Links for this Article


You are not logged in. Sign Up or Log In to join the discussion.

Related Papers

Remove bar
Read by QxMD icon Read

Save your favorite articles in one place with a free QxMD account.


Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"