Assessment of resident surgical skills: is testing feasible?

Barbara Goff, Lynn Mandel, Gretchen Lentz, Amy Vanblaricom, Anne-Marie Amies Oelschlager, David Lee, Andrew Galakatos, Matthew Davies, Peter Nielsen
American Journal of Obstetrics and Gynecology 2005, 192 (4): 1331-8; discussion 1338-40

OBJECTIVE: We have previously shown that in a single residency program objective structured assessment of technical skills (OSATS) is a reliable and valid method of assessing surgical competency. Our goal was to establish feasibility of this evaluation instrument when administered at multiple residency programs throughout the US, and assess the impact of a laboratory-based surgical curriculum on results.

STUDY DESIGN: An OSATS exam was administered to 116 residents from 5 residency programs. One of the residency programs had participated in a comprehensive surgical curriculum over a 4-year period of time. The exam consisted of 3 open and 3 laparoscopic tasks. Residents were graded by both a blinded and unblinded examiner with task-specific checklist, global rating scale, pass/fail assessment, and tasks were timed. All tasks were performed on life-like models.

RESULTS: Examinations were successfully completed at all sites. Each exam required 90 minutes of resident time. Reliability indices calculated with Cronbach's alpha were .97 for overall global rating and .95 for checklists. Interrater reliability between blinded and unblinded examiners ranged from .71 to .97 for individual tasks and was .95 overall. Assessment of construct validity (the ability to distinguish among residency levels) found significant differences among the residents for both blinded and unblinded examiners for all evaluation outcomes except time. For the test overall, the global rating scale showed significant differences among all 4 residency levels. The checklist showed significant differences at three levels (PGY3-4 >PGY2 >PGY1). Approximate cost for replaceable items was 40 US dollars to 150 per resident depending on which tasks were chosen. Comparison of scores between residents who received a laboratory-based curriculum and those who did not revealed significantly higher scores and shorter time to complete tasks for the group who received additional training.

CONCLUSION: Large-scale testing has confirmed that OSATS is an objective, reliable, and valid method to assess surgical skills, and can easily be administered in most residency programs. A laboratory-based surgical curriculum improved test results and reduced time to complete tasks.

Full Text Links

Find Full Text Links for this Article


You are not logged in. Sign Up or Log In to join the discussion.

Related Papers

Remove bar
Read by QxMD icon Read

Save your favorite articles in one place with a free QxMD account.


Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"