Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents

Liane S Feldman, Sarah E Hagarty, Gabriela Ghitulescu, Donna Stanbridge, Gerald M Fried
Journal of the American College of Surgeons 2004, 198 (1): 105-10

BACKGROUND: Technical skills of residents have traditionally been evaluated using subjective In-Training Evaluation Reports (ITERs). We have developed the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS), an objective measure of laparoscopic technical ability. The purpose of the study was to assess the concurrent validity of the MISTELS by exploring the relationship between MISTELS score and ITER assessment.

STUDY DESIGN: Fifty surgery residents were assessed on the MISTELS system. Concurrent ITER assessments of technical skill were collected, and the proportion of superior ratings for the year was calculated. Statistical comparisons were performed by ANOVA and chi-square analysis. The Pearson correlation coefficient was used to compare the scores in the MISTELS with the ITER ratings.

RESULTS: The 50 residents received 277 ITERs for the year, of which 103 (37%) were "superior," 170 (61%) "satisfactory," 4 (1%) "borderline," and 0 "unsatisfactory." The MISTELS score correlated moderately well with the proportion of superior ITER scores (r = 0.51, p < 0.01). Residents who passed the MISTELS had a higher proportion of superior ITER assessments than those who failed the MISTELS (p = 0.02), but residents who performed below their expected level on the MISTELS still received mainly satisfactory ITERs (82 +/- 18%).

CONCLUSIONS: The ITER assessment is poor at identifying residents with below-average technical skills. Residents who perform well in the MISTELS laparoscopic simulator also have better ITER evaluations, providing evidence for the concurrent validity of the MISTELS. Multiple assessment instruments are recommended for assessment of technical competency.

Full Text Links

Find Full Text Links for this Article


You are not logged in. Sign Up or Log In to join the discussion.

Related Papers

Remove bar
Read by QxMD icon Read

Save your favorite articles in one place with a free QxMD account.


Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"