Improving the National Board of Medical Examiners internal Medicine Subject Exam for use in clerkship evaluation.

OBJECTIVE: To provide a consensus opinion on modifying the National Board of Medical Examiners (NBME) Medicine Subject Exam (Shelf) to: 1) reflect the internal medicine clerkship curriculum, developed by the Society of General Internal Medicine (SGIM) and the Clerkship Directors in Internal Medicine (CDIM); 2) emphasize knowledge important for a clerkship student; and 3) obtain feedback about students' performances on the Shelf.

DESIGN: Two-round Delphi technique.

PARTICIPANTS: The CDIM Research and Evaluation Committee and CDIM members on NBME Step 2 Committees.

MEASUREMENTS: Using 1-5 Likert scales (5 = highest ratings), the group rated test question content for relevance to the SGIM-CDIM Curriculum Guide and importance for clerkship students' knowledge. The Shelf content is organized into 4 physician tasks and into 11 sections that are generally organ system based. Each iteration of the Shelf has 100 questions. Participants indicated a desired distribution of questions by physician task and section, topics critical for inclusion on each exam, and new topics to include. They specified the types of feedback clerkship directors desired on students' performances. Following the first round, participants viewed pooled results prior to submitting their second-round responses.

RESULTS: Of 15 individuals contacted, 12 (80%) participated in each round. The desired distribution by physician task was: diagnosis (43), treatment (23), mechanism of disease (20), and health maintenance (15). The sections with the most questions requested were the cardiovascular (17), respiratory (15), and gastroenterology (12) sections. The fewest were requested in aging/ethics (4) and neurology, dermatology, and immunology (5 each). Examples of low-rated content were Wilson's Disease, chancroid and tracheal rupture (all <2.0). Health maintenance in type 2 diabetes, hypertension, and cardiovascular disease all received 5.0 ratings. Participants desired feedback by: section (4.6) and physician task (3.9), on performances of the entire class (4.0), and for individual students (3.8).

CONCLUSION: Clerkship directors identified test content that was relevant to the curricular content and important for clerkship students to know, and they indicated a desired question distribution. They would most like feedback on their students' performance by organ system-based sections for the complete academic year. This collaborative effort could serve as a model for aligning national exams with course goals.