Testing a rating scale of video-taped consultations to assess performance of trainee nurse practitioners in general practice.

BACKGROUND: Nurse practitioners (NPs) in the United Kingdom are taking on some of the consultation work previously done by general practitioners (GPs) without there being any established professional standards that they must achieve before doing so. There is a need to develop and test methods of assessing their consultation performance for reasons of professional accreditation and patient safety.

AIMS: 1. To make independent summative assessments of trainee nurse practitioners' (TNPs) consultation performance. 2. To assess the validity and reliability of an existing video-taped assessment tool.

METHOD: Four TNPs taking part in the EROS (extended roles of staff) study video recorded seven or eight consecutive consultations with typical patients during one surgery. Each consultation was rated nine times by members of a panel comprising eight independent GP trainers, four NPs and the GPs and TNPs in the EROS practices. A rating scale developed by Cox & Mulholland was used for the purpose.

RESULTS: Eight of the 37 items and four consultations had more than 10% missing data, mean = 7.7 items per rater. Factor analysis yielded a single factor solution explaining 32.6% of the variance and indicated that items could be summed to provide a single score. Internal consistency was high, alpha coefficient = 0.92. Individual differences between raters in scoring consultations were taken into account in providing a score for each consultation. Scores obtained were found to cluster at the positive end of the distribution indicating a high level of performance. Greater differences were found between scorers than between consultations.

CONCLUSIONS: This instrument is appropriate for scoring NP consultations and this small sample was rated as showing a uniformly high standard of performance. Some items could be deleted since they do not feature in the range of consultations currently performed. If this or a similar tool was to be adopted more widely for summative rating purposes then it should be tested rigorously for validity and reliability, training should be given to raters and criteria provided by which to make judgements.