Comparative Analysis of Two Methods of Perceptual Voice Assessment.

OBJECTIVES: The primary aim was to compare two methods for perceptual evaluation of voice - paired comparison (PC) and visual analog scale (VAS) ratings. Secondary aims were to assess the correspondence between two dimensions of voice- overall severity of voice quality and resonant voice, and to investigate the influence of rater experience on perceptual rating scores and rating confidence scores.

STUDY DESIGN: Experimental design.

METHODS: Voice samples from six children (pre and post therapy) were rated by 15 Speech-Language Pathologists specialized in voice. Raters completed four tasks corresponding to the two rating methods and voice qualities: PC-severity, PC-resonance, VAS-severity, and VAS-resonance. For PC tasks, raters chose the better of two voice samples (better voice quality or better resonance, depending on the task) and indicated the degree of confidence in each choice. Rating and confidence score were combined to produce a number on a 1-10 scale (PC-confidence adjusted). VAS ratings involved rating voices on a scale for degree of severity and resonance, respectively.

RESULTS: PC-confidence adjusted and VAS ratings were moderately correlated for overall severity and also vocal resonance. VAS ratings were normally distributed and had greater rater consistency than PC-confidence adjusted ratings. VAS scores reliably predicted binary PC choices (choice of voice sample only). Overall severity and vocal resonance were weakly correlated and rater experience was not linearly related to rating scores or confidence.

CONCLUSIONS: Results suggest that the VAS rating method holds advantages over PC, including normally distributed ratings, superior consistency of ratings, and the ability to provide more finely grained detail regarding the auditory perception of voice. Overall severity and vocal resonance were not redundant in the current data set, suggesting that resonant voice and overall severity are not isomorphic. Finally, the number of years of clinical experience was not linearly related to perceptual ratings or rating confidence.