The effect of changing disease risk on clinical reasoning.

OBJECTIVE: To assess the ability of health care professionals to evaluate the effect of clinical test results in different settings.

DESIGN: Subjects were presented with a series of generic clinical scenarios in which information about the test performance and the pretest probability of disease was varied. The subject estimates of posttest probability were compared with those calculated on the basis of Bayes' theorem.

PARTICIPANTS: Fifty health care professionals, including 31 physicians and 19 nonphysicians, associated with a university teaching hospital.

MEASUREMENTS AND MAIN RESULTS: Under a variety of testing conditions, both the physicians and the nonphysicians inaccurately estimated the posttest probability of disease. Based on a logarithmic transformation, the error in probability estimation was divided into a portion related to the pretest probability of disease and a portion related to the test performance. Most of the error in posttest probability estimation was associated with the incorrect use of pretest probabilities. The subjects consistently overestimated the posttest probability of disease expected under Bayes' theorem, with increasing error associated with decreasing pretest probability. Physician estimates of posttest probability increased with increasing likelihood ratios for each scenario. Nonphysician estimates of posttest probabilities increased with increasing likelihood ratios for a positive test, but the estimates associated with a negative test result were inconsistent.

CONCLUSIONS: Physicians and nonphysicians overestimate posttest probabilities with increasing error associated with decreasing disease risk. Some nonphysicians may not fully understand the effect of test performance on risk estimation, particularly in the setting of a negative test. Health care professionals should receive training in the proper evaluation of test information, with particular emphasis on the influence of pretest disease risk on the posttest probability of disease.