An evidence rating service provided valid correlates of the clinical importance of medical articles and journals.

OBJECTIVES: To determine reliability and validity of McMaster PLUS measures of scientific merit and clinical importance of articles in medical journals.

STUDY DESIGN AND SETTING: Analytic survey of peer-reviewed medical journals. Articles qualified for inclusion by meeting: 1) scientific criteria and 2) a clinical importance rating threshold. Included articles were sent as e-mail alerts to physicians according to their clinical interests. Internal measures included the number of high-quality, clinically important studies published in source journals and response to alerts. For external validation, we correlated internal measures with the Journal Impact Factor (JIF) and citation in DynaMed Plus (DMPC).

RESULTS: We evaluated 34,232 articles from 57 journals. Inclusion criteria were met by 2638 articles (7.71%). The number of qualifying articles per journal was correlated with the number of articles with high clinical importance ratings (r 0.96, p<0.001), article alert clicks (r 0.86, p<0.001), and DMPC (r 0.99, p< 0.001). Correlation was lower with the JIF (r 0.68, p<0.01).

CONCLUSIONS: Measures of scientific merit and clinical importance of medical journal articles were strongly correlated with each other, less so with journal impact factors. Journals varied widely by these measures but, generally, few articles were both scientifically sound and clinically important.