The fragility of statistically significant findings from randomised controlled trials in the urological literature

Vikram M Narayan, Shreyas Gandhi, Kristin Chrouser, Nathan Evaniew, Philipp Dahm
BJU International 2018, 122 (1): 160-166

OBJECTIVES: To use the Fragility Index to evaluate the robustness of statistically significant findings from urological randomised controlled trials (RCTs).

MATERIALS AND METHODS: The 'Fragility Index' is defined as the minimum number of patients in one arm of a trial whose status would have to change from 'event' to 'non-event', such that a statistically significant result becomes non-significant. We identified all RCTs published in four major urology journals between 2011 and 2015, and we determined the Fragility Index values for those trials reporting statistically significant results of dichotomous outcomes using the Fisher's exact test.

RESULTS: In all, 332 RCTs were identified, and 41 studies met the inclusion criteria. The median (interquartile range) Fragility Index was 3 (1, 4.5), indicating that an addition of only three alternate events to one arm of a typical trial would have eliminated its statistical significance. In 27/40 cases (67.5% of cases), the number of patients lost to follow-up was larger than its Fragility Index.

CONCLUSIONS: The results of urology RCTs that study dichotomous outcomes and report statistically significant differences between groups are sometimes fragile and depend on few events. Urologists should interpret these RCTs cautiously, particularly when the number of participants lost to follow-up exceeds the Fragility Index. Routine reporting of Fragility Index values alongside P values may provide additional guidance about the robustness of statistically significant findings.


You are not logged in. Sign Up or Log In to join the discussion.

Related Papers

Available on the App Store

Available on the Play Store
Remove bar
Read by QxMD icon Read

Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"