Fragility of Results in Ophthalmology Randomized Controlled Trials: A Systematic Review

Carl Shen, Isabel Shamsudeen, Forough Farrokhyar, Kourosh Sabri
Ophthalmology 2018, 125 (5): 642-648

PURPOSE: Evidence-based medicine is guided by our interpretation of randomized controlled trials (RCTs) that address important clinical questions. Evaluation of the robustness of statistically significant outcomes adds a crucial element to the global assessment of trial findings. The purpose of this systematic review was to determine the robustness of ophthalmology RCTs through application of the Fragility Index (FI), a novel metric of the robustness of statistically significant outcomes.

DESIGN: Systematic review.

METHODS: A literature search (MEDLINE) was performed for all RCTs published in top ophthalmology journals and ophthalmology-related RCTs published in high-impact journals in the past 10 years. Two reviewers independently screened 1811 identified articles for inclusion if they (1) were a human ophthalmology-related trial, (2) had a 1:1 prospective study design, and (3) reported a statistically significant dichotomous outcome in the abstract. All relevant data, including outcome, P value, number of patients in each group, number of events in each group, number of patients lost to follow-up, and trial characteristics, were extracted. The FI of each RCT was calculated and multivariate regression applied to determine predictive factors.

RESULTS: The 156 trials had a median sample size of 91.5 (range, 13-2593) patients/eyes, and a median of 28 (range, 4-2217) events. The median FI of the included trials was 2 (range, 0-48), meaning that if 2 non-events were switched to events in the treatment group, the result would lose its statistical significance. A quarter of all trials had an FI of 1 or less, and 75% of trials had an FI of 6 or less. The FI was less than the number of missing data points in 52.6% of trials. Predictive factors for FI by multivariate regression included smaller P value (P < 0.001), larger sample size (P = 0.001), larger number of events (P = 0.011), and journal impact factor (P = 0.029).

CONCLUSIONS: In ophthalmology trials, statistically significant dichotomous results are often fragile, meaning that a difference of only a couple of events can change the statistical significance. An application of the FI in RCTs may aid in the interpretation of results and assessment of quality of evidence.


You are not logged in. Sign Up or Log In to join the discussion.

Related Papers

Available on the App Store

Available on the Play Store
Remove bar
Read by QxMD icon Read

Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"