Fragility of clinical trials across research fields: A synthesis of methodological reviews

Matthew Holek, Faris Bdair, Mohammed Khan, Michael Walsh, P J Devereaux, Stephen D Walter, Lehana Thabane, Lawrence Mbuagbaw
Contemporary Clinical Trials 2020 September 14, : 106151

BACKGROUND: Randomized controlled trials (RCTs) are often used to inform clinical practice and it is desirable that their results be robust. A fragility index (FI), defined as the smallest number of participants in whom an outcome change from non-event to event would turn a statistically significant result to a non-significant result, can be computed to measure robustness. We sought to determine the distribution of fragility indices across various research areas and summarized the factors associated with fragility.

METHODS: We searched PubMed between February 2014 and May 2019 and included reviews that reported on fragility indices and the associated factors. Two investigators independently screened articles for eligibility and extracted all relevant data from each review. Fragility indices were pooled using random effects meta-analysis.

RESULTS: Twenty-four (24) reviews met the inclusion criteria. They contained a median of 41 trials (first quartile [Q1]-third quartile [Q3]: 17-120). The overall mean FI across different fields of research was 4 (95% confidence interval [CI] 3-5), indicating a high level of fragility. Higher journal impact factor, larger sample size, bigger effect size, more outcome events, a lower p-value, and adequate allocation concealment were reported to be associated with the higher FI. The ecological correlation between median FI and median sample size (22 studies) was 0.95 (95% CI0.58-0.99).

CONCLUSION: Trials across various fields of research are frequently fragile. We also identified some factors associated with fragility. Researchers should consider strategies to enhance the robustness of studies and minimize fragility.

Full Text Links

Find Full Text Links for this Article


You are not logged in. Sign Up or Log In to join the discussion.

Related Papers

Remove bar
Read by QxMD icon Read

Save your favorite articles in one place with a free QxMD account.


Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"