The Fragility of Statistically Significant Results in Pediatric Orthopaedic Randomized Controlled Trials as Quantified by the Fragility Index: A Systematic Review

Sariah Khormaee, Judy Choe, Joseph J Ruzbarsky, Kunal N Agarwal, John S Blanco, Shevaun M Doyle, Emily R Dodwell
Journal of Pediatric Orthopedics 2018, 38 (8): e418-e423

BACKGROUND: The randomized controlled trial (RCT) is the gold standard study design allowing critical comparison of clinical outcomes while minimizing bias. Traditionally clinical trials are evaluated through statistical significance, expressed by P-values and confidence intervals. However, until recently, the robustness of a study's conclusions has been given little attention. A new metric, the fragility index, quantifies the number of patients theoretically required to switch outcomes in order to reverse the study conclusions. The primary aim of our work was to determine the fragility index of RCTs in the pediatric orthopaedic literature. The secondary aim was to determine study factors associated with lower fragility index.

METHODS: Pubmed and Embase were systematically searched for pediatric orthopaedic RCTs published September 1, 2006 to September 1, 2016. Two independent reviewers screened titles, abstracts, and manuscripts to identify studies published in English involving 2 treatment arms. Trials without dichotomous primary or secondary outcomes or with patients >18 years were excluded. Data were extracted from each eligible article in duplicate and the fragility index was determined using Fisher exact test, with previously published methods. Univariate analysis was used to determine factors associated with lower fragility index.

RESULTS: Seventeen trials were eligible for inclusion. The median treatment arm size was 58 and overall sample size was 116 patients. The median fragility index was 3 (range, 0 to 18). A fragility index of 3 means that just 3 patients would need to switch treatment outcomes in order for the trial results to become statistically nonsignificant. In 1 study, the number of patients lost to follow-up exceeded the fragility index, such that the study conclusions could be completely reversed purely depending on the outcomes of the patients lost to follow-up. Lower fragility index was associated with smaller patient sample sizes and greater P-values.

CONCLUSIONS: The fragility index is a useful adjunct metric to the P-value and confidence intervals, allowing analysis of the robustness of study conclusions. RCTs in pediatric orthopaedics often have small sample sizes, many with low fragility indices. Future efforts could focus on encouraging institutional collaboration and patient recruitment with the ultimate goal of improving RCT sample sizes, and potentially improving the robustness of RCT results.


Full Text Links

Find Full Text Links for this Article


You are not logged in. Sign Up or Log In to join the discussion.

Trending Papers

Remove bar
Read by QxMD icon Read

Save your favorite articles in one place with a free QxMD account.


Search Tips

Use Boolean operators: AND/OR

diabetic AND foot
diabetes OR diabetic

Exclude a word using the 'minus' sign

Virchow -triad

Use Parentheses

water AND (cup OR glass)

Add an asterisk (*) at end of a word to include word stems

Neuro* will search for Neurology, Neuroscientist, Neurological, and so on

Use quotes to search for an exact phrase

"primary prevention of cancer"
(heart or cardiac or cardio*) AND arrest -"American Heart Association"