Research Synthesis Methods

A random-effects model is often applied in meta-analysis when considerable heterogeneity among studies is observed due to the differences in patient characteristics, timeframe, treatment regimens, and other study characteristics. Since 2014, the journals Research Synthesis Methods and the Annals of Internal Medicine have published a few noteworthy papers that explained why the most widely used method for pooling heterogeneous studies-the DerSimonian-Laird (DL) estimator-can produce biased estimates with falsely high precision and recommended to use other several alternative methods...

Qualitative comparative analysis (QCA) is a hybrid method designed to bridge the gap between qualitative and quantitative research in a case-sensitive approach that considers each case holistically as a complex configuration of conditions and outcomes. QCA allows for multiple conjunctural causation, implying that it is often a combination of conditions that produces an outcome, that multiple pathways may lead to the same outcome, and that in different contexts, the same condition may have a different impact on the outcome...

Literature screening is the process of identifying all relevant records from a pool of candidate paper records in systematic review, meta-analysis, and other research synthesis tasks. This process is time consuming, expensive, and prone to human error. Screening prioritization methods attempt to help reviewers identify most relevant records while only screening a proportion of candidate records with high priority. In previous studies, screening prioritization is often referred to as automatic literature screening or automatic literature identification...

This scoping review aims to identify and systematically review published mapping reviews to assess their commonality and heterogeneity and determine whether additional efforts should be made to standardise methodology and reporting. The following databases were searched; Ovid MEDLINE, Embase, CINAHL, PsycINFO, Campbell collaboration database, Social Science Abstracts, Library and Information Science Abstracts (LISA). Following a pilot-test on a random sample of 20 citations included within title and abstracts, two team members independently completed all screening...

This study builds on the simulation framework of a recent paper by Stanley and Doucouliagos (Research Synthesis Methods 2023;14;515-519). S&D use simulations to make the argument that meta-analyses using partial correlation coefficients (PCCs) should employ a "suboptimal" estimator of the PCC standard error when constructing weights for fixed effect and random effects estimation. We address concerns that their simulations and subsequent recommendation may give meta-analysts a misleading impression. While the estimator they promote dominates the "correct" formula in their Monte Carlo framework, there are other estimators that perform even better...

In Bayesian random-effects meta-analysis, the use of weakly informative prior distributions is of particular benefit in cases where only a few studies are included, a situation often encountered in health technology assessment (HTA). Suggestions for empirical prior distributions are available in the literature but it is unknown whether these are adequate in the context of HTA. Therefore, a database of all relevant meta-analyses conducted by the Institute for Quality and Efficiency in Health Care (IQWiG, Germany) was constructed to derive empirical prior distributions for the heterogeneity parameter suitable for HTA...

External validity is an important parameter that needs to be considered for decision making in health research, but no widely accepted measurement tool for the assessment of external validity of randomized controlled trials (RCTs) exists. One of the most limiting factors for creating such a tool is probably the substantial heterogeneity and lack of consensus in this field. The objective of this study was to reach consensus on a definition of external validity and on criteria to assess the external validity of RCTs included in systematic reviews...

The amount of grey literature and 'softer' intelligence from social media or websites is vast. Given the long lead-times of producing high-quality peer-reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health-related grey literature and soft data, with a focus on (semi)automating horizon scans, health technology assessments (HTA), evidence maps, or other literature reviews...

Outcomes of meta-analyses are increasingly used to inform evidence-based decision making in various research fields. However, a number of recent studies have reported rapid temporal changes in magnitude and significance of the reported effects which could make policy-relevant recommendations from meta-analyses to quickly go out of date. We assessed the extent and patterns of temporal trends in magnitude and statistical significance of the cumulative effects in meta-analyses in applied ecology and conservation published between 2004 and 2018...

Simulated treatment comparison (STC) is an established method for performing population adjustment for the indirect comparison of two treatments, where individual patient data (IPD) are available for one trial but only aggregate level information is available for the other. The most commonly used method is what we call 'standard STC'. Here we fit an outcome model using data from the trial with IPD, and then substitute mean covariate values from the trial where only aggregate level data are available, to predict what the first of these trial's outcomes would have been if its population had been the same as the second...

The trace plot is seldom used in meta-analysis, yet it is a very informative plot. In this article, we define and illustrate what the trace plot is, and discuss why it is important. The Bayesian version of the plot combines the posterior density of <mml:math xmlns:mml="https://www.w3.org/1998/Math/MathML"> <mml:semantics><mml:mrow><mml:mi>τ</mml:mi></mml:mrow> <mml:annotation>$$ \tau $$</mml:annotation></mml:semantics> </mml:math> , the between-study standard deviation, and the shrunken estimates of the study effects as a function of <mml:math xmlns:mml="https://www...

The literature search underpins data collection for all systematic reviews (SRs). The SR reporting guideline PRISMA, and its extensions, aim to facilitate research transparency and reproducibility, and ultimately improve the quality of research, by instructing authors to provide specific research materials and data upon publication of the manuscript. Search strategies are one item of data that are explicitly included in PRISMA and the critical appraisal tool AMSTAR2. Yet some authors use search availability statements implying that the search strategies are available upon request instead of providing strategies up front...

When performing an aggregate data meta-analysis of a continuous outcome, researchers often come across primary studies that report the sample median of the outcome. However, standard meta-analytic methods typically cannot be directly applied in this setting. In recent years, there has been substantial development in statistical methods to incorporate primary studies reporting sample medians in meta-analysis, yet there are currently no comprehensive software tools implementing these methods. In this paper, we present the metamedian R package, a freely available and open-source software tool for meta-analyzing primary studies that report sample medians...

Data extraction is a time-consuming and resource-intensive task in the systematic review process. Natural language processing (NLP) artificial intelligence (AI) techniques have the potential to automate data extraction saving time and resources, accelerating the review process, and enhancing the quality and reliability of extracted data. In this paper, we propose a method for using Bing AI and Microsoft Edge as a second reviewer to verify and enhance data items first extracted by a single human reviewer. We describe a worked example of the steps involved in instructing the Bing AI Chat tool to extract study characteristics as data items from a PDF document into a table so that they can be compared with data extracted manually...

Predatory journals are a blemish on scholarly publishing and academia and the studies published within them are more likely to contain data that is false. The inclusion of studies from predatory journals in evidence syntheses is potentially problematic due to this propensity for false data to be included. To date, there has been little exploration of the opinions and experiences of evidence synthesisers when dealing with predatory journals in the conduct of their evidence synthesis. In this paper, the thoughts, opinions, and attitudes of evidence synthesisers towards predatory journals and the inclusion of studies published within these journals in evidence syntheses were sought...

Drug repurposing refers to the process of discovering new therapeutic uses for existing medicines. Compared to traditional drug discovery, drug repurposing is attractive for its speed, cost, and reduced risk of failure. However, existing approaches for drug repurposing involve complex, computationally-intensive analytical methods that are not widely used in practice. Instead, repurposing decisions are often based on subjective judgments from limited empirical evidence. In this article, we develop a novel Bayesian network meta-analysis (NMA) framework that can predict the efficacy of an approved treatment in a new indication and thereby identify candidate treatments for repurposing...

Checking for possible inconsistency between direct and indirect evidence is an important task in network meta-analysis. Recently, an evidence-splitting (ES) model has been proposed, that allows separating direct and indirect evidence in a network and hence assessing inconsistency. A salient feature of this model is that the variance for heterogeneity appears in both the mean and the variance structure. Thus, full maximum likelihood (ML) has been proposed for estimating the parameters of this model. Maximum likelihood is known to yield biased variance component estimates in linear mixed models, and this problem is expected to also affect the ES model...

No abstract text is available yet for this article.

A recent paper proposed an alternative weighting scheme when performing matching-adjusted indirect comparisons. This alternative approach follows the conventional one in matching the covariate means across two studies but differs in that it maximizes the effective sample size when doing so. The appendix of this paper showed, assuming there is one covariate and negative weights are permitted, that the resulting weights are linear in the covariates. This explains how the alternative method achieves a larger effective sample size and results in a metric that quantifies the difficulty of matching on particular covariates...

This study aimed to assess the methods and outcomes of The Measurement Tool to Assess systematic Reviews (AMSTAR) 2 appraisals in overviews of reviews (overviews) of interventions in the cardiovascular field and identify factors that are associated with these outcomes. MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched until November 2022. Eligible were overviews of cardiovascular interventions, analyzing systematic reviews (SRs) of randomized controlled trials (RCTs). Extracted data included characteristics of overviews and SRs and AMSTAR 2 appraisal methods and outcomes...