Dataset Information

Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews.

ABSTRACT:

Background

Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap.

Methods

We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use.

Results

Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview - the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview.

Conclusions

Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.

SUBMITTER: Lunny C

PROVIDER: S-EPMC8265144 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionOne of the most conflicting methodological issues when conducting an overview is the overlap of primary studies across systematic reviews (SRs). Overlap in the pooled effect estimates across SRs may lead to overly precise effect estimates in the overview. SRs that focus on exercise-related interventions are often included in overviews aimed at grouping and determining the effectiveness of various interventions for managing specific health conditions. The aim of this systematic methodological review is to describe the strategies used by authors of overviews focusing on exercise-related interventions to manage the overlap of primary studies.Methods and analysisA comprehensive search strategy has been developed for different databases and their platforms. The databases to be consulted will be MEDLINE (Ovid), Embase (Ovid), The Cochrane Database of Systematic Reviews (Cochrane Library) and Epistemonikos. Two reviewers will independently screen the records identified through the search strategy and extract the information from the included overviews. The frequency and the type of overlap management strategies of the primary studies included in the SRs will be considered as the main outcome. In addition, the recognition of the lack of use of any overlap management strategy and the congruence between planning and conducting the overview focusing on overlap management strategies will be assessed. A subgroup analysis will be carried out according to the journal impact factor, year of publication and compliance with the Preferred Reporting Items for Overviews of Reviews statement.Ethics and disseminationThis study will not involve human subjects and therefore does not require ethics committee approval. However, the conduct and reporting of the findings of this review will be conducted in a rigorous, systematic and transparent manner, which relates to research ethics.The findings of this review will be presented at scientific conferences and published as one or more studies in peer-review scientific journals related to rehabilitation or research methods.

Project description:BACKGROUND:Despite the growing reputation and subject coverage of the Cochrane Database of Systematic Reviews, many systematic reviews continue to be published solely in paper-based health care journals. This study was designed to determine why authors choose to publish their systematic reviews outside of the Cochrane Collaboration and if they might be interested in converting their reviews to Cochrane format for publication in the Cochrane Database of Systematic Reviews. METHODS:Cross-sectional survey of Australian primary authors of systematic reviews not published on the Cochrane Database of Systematic Reviews identified from the Database of Abstracts of Reviews of Effectiveness. RESULTS:We identified 88 systematic reviews from the Database of Abstracts of Reviews of Effectiveness with an Australian as the primary author. We surveyed 52 authors for whom valid contact information was available. The response rate was 88 per cent (46/52). Ten authors replied without completing the survey, leaving 36 valid surveys for analysis. The most frequently cited reasons for not undertaking a Cochrane review were: lack of time (78%), the need to undergo specific Cochrane training (46%), unwillingness to update reviews (36%), difficulties with the Cochrane process (26%) and the review topic already registered with the Cochrane Collaboration (21%). (Percentages based on completed responses to individual questions.) Nearly half the respondents would consider converting their review to Cochrane format. Dedicated time emerged as the most important factor in facilitating the potential conversion process. Other factors included navigating the Cochrane system, assistance with updating and financial support. Eighty-six per cent were willing to have their review converted to Cochrane format by another author. CONCLUSION:Time required to complete a Cochrane review and the need for specific training are the primary reasons why some authors publish systematic reviews outside of the Cochrane Collaboration. Encouragingly, almost half of the authors would consider converting their review to Cochrane format. Based on the current number of reviews in the Database of Abstracts of Reviews of Effectiveness, this could result in more than 700 additional Cochrane reviews. Ways of supporting these authors and how to provide dedicated time to convert systematic reviews needs further consideration.

Project description:BackgroundOverviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews-for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews.MethodsWe selected seven overviews of healthcare interventions and included all SRs meeting each overview's inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions.ResultsNinety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; "almost perfect" vs. "substantial" using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions.ConclusionsResults indicate that the AMSTAR tool can be used successfully in overviews that include Cochrane and non-Cochrane SRs, though decision rules may be useful to circumvent common challenges. Findings support existing recommendations that quality assessments of SRs in overviews be conducted independently, in duplicate, with a process for consensus. Results also suggest that using methodological quality to guide inclusion decisions (e.g., to exclude poorly conducted and reported SRs) may not introduce bias into the overview process.

Project description:Previous research shows that many authors of Cochrane overviews were also involved in some of the included systematic reviews (SRs). This type of dual (co-)authorship (DCA) may be a conflict of interest and a potential source of bias. Our objectives were to (1) additionally investigate DCA in non-Cochrane overviews; (2) investigate whether there is an association between DCA and quality assessments of SRs in Cochrane and non-Cochrane overviews.We selected a sample of Cochrane (n =?20) and non-Cochrane (n =?78) overviews for analysis. We extracted data on the number of reviews affected by DCA and whether quality assessment of included reviews was conducted independently. Differences in mean quality scores between SRs with and without DCA were calculated in each overview. These differences were standardized (using the standardized mean difference (SMD)) and meta-analyzed using a random effects model.Forty out of 78 non-Cochrane overviews (51%) and 18 out of 20 Cochrane overviews (90%) had included at least one SR with DCA. For Cochrane overviews, a median of 5 [interquartile range (IQR) 2.5 to 7] SRs were affected by DCA (median of included reviews 10). For non-Cochrane overviews a median of 1 [IQR 0 to 2] of the included SRs were affected (median of included reviews 14). The meta-analysis showed a SMD of 0.58 (95% confidence interval (CI) 0.27 to 0.90) indicating higher quality scores in reviews with overlapping authors. The test for subgroup differences shows no evidence of a difference between Cochrane (SMD 0.44; 95% CI 0.07 to 0.81) and non-Cochrane overviews (SMD 0.62; 95% CI 0.06 to 1.17).Many authors of overviews also often have an authorship on one or more of the underlying reviews. Our analysis shows that, on average, authors of overviews give higher quality ratings to SRs in which they were involved themselves than to other SRs. Conflict of interest is one explanation, but there are several others such as reviewer expertise. Independent and blinded reassessments of the reviews would provide more robust evidence on potential bias arising from DCA.