A new taxonomy was developed for overlap across 'overviews of systematic reviews': A meta-research study of research waste.
Ontology highlight
ABSTRACT:
SUBMITTER: Lunny C
PROVIDER: S-EPMC9303867 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundOverviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap.MethodsWe first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use.ResultsNine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview - the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview.ConclusionsOur study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.
| S-EPMC8265144 | biostudies-literature
Project description:BACKGROUND:A standard or consensus definition of a systematic review does not exist. Therefore, if there is no definition about a systematic review in secondary studies that analyse them or the definition is too broad, inappropriate studies might be included in such evidence synthesis. The aim of this study was to analyse the definition of a systematic review (SR) in health care literature, elements of the definitions that are used and to propose a starting point for an explicit and non-ambiguous SR definition. METHODS:We included overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks. We extracted the definitions of SRs, as well as the inclusion and exclusion criteria that could indicate which definition of a SR the authors used. We extracted individual elements of SR definitions, categorised and quantified them. RESULTS:Among the 535 analysed sources of information, 188 (35%) provided a definition of a SR. The most commonly used reference points for the definitions of SRs were Cochrane and the PRISMA statement. We found 188 different elements of SR definitions and divided them into 14 categories. The highest number of SR definition elements was found in categories related to searching (N?=?51), analysis/synthesis (N?=?23), overall methods (N?=?22), quality/bias/appraisal/validity (N?=?22) and aim/question (N?=?13). The same five categories were also the most commonly used combination of categories in the SR definitions. CONCLUSION:Currently used definitions of SRs are vague and ambiguous, often using terms such as clear, explicit and systematic, without further elaboration. In this manuscript we propose a more specific definition of a systematic review, with the ultimate aim of motivating the research community to establish a clear and unambiguous definition of this type of research.
| S-EPMC6829801 | biostudies-literature
Project description:BackgroundWhen conducting an Overviews of Reviews on health-related topics, it is unclear which combination of bibliographic databases authors should use for searching for SRs. Our goal was to determine which databases included the most systematic reviews and identify an optimal database combination for searching systematic reviews.MethodsA set of 86 Overviews of Reviews with 1219 included systematic reviews was extracted from a previous study. Inclusion of the systematic reviews was assessed in MEDLINE, CINAHL, Embase, Epistemonikos, PsycINFO, and TRIP. The mean inclusion rate (% of included systematic reviews) and corresponding 95% confidence interval were calculated for each database individually, as well as for combinations of MEDLINE with each other database and reference checking.ResultsInclusion of systematic reviews was higher in MEDLINE than in any other single database (mean inclusion rate 89.7%; 95% confidence interval [89.0-90.3%]). Combined with reference checking, this value increased to 93.7% [93.2-94.2%]. The best combination of two databases plus reference checking consisted of MEDLINE and Epistemonikos (99.2% [99.0-99.3%]). Stratification by Health Technology Assessment reports (97.7% [96.5-98.9%]) vs. Cochrane Overviews (100.0%) vs. non-Cochrane Overviews (99.3% [99.1-99.4%]) showed that inclusion was only slightly lower for Health Technology Assessment reports. However, MEDLINE, Epistemonikos, and reference checking remained the best combination. Among the 10/1219 systematic reviews not identified by this combination, five were published as websites rather than journals, two were included in CINAHL and Embase, and one was included in the database ERIC.ConclusionsMEDLINE and Epistemonikos, complemented by reference checking of included studies, is the best database combination to identify systematic reviews on health-related topics.
| S-EPMC7268249 | biostudies-literature
Project description:BackgroundResults of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.MethodsWe searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.ResultsWe included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.ConclusionOur systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.Systematic review registrationOpen Science registration number https://osf.io/8gkzu/.
| S-EPMC9446778 | biostudies-literature
Project description:BackgroundOverviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews-for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews.MethodsWe selected seven overviews of healthcare interventions and included all SRs meeting each overview's inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions.ResultsNinety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; "almost perfect" vs. "substantial" using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions.ConclusionsResults indicate that the AMSTAR tool can be used successfully in overviews that include Cochrane and non-Cochrane SRs, though decision rules may be useful to circumvent common challenges. Findings support existing recommendations that quality assessments of SRs in overviews be conducted independently, in duplicate, with a process for consensus. Results also suggest that using methodological quality to guide inclusion decisions (e.g., to exclude poorly conducted and reported SRs) may not introduce bias into the overview process.
| S-EPMC5364717 | biostudies-literature
Project description:BackgroundSeveral studies have documented the production of wasteful research, defined as research of no scientific importance and/or not meeting societal needs. We argue that this redundancy in research may to a large degree be due to the lack of a systematic evaluation of the best available evidence and/or of studies assessing societal needs.ObjectivesThe aim of this scoping review is to (A) identify meta-research studies evaluating if redundancy is present within biomedical research, and if so, assessing the prevalence of such redundancy, and (B) to identify meta-research studies evaluating if researchers had been trying to minimise or avoid redundancy.Eligibility criteriaMeta-research studies (empirical studies) were eligible if they evaluated whether redundancy was present and to what degree; whether health researchers referred to all earlier similar studies when justifying and designing a new study and/or when placing new results in the context of earlier similar trials; and whether health researchers systematically and transparently considered end users' perspectives when justifying and designing a new study.Sources of evidenceThe initial overall search was conducted in MEDLINE, Embase via Ovid, CINAHL, Web of Science, Social Sciences Citation Index, Arts & Humanities Citation Index, and the Cochrane Methodology Register from inception to June 2015. A 2nd search included MEDLINE and Embase via Ovid and covered January 2015 to 26 May 2021. No publication date or language restrictions were applied.Charting methodsCharting methods included description of the included studies, bibliometric mapping, and presentation of possible research gaps in the identified meta-research.ResultsWe identified 69 meta-research studies. Thirty-four (49%) of these evaluated the prevalence of redundancy and 42 (61%) studies evaluated the prevalence of a systematic and transparent use of earlier similar studies when justifying and designing new studies, and/or when placing new results in context, with seven (10%) studies addressing both aspects. Only one (1%) study assessed if the perspectives of end users had been used to inform the justification and design of a new study. Among the included meta-research studies evaluating whether redundancy was present, only two of nine health domains (medical areas) and only two of 10 research topics (different methodological types) were represented. Similarly, among the included meta-research studies evaluating whether researchers had been trying to minimise or avoid redundancy, only one of nine health domains and only one of 10 research topics were represented.Conclusions that relate to the review questions and objectivesEven with 69 included meta-research studies, there was a lack of information for most health domains and research topics. However, as most included studies were evaluating across different domains, there is a clear indication of a high prevalence of redundancy and a low prevalence of trying to minimise or avoid redundancy. In addition, only one meta-research study evaluated whether the perspectives of end users were used to inform the justification and design of a new study.Systematic review registrationProtocol registered at Open Science Framework: https://osf.io/3rdua/ (15 June 2021).
| S-EPMC9667610 | biostudies-literature
Project description:ObjectiveTo develop ROBIS, a new tool for assessing the risk of bias in systematic reviews (rather than in primary studies).Study design and settingWe used four-stage approach to develop ROBIS: define the scope, review the evidence base, hold a face-to-face meeting, and refine the tool through piloting.ResultsROBIS is currently aimed at four broad categories of reviews mainly within health care settings: interventions, diagnosis, prognosis, and etiology. The target audience of ROBIS is primarily guideline developers, authors of overviews of systematic reviews ("reviews of reviews"), and review authors who might want to assess or avoid risk of bias in their reviews. The tool is completed in three phases: (1) assess relevance (optional), (2) identify concerns with the review process, and (3) judge risk of bias. Phase 2 covers four domains through which bias may be introduced into a systematic review: study eligibility criteria; identification and selection of studies; data collection and study appraisal; and synthesis and findings. Phase 3 assesses the overall risk of bias in the interpretation of review findings and whether this considered limitations identified in any of the phase 2 domains. Signaling questions are included to help judge concerns with the review process (phase 2) and the overall risk of bias in the review (phase 3); these questions flag aspects of review design related to the potential for bias and aim to help assessors judge risk of bias in the review process, results, and conclusions.ConclusionsROBIS is the first rigorously developed tool designed specifically to assess the risk of bias in systematic reviews.
| S-EPMC4687950 | biostudies-literature
Project description:ObjectiveThe objective of this study was to analyze the 100 top-cited systematic reviews/meta-analyses on diabetic research.MethodsThe Science Citation Index Expanded database was searched to identify top-cited studies on diabetic research up to March 4th, 2020. Studies were analyzed using the following characteristics: citation number, publication year, country and institution of origin, authorship, topics, and journals.ResultsThe 100 top-cited diabetic systematic reviews/meta-analyses were published in 43 different journals, with Diabetes Care having the highest numbers (n = 17), followed by The Journal of the American Medical Association (n = 14) and Lancet (n = 9). The majority of studies are published in the 2000s. The number of citations ranged from 2197 to 301. The highest number of contributions was from the USA, followed by England and Australia. The leading institution was Harvard University. The hot topic was a risk factor (n = 33), followed by comorbidity (n = 27).ConclusionsThe 100 top-cited systematic reviews/meta-analyses on diabetic research identify impactful authors, journals, institutes, and countries. It will also provide the most important references to evidence-based medicine in diabetes and serve as a guide to the features of a citable paper in this field.
| S-EPMC7509559 | biostudies-literature
Project description:BackgroundRequiring authors to base their research on a systematic review of the existing literature prevents the generation of redundant scientific studies, thereby avoiding the deprivation of effective therapies for trial participants and the waste of research funds. Scientific medical journals could require this in their author guidelines. While this applies to all areas of research, it is also relevant to physiotherapy and rehabilitation research, which predominantly involve interventional trials in patients.ObjectiveThe aim of this study was to determine the extent to which the use of systematic reviews to justify a new trial is already being requested by physiotherapy-related scientific medical journals (PTJs). In addition, a comparison was made between PTJs and scientific medical journals with the highest impact factor in the Science Citation Index Extended (SCIE).MethodsThis meta-research study is based on a systematic examination of the author guidelines of 149 PTJs. The journals were identified and included based on the number of publications with physiotherapy as a keyword in the databases PEDro, and Medline (Pubmed). The included author guidelines were analysed for the extent to which they specified that a new trial should be justified by a systematic review of the literature. Additionally, they were compared with 14 scientific medical journals with the highest impact factor in the SCIE (LJs).ResultsIn their author guidelines, none of the included PTJs required or recommended the use of a systematic review to justify a new trial. Among LJs, four journals (28.57%), all associated with the Lancet group, required the study justification through a systematic review of the literature.ConclusionNeither PTJs nor LJs require or recommend the use of a systematic review to justify a new trial in their author guidelines. This potentially leaves room for unethical scientific practices and should be critically considered in future research.
| S-EPMC10787449 | biostudies-literature
Project description:BackgroundStudies evaluating the effectiveness and safety of telerehabilitation in patients undergoing total knee arthroplasty (TKA) have increased. However, the study quality and results differ, systematic reviews are limited. We aimed to synthesise systematic reviews and meta-analyses to assess the effects of telerehabilitation in patients post-TKA.Materials and methodsSystematic reviews and meta-analyses regarding the effectiveness and safety of TKA telerehabilitation were retrieved from eight databases from establishment to 18 December 2022. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2), Risk of Bias in Systematic Reviews (ROBIS) and GRADE system were used to evaluate results, methods, bias and evidence quality.ResultsThirteen systematic reviews and meta-analyses were analysed. The AMSTAR 2 showed low methodological quality in seven studies and very low quality in six. Among the key items, item 2 had been registered on website before systematic review in four reviews. Concerning item 4, did not provide a comprehensive search strategy in 4 reviews. For item 7, none of the reviews provided a list of reasons for excluding an article. For item 9, regarding whether appropriate tools were used to assess the risk of bias of each included study, one review was assessed as 'partially yes', one review only included RCTs, and the remainder were assessed as 'yes'. For item 11, one review did not specify the statistical methods used, and three reviews did not conduct a meta-analysis. For item 13, four reviews considered the risk of bias when interpreting or discussing the study results. For item 15, seven reviews did not evaluate publication bias. The PRISMA scores of the 13 reviews ranged from 17.5 to 26.0. The PRISMA indicated that 69.2% had no protocol registration, 38.5% did not provide other materials and evidence certainty, 23.1% did not provide certainty assessment, 30.8% did not report study bias. According to the ROBIS scale, diferrent domains have diferrent risks in all the reviews.ConclusionTelerehabilitation positively affects walking ability, knee extension and patient costs post-TKA surgery. Regarding the quality of life, patient satisfaction and the WOMAC, telerehabilitation had similar effects to conventional rehabilitation. Owing to the low quality of the studies, these conclusions should be interpreted cautiously, high-quality studies are needed in the future.
| S-EPMC10560426 | biostudies-literature