Heterogeneity of systematic reviews in oncology.
Ontology highlight
ABSTRACT:
SUBMITTER: Holmes J
PROVIDER: S-EPMC5349813 | biostudies-literature | 2017 Apr
REPOSITORIES: biostudies-literature
Publications
Heterogeneity of systematic reviews in oncology.
Holmes Jonathan J Herrmann David D Koller Chelsea C Khan Sarah S Umberham Blake B Worley Jody A JA Vassar Matt M
Proceedings (Baylor University. Medical Center) 20170401 2
Systematic reviews synthesize data across multiple studies to answer a research question, and an important component of the review process is to evaluate the heterogeneity of primary studies considered for inclusion. Little is known, however, about the ways that systematic reviewers evaluate heterogeneity, especially in clinical specialties like oncology. We examined a sample of systematic reviews from this body of literature to determine how meta-analysts assessed and reported heterogeneity. A ...[more]
Similar Datasets
Project description:BackgroundCritics of systematic reviews have argued that these studies often fail to inform clinical decision making because their results are far too general, that the data are sparse, such that findings cannot be applied to individual patients or for other decision making. While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. Clinical heterogeneity, true effect heterogeneity, can be defined as variability among studies in the participants, the types or timing of outcome measurements, and the intervention characteristics. The objective of this project was to develop recommendations for investigating clinical heterogeneity in systematic reviews.MethodsWe used a modified Delphi technique with three phases: (1) pre-meeting item generation; (2) face-to-face consensus meeting in the form of a modified Delphi process; and (3) post-meeting feedback. We identified and invited potential participants with expertise in systematic review methodology, systematic review reporting, or statistical aspects of meta-analyses, or those who published papers on clinical heterogeneity.ResultsBetween April and June of 2011, we conducted phone calls with participants. In June 2011 we held the face-to-face focus group meeting in Ann Arbor, Michigan. First, we agreed upon a definition of clinical heterogeneity: Variations in the treatment effect that are due to differences in clinically related characteristics. Next, we discussed and generated recommendations in the following 12 categories related to investigating clinical heterogeneity: the systematic review team, planning investigations, rationale for choice of variables, types of clinical variables, the role of statistical heterogeneity, the use of plotting and visual aids, dealing with outlier studies, the number of investigations or variables, the role of the best evidence synthesis, types of statistical methods, the interpretation of findings, and reporting.ConclusionsClinical heterogeneity is common in systematic reviews. Our recommendations can help guide systematic reviewers in conducting valid and reliable investigations of clinical heterogeneity. Findings of these investigations may allow for increased applicability of findings of systematic reviews to the management of individual patients.
| S-EPMC3847163 | biostudies-literature
Project description:BACKGROUND AND OBJECTIVES:Although most people have some experience as caregivers, the nature and context of care are highly variable. Caregiving, socioeconomic factors, and health are all interrelated. For these reasons, caregiver interventions must consider these factors. This review examines the degree to which caregiver intervention research has reported and considered social determinants of health. RESEARCH DESIGN AND METHODS:We examined published systematic reviews and meta-analyses of interventions for older adults with age-related chronic conditions using the PRISMA and AMSTAR 2 checklists. From 2,707 papers meeting search criteria, we identified 197 potentially relevant systematic reviews, and selected 33 for the final analysis. RESULTS:We found scant information on the inclusion of social determinants; the papers lacked specificity regarding race/ethnicity, gender, sexual identity, socioeconomic status, and geographic location. The majority of studies focused on dementia, with other conditions common in later life vastly underrepresented. DISCUSSION AND IMPLICATIONS:Significant gaps in evidence persist, particularly for interventions targeting diverse conditions and populations. To advance health equity and improve the effectiveness of interventions, research should address caregiver heterogeneity and improve assessment, support, and instruction for diverse populations. Research must identify aspects of heterogeneity that matter in intervention design, while recognizing opportunities for common elements and strategies.
| S-EPMC7019663 | biostudies-literature
Project description:Importance:Reproducible research practices are essential to biomedical research because these practices promote trustworthy evidence. In systematic reviews and meta-analyses, reproducible research practices ensure that summary effects used to guide patient care are stable and trustworthy. Objective:To evaluate the reproducibility in theory of meta-analyses in oncology systematic reviews cited by the 49 National Comprehensive Cancer Network (NCCN) guidelines for the treatment of cancer by site and evaluate whether Cochrane reviews or systematic reviews that report adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines use more reproducible research practices. Design, Setting, and Participants:A cross-sectional investigation of all systematic reviews with at least 1 meta-analysis and at least 1 included randomized clinical trial (RCT) that are cited by NCCN guidelines for treatment of cancer by site. We scanned the reference list of all NCCN guidelines (n?=?49) for potential systematic reviews and meta-analyses. All retrieved studies were screened, and data were extracted, independently and in duplicate. The analysis was carried out between May 6, 2018, and January 28, 2019. Main Outcomes and Measures:The frequency of reproducible research practices, defined as (1) effect estimate and measure of precision (eg, hazard ratio with 95% confidence interval); (2) clear list of studies included for each analysis; and (3) for subgroup and sensitivity analyses, it must be clear which studies were included in each group or level. Results:We identified 1124 potential systematic reviews, and 154 meta-analyses comprising 3696 meta-analytic effect size estimates were included. Only 2375 of the 3696 meta-analytic estimates (64.3%), including subgroup and sensitivity analyses, were reproducible in theory. Forest plots appear to improve the reproducibility of meta-analyses. All meta-analytic estimates were reproducible in theory in 100 systematic reviews (64.9%), and in 15 systematic reviews (9.7%), no meta-analytic estimates could potentially be reproduced. Data were said to be imputed in 29 meta-analyses, but none specified which data. Only 1 meta-analysis included a link to an online data set. Conclusions and Relevance:More reproducible research practices are needed in oncology meta-analyses, as suggested by those that are cited by the NCCN. Reporting meta-analyses in forest plots and requirements for full data sharing are recommended.
| S-EPMC6735674 | biostudies-literature
Project description:BACKGROUND: Policy makers and others often require synthesis of knowledge in an area within six months or less. Traditional systematic reviews typically take at least 12 months to conduct. Rapid reviews streamline traditional systematic review methods in order to synthesize evidence within a shortened timeframe. There is great variation in the process of conducting rapid reviews. This review sought to examine methods used for rapid reviews, as well as implications of methodological streamlining in terms of rigour, bias, and results. METHODS: A comprehensive search strategy--including five electronic databases, grey literature, hand searching of relevant journals, and contacting key informants--was undertaken. All titles and abstracts (n = 1,989) were reviewed independently by two reviewers. Relevance criteria included articles published between 1995 and 2009 about conducting rapid reviews or addressing comparisons of rapid reviews versus traditional reviews. Full articles were retrieved for any titles deemed relevant by either reviewer (n = 70). Data were extracted from all relevant methodological articles (n = 45) and from exemplars of rapid review methods (n = 25). RESULTS: Rapid reviews varied from three weeks to six months; various methods for speeding up the process were employed. Some limited searching by years, databases, language, and sources beyond electronic searches. Several employed one reviewer for title and abstract reviewing, full text review, methodological quality assessment, and/or data extraction phases. Within rapid review studies, accelerating the data extraction process may lead to missing some relevant information. Biases may be introduced due to shortened timeframes for literature searching, article retrieval, and appraisal. CONCLUSIONS: This review examined the continuum between diverse rapid review methods and traditional systematic reviews. It also examines potential implications of streamlined review methods. More of these rapid reviews need to be published in the peer-reviewed literature with an emphasis on articulating methods employed. While one consistent methodological approach may not be optimal or appropriate, it is important that researchers undertaking reviews within the rapid to systematic continuum provide detailed descriptions of methods used and discuss the implications of their chosen methods in terms of potential bias introduced. Further research comparing full systematic reviews with rapid reviews will enhance understanding of the limitations of these methods.
| S-EPMC2914085 | biostudies-literature
Project description:Systematic reviews (SRs) are considered an important tool for decision-making. There has been no recent comprehensive identification or description of child-relevant SRs. A description of existing child-relevant SRs would help to identify the extent of available child-relevant evidence available in SRs and gaps in the evidence base where SRs are required. The objective of this study was to describe child-relevant SRs from the Cochrane Database of Systematic Reviews (CDSR, Issue 2, 2009).SRs were assessed for relevance using pre-defined criteria. Data were extracted and entered into an electronic form. Univariate analyses were performed to describe the SRs overall and by topic area.The search yielded 1666 SRs; 793 met the inclusion criteria. 38% of SRs were last assessed as up-to-date prior to 2007. Corresponding authors were most often from the UK (41%). Most SRs (59%) examined pharmacological interventions. 53% had at least one external source of funding. SRs included a median of 7 studies (IQR 3, 15) and 679 participants (IQR 179, 2833). Of all studies, 48% included only children, and 27% only adults. 94% of studies were published in peer-reviewed journals. Primary outcomes were specified in 72% of SRs. Allocation concealment and the Jadad scale were used in 97% and 25% of SRs, respectively. Adults and children were analyzed separately in 12% of SRs and as a subgroup analysis in 14%. Publication bias was assessed in only 14% of SRs. A meta-analysis was conducted in 68% of SRs with a median of 5 trials (IQR 3, 9) each. Variations in these characteristics were observed across topic areas.We described the methodological characteristics and rigour of child-relevant reviews in the CDSR. Many SRs are not up-to-date according to Cochrane criteria. Our study describes variation in conduct and reporting across SRs and reveals clinicians' ability to access child-specific data.
| S-EPMC2881081 | biostudies-other
Project description:BackgroundMany meta-analyses contain only a small number of studies, which makes it difficult to estimate the extent of between-study heterogeneity. Bayesian meta-analysis allows incorporation of external evidence on heterogeneity, and offers advantages over conventional random-effects meta-analysis. To assist in this, we provide empirical evidence on the likely extent of heterogeneity in particular areas of health care.MethodsOur analyses included 14?886 meta-analyses from the Cochrane Database of Systematic Reviews. We classified each meta-analysis according to the type of outcome, type of intervention comparison and medical specialty. By modelling the study data from all meta-analyses simultaneously, using the log odds ratio scale, we investigated the impact of meta-analysis characteristics on the underlying between-study heterogeneity variance. Predictive distributions were obtained for the heterogeneity expected in future meta-analyses.ResultsBetween-study heterogeneity variances for meta-analyses in which the outcome was all-cause mortality were found to be on average 17% (95% CI 10-26) of variances for other outcomes. In meta-analyses comparing two active pharmacological interventions, heterogeneity was on average 75% (95% CI 58-95) of variances for non-pharmacological interventions. Meta-analysis size was found to have only a small effect on heterogeneity. Predictive distributions are presented for nine different settings, defined by type of outcome and type of intervention comparison. For example, for a planned meta-analysis comparing a pharmacological intervention against placebo or control with a subjectively measured outcome, the predictive distribution for heterogeneity is a log-normal (-2.13, 1.58(2)) distribution, which has a median value of 0.12. In an example of meta-analysis of six studies, incorporating external evidence led to a smaller heterogeneity estimate and a narrower confidence interval for the combined intervention effect.ConclusionsMeta-analysis characteristics were strongly associated with the degree of between-study heterogeneity, and predictive distributions for heterogeneity differed substantially across settings. The informative priors provided will be very beneficial in future meta-analyses including few studies.
| S-EPMC3396310 | biostudies-literature
Project description:ObjectivesThe objectives of this study were to benchmark roles that veterinary librarians at universities and colleges play in systematic reviews (SRs) and scoping reviews that are conducted by faculty and students at their institutions, to benchmark the level of training that veterinary librarians have in conducting SRs, to identify barriers to their participation in SRs, and to identify other types of literature reviews that veterinary librarians participate in.MethodsSixty veterinary librarians in universities and colleges in Canada, the United States, England, Scotland, Ireland, Australia, and New Zealand were surveyed online about their roles and training in conducting SRs, barriers to participation in SRs, and participation in other types of literature reviews.ResultsVeterinary librarians' highest participation was at an advising level in traditional librarian roles as question formulator, database selector, search strategy developer, and reference manager. Most respondents reported pretty good to extensive training in traditional roles and no or some training in less traditional roles. Sixty percent of respondents received few or no requests to participate in SRs, and only half of respondents had participated in SRs as a review team member. Sixty percent of respondents stated that their libraries had no policies regarding librarian roles and participation in SRs.ConclusionsThe surveyed veterinary librarians participated in SRs to a lesser degree than human health sciences librarians, experienced low demand from veterinary faculty and students to participate in SRs, and participated as review team members at significantly lower rates than human health sciences librarians. The main barriers to participation in SRs were lack of library policies, insufficient training, and lack of time.
| S-EPMC6774563 | biostudies-literature
Project description:BACKGROUND:Research turnover in the HIV field is rapid, and as a result, maintaining high-quality, up-to-date, and relevant systematic reviews is a challenge. One approach is to frequently update published reviews. METHODS:We evaluated the methods and relevance of all HIV systematic reviews and protocols published in the Cochrane Library over a 16-year period (2000-2016) to determine the need to update published reviews or complete of reviews in progress. RESULTS:Of 148 published reviews and protocols, 129 (87%) were identified as not for updating or progression to publication, mostly due to research questions which were either entirely outdated or addressed questions in an outdated manner (N?=?89; 60%); this was anticipated for older reviews, but was found also to be the case for recent publications. Some research questions were also inadequately conceptualized, particularly when complex pragmatic trials or behavioral interventions were included. CONCLUSIONS:We suggest that authors clearly characterize interventions and synthesis approaches in their review protocols. In research fields, such as HIV, where questions change frequently, systematic reviews and protocols should be regularly re-evaluated to ensure relevance to current questions. This process of re-evaluation should be incorporated into the methods of living systematic reviews.
| S-EPMC6366015 | biostudies-literature
Project description:Evidence-based decision making is a hallmark of effective veterinary clinical practice. Scoping reviews, systematic reviews, and meta-analyses all are methods intended to provide transparent and replicable ways of summarizing a body of research to address an important clinical or public health issue. As these methods increasingly are being used by researchers and read by practitioners, it is important to understand the distinction between these techniques and to understand what research questions they can, and cannot, address. This review provides an overview of scoping reviews, systematic reviews, and meta-analysis, including a discussion of the method and uses. A sample dataset and coding to conduct a simple meta-analysis in the statistical program R also are provided. Scoping reviews are a descriptive approach, designed to chart the literature around a particular topic. The approach involves an extensive literature search, following by a structured mapping, or charting, of the literature. The results of scoping reviews can help to inform future research by identifying gaps in the existing literature and also can be used to identify areas where there may be a sufficient depth of literature to warrant a systematic review. Systematic reviews are intended to address a specific question by identifying and summarizing all of the available research that has addressed the review question. Questions types that can be addressed by a systematic review include prevalence/incidence questions, and questions related to etiology, intervention efficacy, and diagnostic test accuracy. The systematic review process follows structured steps with multiple reviewers working in parallel to reduce the potential for bias. An extensive literature search is undertaken and, for each relevant study identified by the search, a formal extraction of data, including the effect size, and assessment of the risk of bias is performed. The results from multiple studies can be combined using meta-analysis. Meta-analysis provides a summary effect size, and allows heterogeneity of effect among studies to be quantified and explored. These evidence synthesis approaches can provide scientific input to evidence-based clinical decision-making for veterinarians and regulatory bodies, and also can be useful for identifying gaps in the literature to enhance the efficiency of future research in a topic area.
| S-EPMC6997489 | biostudies-literature
Project description:BACKGROUND:Despite the growing reputation and subject coverage of the Cochrane Database of Systematic Reviews, many systematic reviews continue to be published solely in paper-based health care journals. This study was designed to determine why authors choose to publish their systematic reviews outside of the Cochrane Collaboration and if they might be interested in converting their reviews to Cochrane format for publication in the Cochrane Database of Systematic Reviews. METHODS:Cross-sectional survey of Australian primary authors of systematic reviews not published on the Cochrane Database of Systematic Reviews identified from the Database of Abstracts of Reviews of Effectiveness. RESULTS:We identified 88 systematic reviews from the Database of Abstracts of Reviews of Effectiveness with an Australian as the primary author. We surveyed 52 authors for whom valid contact information was available. The response rate was 88 per cent (46/52). Ten authors replied without completing the survey, leaving 36 valid surveys for analysis. The most frequently cited reasons for not undertaking a Cochrane review were: lack of time (78%), the need to undergo specific Cochrane training (46%), unwillingness to update reviews (36%), difficulties with the Cochrane process (26%) and the review topic already registered with the Cochrane Collaboration (21%). (Percentages based on completed responses to individual questions.) Nearly half the respondents would consider converting their review to Cochrane format. Dedicated time emerged as the most important factor in facilitating the potential conversion process. Other factors included navigating the Cochrane system, assistance with updating and financial support. Eighty-six per cent were willing to have their review converted to Cochrane format by another author. CONCLUSION:Time required to complete a Cochrane review and the need for specific training are the primary reasons why some authors publish systematic reviews outside of the Cochrane Collaboration. Encouragingly, almost half of the authors would consider converting their review to Cochrane format. Based on the current number of reviews in the Database of Abstracts of Reviews of Effectiveness, this could result in more than 700 additional Cochrane reviews. Ways of supporting these authors and how to provide dedicated time to convert systematic reviews needs further consideration.
| S-EPMC149345 | biostudies-literature