Heterogeneity of systematic reviews in oncology.
Ontology highlight
ABSTRACT:
SUBMITTER: Holmes J
PROVIDER: S-EPMC5349813 | biostudies-literature | 2017 Apr
REPOSITORIES: biostudies-literature
Publications
Heterogeneity of systematic reviews in oncology.
Holmes Jonathan J Herrmann David D Koller Chelsea C Khan Sarah S Umberham Blake B Worley Jody A JA Vassar Matt M
Proceedings (Baylor University. Medical Center) 20170401 2
Systematic reviews synthesize data across multiple studies to answer a research question, and an important component of the review process is to evaluate the heterogeneity of primary studies considered for inclusion. Little is known, however, about the ways that systematic reviewers evaluate heterogeneity, especially in clinical specialties like oncology. We examined a sample of systematic reviews from this body of literature to determine how meta-analysts assessed and reported heterogeneity. A ...[more]
Similar Datasets
Project description:BackgroundCritics of systematic reviews have argued that these studies often fail to inform clinical decision making because their results are far too general, that the data are sparse, such that findings cannot be applied to individual patients or for other decision making. While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. Clinical heterogeneity, true effect heterogeneity, can be defined as variability among studies in the participants, the types or timing of outcome measurements, and the intervention characteristics. The objective of this project was to develop recommendations for investigating clinical heterogeneity in systematic reviews.MethodsWe used a modified Delphi technique with three phases: (1) pre-meeting item generation; (2) face-to-face consensus meeting in the form of a modified Delphi process; and (3) post-meeting feedback. We identified and invited potential participants with expertise in systematic review methodology, systematic review reporting, or statistical aspects of meta-analyses, or those who published papers on clinical heterogeneity.ResultsBetween April and June of 2011, we conducted phone calls with participants. In June 2011 we held the face-to-face focus group meeting in Ann Arbor, Michigan. First, we agreed upon a definition of clinical heterogeneity: Variations in the treatment effect that are due to differences in clinically related characteristics. Next, we discussed and generated recommendations in the following 12 categories related to investigating clinical heterogeneity: the systematic review team, planning investigations, rationale for choice of variables, types of clinical variables, the role of statistical heterogeneity, the use of plotting and visual aids, dealing with outlier studies, the number of investigations or variables, the role of the best evidence synthesis, types of statistical methods, the interpretation of findings, and reporting.ConclusionsClinical heterogeneity is common in systematic reviews. Our recommendations can help guide systematic reviewers in conducting valid and reliable investigations of clinical heterogeneity. Findings of these investigations may allow for increased applicability of findings of systematic reviews to the management of individual patients.
| S-EPMC3847163 | biostudies-literature
Project description:ImportanceSystematic reviews of medical imaging diagnostic test accuracy (DTA) studies are affected by between-study heterogeneity due to a range of factors. Failure to appropriately assess the extent and causes of heterogeneity compromises the interpretability of systematic review findings.ObjectiveTo assess how heterogeneity has been examined in medical imaging DTA studies.Evidence reviewThe PubMed database was searched for systematic reviews of medical imaging DTA studies that performed a meta-analysis. The search was limited to the 40 journals with highest impact factor in the radiology, nuclear medicine, and medical imaging category in the InCites Journal Citation Reports of 2021 to reach a sample size of 200 to 300 included studies. Descriptive analysis was performed to characterize the imaging modality, target condition, type of meta-analysis model used, strategies for evaluating heterogeneity, and sources of heterogeneity identified. Multivariable logistic regression was performed to assess whether any factors were associated with at least 1 source of heterogeneity being identified in the included meta-analyses. Methodological quality evaluation was not performed. Data analysis occurred from October to December 2022.FindingsA total of 242 meta-analyses involving a median (range) of 987 (119-441 510) patients across a diverse range of disease categories and imaging modalities were included. The extent of heterogeneity was adequately described (ie, whether it was absent, low, moderate, or high) in 220 studies (91%) and was most commonly assessed using the I2 statistic (185 studies [76%]) and forest plots (181 studies [75%]). Heterogeneity was rated as moderate to high in 191 studies (79%). Of all included meta-analyses, 122 (50%) performed subgroup analysis and 87 (36%) performed meta-regression. Of the 242 studies assessed, 189 (78%) included 10 or more primary studies. Of these 189 studies, 60 (32%) did not perform meta-regression or subgroup analysis. Reasons for being unable to investigate sources of heterogeneity included inadequate reporting of primary study characteristics and a low number of included primary studies. Use of meta-regression was associated with identification of at least 1 source of variability (odds ratio, 1.90; 95% CI, 1.11-3.23; P = .02).Conclusions and relevanceIn this systematic review of assessment of heterogeneity in medical imaging DTA meta-analyses, most meta-analyses were impacted by a moderate to high level of heterogeneity, presenting interpretive challenges. These findings suggest that, despite the development and availability of more rigorous statistical models, heterogeneity appeared to be incomplete, inconsistently evaluated, or methodologically questionable in many cases, which lessened the interpretability of the analyses performed; comprehensive heterogeneity assessment should be addressed at the author level by improving personal familiarity with appropriate statistical methodology for assessing heterogeneity and involving biostatisticians and epidemiologists in study design, as well as at the editorial level, by mandating adherence to methodologic standards in primary DTA studies and DTA meta-analyses.
| S-EPMC10905313 | biostudies-literature
Project description:BACKGROUND AND OBJECTIVES:Although most people have some experience as caregivers, the nature and context of care are highly variable. Caregiving, socioeconomic factors, and health are all interrelated. For these reasons, caregiver interventions must consider these factors. This review examines the degree to which caregiver intervention research has reported and considered social determinants of health. RESEARCH DESIGN AND METHODS:We examined published systematic reviews and meta-analyses of interventions for older adults with age-related chronic conditions using the PRISMA and AMSTAR 2 checklists. From 2,707 papers meeting search criteria, we identified 197 potentially relevant systematic reviews, and selected 33 for the final analysis. RESULTS:We found scant information on the inclusion of social determinants; the papers lacked specificity regarding race/ethnicity, gender, sexual identity, socioeconomic status, and geographic location. The majority of studies focused on dementia, with other conditions common in later life vastly underrepresented. DISCUSSION AND IMPLICATIONS:Significant gaps in evidence persist, particularly for interventions targeting diverse conditions and populations. To advance health equity and improve the effectiveness of interventions, research should address caregiver heterogeneity and improve assessment, support, and instruction for diverse populations. Research must identify aspects of heterogeneity that matter in intervention design, while recognizing opportunities for common elements and strategies.
| S-EPMC7019663 | biostudies-literature
Project description:BackgroundCochrane plain language summaries (PLSs) are an important format to present high-quality healthcare evidence to patients with cancer and their families. They should be written in a way everyone can understand, since they serve as a tool in decision-making and present a bridge to overcome the gap between the healthcare users and professionals.ObjectiveThe aim of the study was to assess the language characteristics of PLSs of Cochrane systematic reviews of oncology interventions in comparison with corresponding Cochrane scientific abstracts (SAs).MethodsIn this cross-sectional study, we included all Cochrane PLSs and SAs of systematic reviews of oncology interventions available in the Cochrane Database of Systematic Reviews. We assessed text readability, measured using the Simple Measure of Gobbledygook (SMOG) index, and the prevalence of words related to different language tones (clout, authenticity, emotions and analytical tones). Two independent assessors categorized the conclusiveness of the efficacy of interventions into nine categories.ResultsThe overall median SMOG index for 275 PLSs was 13.0 (95% confidence interval [CI] 12.8-13.3). Readability scores did not differ across Cochrane Review Groups. SAs had a higher readability index than the corresponding PLSs (median = 16.6, 95% CI = 16.4-16.8). Regarding linguistic characteristics, PLSs were shorter than SAs, with less use of analytical tone, but more use of a positive emotional tone and authenticity. Overall, the 'Unclear' category of conclusiveness was the most common among all PLSs. Also, PLSs with 'No evidence' conclusions were the shortest and had the lowest SMOG index.ConclusionPLSs of Cochrane systematic reviews of oncological interventions have low readability and most give unclear conclusions about the efficacy of interventions. PLSs should be simplified so that patients and their families can benefit from appropriate health information on evidence synthesis. Further research is needed into reasons for unclear language to describe evidence from oncology trials.
| S-EPMC10225178 | biostudies-literature
Project description:Systematic reviews (SRs) are considered an important tool for decision-making. There has been no recent comprehensive identification or description of child-relevant SRs. A description of existing child-relevant SRs would help to identify the extent of available child-relevant evidence available in SRs and gaps in the evidence base where SRs are required. The objective of this study was to describe child-relevant SRs from the Cochrane Database of Systematic Reviews (CDSR, Issue 2, 2009).SRs were assessed for relevance using pre-defined criteria. Data were extracted and entered into an electronic form. Univariate analyses were performed to describe the SRs overall and by topic area.The search yielded 1666 SRs; 793 met the inclusion criteria. 38% of SRs were last assessed as up-to-date prior to 2007. Corresponding authors were most often from the UK (41%). Most SRs (59%) examined pharmacological interventions. 53% had at least one external source of funding. SRs included a median of 7 studies (IQR 3, 15) and 679 participants (IQR 179, 2833). Of all studies, 48% included only children, and 27% only adults. 94% of studies were published in peer-reviewed journals. Primary outcomes were specified in 72% of SRs. Allocation concealment and the Jadad scale were used in 97% and 25% of SRs, respectively. Adults and children were analyzed separately in 12% of SRs and as a subgroup analysis in 14%. Publication bias was assessed in only 14% of SRs. A meta-analysis was conducted in 68% of SRs with a median of 5 trials (IQR 3, 9) each. Variations in these characteristics were observed across topic areas.We described the methodological characteristics and rigour of child-relevant reviews in the CDSR. Many SRs are not up-to-date according to Cochrane criteria. Our study describes variation in conduct and reporting across SRs and reveals clinicians' ability to access child-specific data.
| S-EPMC2881081 | biostudies-other
Project description:Importance:Reproducible research practices are essential to biomedical research because these practices promote trustworthy evidence. In systematic reviews and meta-analyses, reproducible research practices ensure that summary effects used to guide patient care are stable and trustworthy. Objective:To evaluate the reproducibility in theory of meta-analyses in oncology systematic reviews cited by the 49 National Comprehensive Cancer Network (NCCN) guidelines for the treatment of cancer by site and evaluate whether Cochrane reviews or systematic reviews that report adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines use more reproducible research practices. Design, Setting, and Participants:A cross-sectional investigation of all systematic reviews with at least 1 meta-analysis and at least 1 included randomized clinical trial (RCT) that are cited by NCCN guidelines for treatment of cancer by site. We scanned the reference list of all NCCN guidelines (n?=?49) for potential systematic reviews and meta-analyses. All retrieved studies were screened, and data were extracted, independently and in duplicate. The analysis was carried out between May 6, 2018, and January 28, 2019. Main Outcomes and Measures:The frequency of reproducible research practices, defined as (1) effect estimate and measure of precision (eg, hazard ratio with 95% confidence interval); (2) clear list of studies included for each analysis; and (3) for subgroup and sensitivity analyses, it must be clear which studies were included in each group or level. Results:We identified 1124 potential systematic reviews, and 154 meta-analyses comprising 3696 meta-analytic effect size estimates were included. Only 2375 of the 3696 meta-analytic estimates (64.3%), including subgroup and sensitivity analyses, were reproducible in theory. Forest plots appear to improve the reproducibility of meta-analyses. All meta-analytic estimates were reproducible in theory in 100 systematic reviews (64.9%), and in 15 systematic reviews (9.7%), no meta-analytic estimates could potentially be reproduced. Data were said to be imputed in 29 meta-analyses, but none specified which data. Only 1 meta-analysis included a link to an online data set. Conclusions and Relevance:More reproducible research practices are needed in oncology meta-analyses, as suggested by those that are cited by the NCCN. Reporting meta-analyses in forest plots and requirements for full data sharing are recommended.
| S-EPMC6735674 | biostudies-literature
Project description:BackgroundPolicy makers and others often require synthesis of knowledge in an area within six months or less. Traditional systematic reviews typically take at least 12 months to conduct. Rapid reviews streamline traditional systematic review methods in order to synthesize evidence within a shortened timeframe. There is great variation in the process of conducting rapid reviews. This review sought to examine methods used for rapid reviews, as well as implications of methodological streamlining in terms of rigour, bias, and results.MethodsA comprehensive search strategy--including five electronic databases, grey literature, hand searching of relevant journals, and contacting key informants--was undertaken. All titles and abstracts (n = 1,989) were reviewed independently by two reviewers. Relevance criteria included articles published between 1995 and 2009 about conducting rapid reviews or addressing comparisons of rapid reviews versus traditional reviews. Full articles were retrieved for any titles deemed relevant by either reviewer (n = 70). Data were extracted from all relevant methodological articles (n = 45) and from exemplars of rapid review methods (n = 25).ResultsRapid reviews varied from three weeks to six months; various methods for speeding up the process were employed. Some limited searching by years, databases, language, and sources beyond electronic searches. Several employed one reviewer for title and abstract reviewing, full text review, methodological quality assessment, and/or data extraction phases. Within rapid review studies, accelerating the data extraction process may lead to missing some relevant information. Biases may be introduced due to shortened timeframes for literature searching, article retrieval, and appraisal.ConclusionsThis review examined the continuum between diverse rapid review methods and traditional systematic reviews. It also examines potential implications of streamlined review methods. More of these rapid reviews need to be published in the peer-reviewed literature with an emphasis on articulating methods employed. While one consistent methodological approach may not be optimal or appropriate, it is important that researchers undertaking reviews within the rapid to systematic continuum provide detailed descriptions of methods used and discuss the implications of their chosen methods in terms of potential bias introduced. Further research comparing full systematic reviews with rapid reviews will enhance understanding of the limitations of these methods.
| S-EPMC2914085 | biostudies-literature
Project description:BackgroundMany meta-analyses contain only a small number of studies, which makes it difficult to estimate the extent of between-study heterogeneity. Bayesian meta-analysis allows incorporation of external evidence on heterogeneity, and offers advantages over conventional random-effects meta-analysis. To assist in this, we provide empirical evidence on the likely extent of heterogeneity in particular areas of health care.MethodsOur analyses included 14?886 meta-analyses from the Cochrane Database of Systematic Reviews. We classified each meta-analysis according to the type of outcome, type of intervention comparison and medical specialty. By modelling the study data from all meta-analyses simultaneously, using the log odds ratio scale, we investigated the impact of meta-analysis characteristics on the underlying between-study heterogeneity variance. Predictive distributions were obtained for the heterogeneity expected in future meta-analyses.ResultsBetween-study heterogeneity variances for meta-analyses in which the outcome was all-cause mortality were found to be on average 17% (95% CI 10-26) of variances for other outcomes. In meta-analyses comparing two active pharmacological interventions, heterogeneity was on average 75% (95% CI 58-95) of variances for non-pharmacological interventions. Meta-analysis size was found to have only a small effect on heterogeneity. Predictive distributions are presented for nine different settings, defined by type of outcome and type of intervention comparison. For example, for a planned meta-analysis comparing a pharmacological intervention against placebo or control with a subjectively measured outcome, the predictive distribution for heterogeneity is a log-normal (-2.13, 1.58(2)) distribution, which has a median value of 0.12. In an example of meta-analysis of six studies, incorporating external evidence led to a smaller heterogeneity estimate and a narrower confidence interval for the combined intervention effect.ConclusionsMeta-analysis characteristics were strongly associated with the degree of between-study heterogeneity, and predictive distributions for heterogeneity differed substantially across settings. The informative priors provided will be very beneficial in future meta-analyses including few studies.
| S-EPMC3396310 | biostudies-literature
Project description:In searches for systematic reviews, it is recommended that authors review references from the reference lists of retrieved relevant reviews for possible additional, relevant references. This process can be time consuming, since there often is overlap between the reference lists and the lists contain references that were already retrieved in the initial searches. The author proposes a method in which EndNote is used in combination with the Scopus or Web of Science databases to semi-automatically download these references into an existing EndNote library.
| S-EPMC6148618 | biostudies-other
Project description:ObjectivesThe objectives of this study were to benchmark roles that veterinary librarians at universities and colleges play in systematic reviews (SRs) and scoping reviews that are conducted by faculty and students at their institutions, to benchmark the level of training that veterinary librarians have in conducting SRs, to identify barriers to their participation in SRs, and to identify other types of literature reviews that veterinary librarians participate in.MethodsSixty veterinary librarians in universities and colleges in Canada, the United States, England, Scotland, Ireland, Australia, and New Zealand were surveyed online about their roles and training in conducting SRs, barriers to participation in SRs, and participation in other types of literature reviews.ResultsVeterinary librarians' highest participation was at an advising level in traditional librarian roles as question formulator, database selector, search strategy developer, and reference manager. Most respondents reported pretty good to extensive training in traditional roles and no or some training in less traditional roles. Sixty percent of respondents received few or no requests to participate in SRs, and only half of respondents had participated in SRs as a review team member. Sixty percent of respondents stated that their libraries had no policies regarding librarian roles and participation in SRs.ConclusionsThe surveyed veterinary librarians participated in SRs to a lesser degree than human health sciences librarians, experienced low demand from veterinary faculty and students to participate in SRs, and participated as review team members at significantly lower rates than human health sciences librarians. The main barriers to participation in SRs were lack of library policies, insufficient training, and lack of time.
| S-EPMC6774563 | biostudies-literature