Assessment for Risk of Bias in Systematic Reviews and Meta-Analyses in the Field of Hepatology.
Ontology highlight
ABSTRACT:
SUBMITTER: Kim G
PROVIDER: S-EPMC4625697 | biostudies-other | 2015 Nov
REPOSITORIES: biostudies-other
Publications
Assessment for Risk of Bias in Systematic Reviews and Meta-Analyses in the Field of Hepatology.
Kim Gaeun G Cho Youn Zoo YZ Baik Soon Koo SK
Gut and liver 20151101 6
A systematic review (SR) provides the best and most objective analysis of the existing evidence in a particular field. SRs and derived conclusions are essential for evidence-based strategies in medicine and evidence-based guidelines in clinical practice. The popularity of SRs has also increased markedly in the field of hepatology. However, although SRs are considered to provide a higher level of evidence with greater confidence than original articles, there have been no reports on the quality of ...[more]
Similar Datasets
Project description:BACKGROUND: Use of a risk of bias (ROB) tool has been encouraged and advocated to reviewers writing systematic reviews (SRs) and meta-analyses (MAs). Selective outcome reporting and other sources of bias are included in the Cochrane ROB tool. It is important to know how this specific tool for assessing ROB has been applied since its release. Our objectives were to evaluate whether and to what extent the new Cochrane ROB tool has been used in Chinese journal papers of acupuncture. METHODS: We searched CBM, TCM database, CJFD, CSJD, and the Wanfang Database from inception to March 2011. Two reviewers independently selected SRs that primarily focused on acupuncture and moxibustion, from which the data was extracted and analyzed. RESULTS: A total of 836 SRs were identified from the search, of which, 105 were included and four are awaiting assessment. Thirty-six of the 105 SRs were published before release of the Cochrane ROB tool (up to 2009). Most used the Cochrane Handbook 4.2 or Jadad's scale for risk or quality assessment. From 2009 to March 2011 69 SRs were identified. While "risk of bias" was reported for approximately two-thirds of SRs, only two SRs mentioned use of a "risk of bias tool" in their assessment. Only 5.8% (4/69) of reviews reported information on all six domains which are involved in the ROB tool. A risk of bias graph/summary figure was provided in 2.9% (2/69) of reviews. Most SRs gave information about sequence generation, allocation concealment, blindness, and incomplete outcome data, however, few reviews (5.8%; 4/69) described selective reporting or other potential sources of bias. CONCLUSIONS: The Cochrane "risk of bias" tool has not been used in all SRs/MAs of acupuncture published in Chinese Journals after 2008. When the ROB tool was used, reporting of relevant information was often incomplete.
| S-EPMC3235108 | biostudies-literature
Project description:Rationale, aims, and objectivesCOVID-19 has caused an ongoing public health crisis. Many systematic reviews and meta-analyses have been performed to synthesize evidence for better understanding this new disease. However, some concerns have been raised about rapid COVID-19 research. This meta-epidemiological study aims to methodologically assess the current systematic reviews and meta-analyses on COVID-19.MethodsWe searched in various databases for systematic reviews with meta-analyses published between 1 January 2020 and 31 October 2020. We extracted their basic characteristics, data analyses, evidence appraisal, and assessment of publication bias and heterogeneity.ResultsWe identified 295 systematic reviews on COVID-19. The median time from submission to acceptance was 33 days. Among these systematic reviews, 73.9% evaluated clinical manifestations or comorbidities of COVID-19. Stata was the most used software programme (43.39%). The odds ratio was the most used effect measure (34.24%). Moreover, 28.14% of the systematic reviews did not present evidence appraisal. Among those reporting the risk of bias results, 14.64% of studies had a high risk of bias. Egger's test was the most used method for assessing publication bias (38.31%), while 38.66% of the systematic reviews did not assess publication bias. The I2 statistic was widely used for assessing heterogeneity (92.20%); many meta-analyses had high values of I2 . Among the meta-analyses using the random-effects model, 75.82% did not report the methods for model implementation; among those meta-analyses reporting implementation methods, the DerSimonian-Laird method was the most used one.ConclusionsThe current systematic reviews and meta-analyses on COVID-19 might suffer from low transparency, high heterogeneity, and suboptimal statistical methods. It is recommended that future systematic reviews on COVID-19 strictly follow well-developed guidelines. Sensitivity analyses may be performed to examine how the synthesized evidence might depend on different methods for appraising evidence, assessing publication bias, and implementing meta-analysis models.
| S-EPMC8242754 | biostudies-literature
Project description:Systematic reviews (SRs) that fail to search non-English databases may miss relevant studies and cause selection bias. The bias may be particularly severe in SRs of traditional Chinese medicine (TCM) as most randomized controlled trials (RCT) in TCM are published and accessible only in Chinese. In this study we investigated how often Chinese databases were not searched in SRs of TCM, how many trials were missed, and whether a bias may occur if Chinese databases were not searched. We searched 5 databases in English and 3 in Chinese for RCTs of Chinese herbal medicine for coronary artery disease and found that 96.64% (115/119) eligible studies could be identified only from Chinese databases. In a random sample of 80 Cochrane reviews on TCM, we found that Chinese databases were only searched in 43 or 53.75%, in which almost all the included studies were identified from Chinese databases. We also compared SRs of the same topic and found that they may draw a different conclusion if Chinese databases were not searched. In conclusion, an overwhelmingly high percentage of eligible trials on TCM could only be identified in Chinese databases. Reviewers in TCM are suggested to search Chinese databases to reduce potential selection bias.
| S-EPMC3816048 | biostudies-literature
Project description:Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007-2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to minimise their occurrence and mitigate their impacts on HSDR systematic reviews, are needed.
| S-EPMC6992172 | biostudies-literature
Project description:Comprehensive monitoring of the quality of systematic reviews (SRs) and meta-analyses (MAs) of endoscopic ultrasound (EUS) requires complete and accurate reporting and methodology.To assess the reporting and methodological quality of SRs/MAs on EUS diagnosis and to explore the potential factors influencing articles' quality.The quality of the reporting and methodology was evaluated in relation to the adherence of papers to the PRISMA checklist and the AMSTAR quality scale. The total scores for every criterion and for every article on the two standards were calculated. Data were evaluated and analyzed using SPSS17.0 and RevMan 5.1 in terms of publication time, category of reviews, category of journals, and funding resource.A total of 72 SRs/MAs was included, but no Cochrane Systematic Reviews (CSRs) were obtained. The number of SRs/MAs ranged from 1 in 1998 to 15 in 2013; 88.1% used the QUADAS tool; the average overall scores by PRISMA statement and AMSTAR tool were 19.9 and 5.4, respectively. Scores on some items showed substantial improvement after publication of PRISMA and AMSTAR. However, no reviews followed the criterion of protocol and registration, and only 11.1% of articles fulfilled the criterion of literature search. SRs/MAs from the Science Citation Index (SCI) were of better quality than non-SCI studies. Funding resource made no difference to quality. Regression analysis showed that time of publication and inclusion in the SCI were significantly correlated with total scores on the two standards.The reporting and methodological quality of SRs/MAs on EUS diagnosis has improved measurably since PRISMA and AMSTAR checklists released. It is hoped that CSR in this field will be produced. Literature searching and protocol criteria, as well as QUADAS-2 tool need to be addressed more in the future. Time of publication and SCI relate more to the overall quality of SRs/MAs than does funding resource.
| S-EPMC4408104 | biostudies-literature
Project description:BackgroundA systematic review is used to investigate the best available evidence of clinical safety and effectiveness of healthcare intervention. This requires methodological rigor in order to minimize bias and random error. The purpose of this study is to assess the quality of systematic reviews or meta-analyses for nursing interventions conducted by Korean researchers.MethodsWe searched electronic databases from 1950 to July 2010, including ovidMEDLINE, ovidEMBASE, and Korean databases, including KoreaMed, Korean Medical Database, and Korean studies Information Service System etc. Two reviewers independently screened and selected all references, and assessed the quality of systematic reviews or meta-analyses using the "Assessment of Multiple Systematic Reviews" (AMSTAR) tool.ResultsTwenty two systematic reviews or meta-analyses were included in this study. The median overall score (out of 11) for included reviews was 5 (range 2-11) and the mean overall score for AMSTAR was 4.7 (95% confidence interval 3.8-5.7). Nine out of 22 reviews were rated as low quality (AMSTAR score 0-4), 11 were rated as moderate quality (AMSTAR score 5-8), and two reviews were categorized as high quality (AMSTAR score 9-11).ConclusionsThe methodological quality of published reviews on nursing interventions conducted by Korean reviewers was assessed as low to moderate. In order to use the best available evidence in clinical decision making, reviewers should conduct systematic reviews or meta- analyses using rigorous research methods.
| S-EPMC3552770 | biostudies-literature
Project description:To identify and critically assess the extent to which systematic reviews of enhanced recovery programmes for patients undergoing colorectal surgery differ in their methodology and reported estimates of effect.Review of published systematic reviews. We searched the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA) Database from 1990 to March 2013. Systematic reviews of enhanced recovery programmes for patients undergoing colorectal surgery were eligible for inclusion.The primary outcome was length of hospital stay. We assessed changes in pooled estimates of treatment effect over time and how these might have been influenced by decisions taken by researchers as well as by the availability of new trials. The quality of systematic reviews was assessed using the Centre for Reviews and Dissemination (CRD) DARE critical appraisal process.10 systematic reviews were included. Systematic reviews of randomised controlled trials have consistently shown a reduction in length of hospital stay with enhanced recovery compared with traditional care. The estimated effect tended to increase from 2006 to 2010 as more trials were published but has not altered significantly in the most recent review, despite the inclusion of several unique trials. The best estimate appears to be an average reduction of around 2.5?days in primary postoperative length of stay. Differences between reviews reflected differences in interpretation of inclusion criteria, searching and analytical methods or software.Systematic reviews of enhanced recovery programmes show a high level of research waste, with multiple reviews covering identical or very similar groups of trials. Where multiple reviews exist on a topic, interpretation may require careful attention to apparently minor differences between reviews. Researchers can help readers by acknowledging existing reviews and through clear reporting of key decisions, especially on inclusion/exclusion and on statistical pooling.
| S-EPMC4039862 | biostudies-other
Project description:Various financial and non-financial conflicts of interests have been shown to influence the reporting of research findings, particularly in clinical medicine. In this study, we examine whether this extends to prognostic instruments designed to assess violence risk. Such instruments have increasingly become a routine part of clinical practice in mental health and criminal justice settings. The present meta-analysis investigated whether an authorship effect exists in the violence risk assessment literature by comparing predictive accuracy outcomes in studies where the individuals who designed these instruments were study authors with independent investigations. A systematic search from 1966 to 2011 was conducted using PsycINFO, EMBASE, MEDLINE, and US National Criminal Justice Reference Service Abstracts to identify predictive validity studies for the nine most commonly used risk assessment tools. Tabular data from 83 studies comprising 104 samples was collected, information on two-thirds of which was received directly from study authors for the review. Random effects subgroup analysis and metaregression were used to explore evidence of an authorship effect. We found a substantial and statistically significant authorship effect. Overall, studies authored by tool designers reported predictive validity findings around two times higher those of investigations reported by independent authors (DOR=6.22 [95% CI=4.68-8.26] in designers' studies vs. DOR=3.08 [95% CI=2.45-3.88] in independent studies). As there was evidence of an authorship effect, we also examined disclosure rates. None of the 25 studies where tool designers or translators were also study authors published a conflict of interest statement to that effect, despite a number of journals requiring that potential conflicts be disclosed. The field of risk assessment would benefit from routine disclosure and registration of research studies. The extent to which similar conflict of interests exists in those developing risk assessment guidelines and providing expert testimony needs clarification.
| S-EPMC3759386 | biostudies-literature
Project description:BackgroundThe QUOROM and PRISMA statements were published in 1999 and 2009, respectively, to improve the consistency of reporting systematic reviews (SRs)/meta-analyses (MAs) of clinical trials. However, not all SRs/MAs adhere completely to these important standards. In particular, it is not clear how well SRs/MAs of acupuncture studies adhere to reporting standards and which reporting criteria are generally ignored in these analyses.ObjectivesTo evaluate reporting quality in SRs/MAs of acupuncture studies.MethodsWe performed a literature search for studies published prior to 2014 using the following public archives: PubMed, EMBASE, Web of Science, the Cochrane Database of Systematic Reviews (CDSR), the Chinese Biomedical Literature Database (CBM), the Traditional Chinese Medicine (TCM) database, the Chinese Journal Full-text Database (CJFD), the Chinese Scientific Journal Full-text Database (CSJD), and the Wanfang database. Data were extracted into pre-prepared Excel data-extraction forms. Reporting quality was assessed based on the PRISMA checklist (27 items).ResultsOf 476 appropriate SRs/MAs identified in our search, 203, 227, and 46 were published in Chinese journals, international journals, and the Cochrane Database, respectively. In 476 SRs/MAs, only 3 reported the information completely. By contrast, approximately 4.93% (1/203), 8.81% (2/227) and 0.00% (0/46) SRs/Mas reported less than 10 items in Chinese journals, international journals and CDSR, respectively. In general, the least frequently reported items (reported≤50%) in SRs/MAs were "protocol and registration", "risk of bias across studies", and "additional analyses" in both methods and results sections.ConclusionsSRs/MAs of acupuncture studies have not comprehensively reported information recommended in the PRISMA statement. Our study underscores that, in addition to focusing on careful study design and performance, attention should be paid to comprehensive reporting standards in SRs/MAs on acupuncture studies.
| S-EPMC4232579 | biostudies-literature
Project description:BackgroundRisk-of-bias assessments are now a standard component of systematic reviews. At present, reviewers need to manually identify relevant parts of research articles for a set of methodological elements that affect the risk of bias, in order to make a risk-of-bias judgement for each of these elements. We investigate the use of text mining methods to automate risk-of-bias assessments in systematic reviews. We aim to identify relevant sentences within the text of included articles, to rank articles by risk of bias and to reduce the number of risk-of-bias assessments that the reviewers need to perform by hand.MethodsWe use supervised machine learning to train two types of models, for each of the three risk-of-bias properties of sequence generation, allocation concealment and blinding. The first model predicts whether a sentence in a research article contains relevant information. The second model predicts a risk-of-bias value for each research article. We use logistic regression, where each independent variable is the frequency of a word in a sentence or article, respectively.ResultsWe found that sentences can be successfully ranked by relevance with area under the receiver operating characteristic (ROC) curve (AUC)?> 0.98. Articles can be ranked by risk of bias with AUC > 0.72. We estimate that more than 33% of articles can be assessed by just one reviewer, where two reviewers are normally required.ConclusionsWe show that text mining can be used to assist risk-of-bias assessments.
| S-EPMC4795562 | biostudies-literature