AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
Ontology highlight
ABSTRACT:
SUBMITTER: Shea BJ
PROVIDER: S-EPMC5833365 | biostudies-literature | 2017 Sep
REPOSITORIES: biostudies-literature
Publications
AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
Shea Beverley J BJ Reeves Barnaby C BC Wells George G Thuku Micere M Hamel Candyce C Moran Julian J Moher David D Tugwell Peter P Welch Vivian V Kristjansson Elizabeth E Henry David A DA
BMJ (Clinical research ed.) 20170921
Similar Datasets
Project description:BackgroundIn 2007, AMSTAR (A MeaSurement Tool to Assess systematic Reviews), a critical appraisal tool for systematic reviews (SRs), was published, and it has since become one of the most widely used instruments for SR appraisal. In September 2017, AMSTAR 2 was published as an updated version of the tool. This mixed-methods study aimed to analyze the extent of the AMSTAR 2 uptake and explore potential barriers to its uptake.MethodsWe analyzed the frequency of AMSTAR or AMSTAR 2 use in articles published in 2018, 2019 and 2020. We surveyed authors who have used AMSTAR but not AMSTAR 2 in the analyzed time frame to identify their reasons and barriers. The inclusion criterion for those authors was that the month of manuscript submission was after September 2017, i.e. after AMSTAR 2 was published.ResultsWe included 871 studies. The majority (N = 451; 52%) used AMSTAR 2, while 44% (N = 382) used AMSTAR, 4% (N = 31) used R-AMSTAR and others used a combination of tools. In 2018, 81% of the analyzed studies used AMSTAR, while 16% used AMSTAR 2. In 2019, 52% used AMSTAR, while 44% used AMSTAR 2. Among articles published in 2020, 28% used AMSTAR, while AMSTAR 2 was used by 69%. An author survey indicated that the authors did not use AMSTAR 2 mostly because they were not aware of it, their protocol was already established, or data collection completed at the time when the new tool was published. Barriers towards AMSTAR 2 use were lack of quantitative assessment, insufficient awareness, length, difficulties with a specific item.ConclusionIn articles published in 2018-2020, that were submitted to a journal after AMSTAR 2 tool was published, almost half of the authors (44%) still used AMSTAR, the old version of the tool. However, the use of AMSTAR has been declining in each subsequent year. Our survey indicated that editors and peer-reviewers did not ask the authors to use the new version of the tool. Few barriers towards using AMSTAR 2 were identified, and thus it is anticipated that the use of the old version of AMSTAR will continue to decline.
| S-EPMC8996416 | biostudies-literature
Project description:BackgroundOverviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews-for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews.MethodsWe selected seven overviews of healthcare interventions and included all SRs meeting each overview's inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions.ResultsNinety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; "almost perfect" vs. "substantial" using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions.ConclusionsResults indicate that the AMSTAR tool can be used successfully in overviews that include Cochrane and non-Cochrane SRs, though decision rules may be useful to circumvent common challenges. Findings support existing recommendations that quality assessments of SRs in overviews be conducted independently, in duplicate, with a process for consensus. Results also suggest that using methodological quality to guide inclusion decisions (e.g., to exclude poorly conducted and reported SRs) may not introduce bias into the overview process.
| S-EPMC5364717 | biostudies-literature
Project description:BackgroundBreast reconstruction is an important aspect in breast cancer treatment.MethodsA comprehensive search of MEDLINE, Embase, and the Cochrane Library of Systematic Reviews was performed. Systematic reviews and meta-analyses that focused on breast reconstruction and were published between 2000 and 2020 were included. Quality assessment was performed using A Measurement Tool to Assess Systematic Reviews (AMSTAR). Study characteristics were extracted, including journal and impact factor, year of publication, country affiliation, reporting adherence to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines, number of citations, and number of studies included.ResultsThe average AMSTAR score was moderate (5.32). There was a significant increase in AMSTAR score (P < 0.01) and number of studies (P < 0.01) over time. There were no significant correlations between AMSTAR score and impact factor (P = 0.038), and AMSTAR score and number of citations (P = 0.52), but there was a significant association between AMSTAR score and number of studies (P = 0.013). Studies that adhered to the PRISMA statement had a higher AMSTAR score on average (P < 0.01).ConclusionsSystematic reviews and meta-analyses about breast reconstruction had, on average, a moderate AMSTAR score. The number of studies and methodological quality have increased over time. Study characteristics including adherence to PRISMA guidelines are associated with improved methodological quality. Further improvements in specific AMSTAR domains would improve the overall methodological quality.
| S-EPMC8604032 | biostudies-literature
Project description:BackgroundThousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.MethodologyAMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.ConclusionsThe sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.
| S-EPMC2131785 | biostudies-literature
Project description:High-quality systematic reviews (SRs) can strengthen the evidence base for prevention and health promotion. A 16-item AMSTAR 2 tool allows the appraisal of SRs by deriving a confidence rating in their results. In this cross-sectional study, we aimed to assess and compare two approaches to appraising 30 SRs of digital interventions for physical activity (PA) promotion using AMSTAR 2. Approach 1 (appraisals with 2/16 items) was used to identify SRs with critically low confidence ratings. Approach 2 (appraisals with all 16 items) was used (1) to derive the confidence ratings, (2) to identify SR strengths and weaknesses, and (3) to compare SR strengths among subgroups of SRs. The appraisal outcomes were summarized and compared using descriptive statistics. Approach 1 was quick (mean of 5 min/SR) at identifying SRs with critically low confidence ratings. Approach 2 was slower (mean of 20 min/SR), but allowed to identify SR strengths and weaknesses. Approach 2 showed that confidence ratings were low to critically low in 29/30 SRs. More strengths were identified in SRs with review protocols relative to SRs without review protocols and in newer SRs (published after AMSTAR 2 release) relative to older SRs. Only two items on AMSTAR 2 can quickly identify SRs with critical weaknesses. Although most SRs received low to critically low confidence ratings, SRs with review protocols and newer SRs tended to have more strengths. Future SRs require review protocols and better adherence to reporting guidelines to improve the confidence in their results.
| S-EPMC10048476 | biostudies-literature
Project description:BackgroundFor over three decades researchers have developed critical appraisal tools (CATs) for assessing the scientific quality of research overviews. Most established CATs for reviews in evidence-based medicine and evidence-based public health (EBPH) focus on systematic reviews (SRs) with studies on experimental interventions or exposure included. EBPH- and implementation-oriented organisations and decision-makers, however, often seek access to rapid reviews (RRs) or scoping reviews (ScRs) for rapid evidence synthesis and research field exploration. Until now, no CAT is available to assess the quality of SRs, RRs, and ScRs following a unified approach. We set out to develop such a CAT.MethodsThe development process of the Critical Appraisal Tool for Health Promotion and Prevention Reviews (CAT HPPR) included six phases: (i) the definition of important review formats and complementary approaches, (ii) the identification of relevant CATs, (iii) prioritisation, selection and adaptation of quality criteria using a consensus approach, (iv) development of the rating system and bilingual guidance documents, (v) engaging with experts in the field for piloting/optimising the CAT, and (vi) approval of the final CAT. We used a pragmatic search approach to identify reporting guidelines/standards (n = 3; e.g. PRISMA, MECIR) as well as guidance documents (n = 17; e.g. for reviews with mixed-methods approach) to develop working definitions for SRs, RRs, ScRs, and other review types (esp. those defined by statistical methods or included data sources).ResultsWe successfully identified 14 relevant CATs, predominantly for SRs (e.g. AMSTAR 2), and extracted 46 items. Following consensual discussions 15 individual criteria were included in our CAT and tailored to the review types of interest. The CAT was piloted with 14 different reviews which were eligible to be included in a new German database looking at interventions in health promotion and prevention in different implementation settings.ConclusionsThe newly developed CAT HPPR follows a unique uniformed approach to assess a set of heterogeneous reviews (e.g. reviews from problem identification to policy evaluations) to assist end-users needs. Feedback of external experts showed general feasibility and satisfaction with the tool. Future studies should further formally test the validity of CAT HPPR using larger sets of reviews.
| S-EPMC9791771 | biostudies-literature
Project description:BackgroundOur objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.MethodsA 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.ResultsThe factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.ConclusionA measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.
| S-EPMC1810543 | biostudies-literature
Project description:BackgroundThe Measurement Tool to Assess systematic Reviews (AMSTAR) 2 is a critical appraisal tool for systematic reviews (SRs) and meta-analyses (MAs) of interventions. We aimed to perform the first AMSTAR 2-based quality assessment of heart failure-related studies.MethodsEleven high-impact journals were searched from 2009 to 2019. The included studies were assessed on the basis of 16 domains. Seven domains were deemed critical for high-quality studies. On the basis of the performance in these 16 domains with different weights, overall ratings were generated, and the quality was determined to be "high," "moderate," "low," or "critically low."ResultsEighty-one heart failure-related SRs with MAs were included. Overall, 79 studies were of "critically low quality" and two were of "low quality." These findings were attributed to insufficiency in the following critical domains: a priori protocols (compliance rate, 5%), complete list of exclusions with justification (5%), risk of bias assessment (69%), meta-analysis methodology (78%), and investigation of publication bias (60%).ConclusionsThe low ratings for these potential high-quality heart failure-related SRs and MAs challenge the discrimination capacity of AMSTAR 2. In addition to identifying certain areas of insufficiency, these findings indicate the need to justify or modify AMSTAR 2's rating rules.
| S-EPMC9308914 | biostudies-literature
Project description:BackgroundThe quality of nutrition-related systematic reviews (SRs) is an unstudied but important factor affecting their usefulness.ObjectivesThe objectives were to evaluate the reporting quality of published SRs and to identify areas of improvement.DesignDescriptive and exploratory analyses of the reporting quality (7 nutrition items and 28 SR reporting items) of all English-language SRs published through July 2007 linking micronutrients and health outcomes in humans were conducted. Factors that may be associated with reporting quality were also evaluated.ResultsWe identified 141 eligible SRs of 21 micronutrients. Ninety SRs that included only interventional studies met a higher proportion of our reporting criteria (median: 62%; interquartile range: 51%, 72%) than did 31 SRs with only observational studies (median: 53%; interquartile range: 47%, 60%) or 20 SRs with both study designs (median: 47%; interquartile range: 39%, 52%) (P < 0.001). SRs published after consensus reporting standards (since 2003) met a higher proportion of the reporting criteria than did earlier SRs (median: 59% compared with 50%; P = 0.01); however, the reporting of nutrition variables remained unchanged (median: 38% compared with 33%; P = 0.7). The least-reported nutrition criteria were baseline nutrient exposures (28%) and effects of measurement errors from nutrition exposures (24%). Only 58 SRs (41%) used quality scales or checklists to assess the methodologic quality of the primary studies included.ConclusionsThe reporting quality of SRs has improved 3 y after publication of SR reporting standards, but the reporting of nutrition variables has not. Improved adherence to consensus methods and reporting standards should improve the utility of nutrition SRs.
| S-EPMC2667458 | biostudies-literature
Project description:BackgroundPostpartum depression (PPD) is a highly prevalent mental health problem that affects parental health with implications for child health in infancy, childhood, adolescence and beyond. The primary aim of this study was to critically appraise available systematic reviews describing interventions for PPD. The secondary aim was to evaluate the methodological quality of the included systematic reviews and their conclusions.MethodsAn electronic database search of MEDLINE, Embase, and the Cochrane Library from 2000 to 2020 was conducted to identify systematic reviews that examined an intervention for PPD. A Measurement Tool to Assess Systematic Reviews was utilized to independently score each included systematic review which was then critically appraised to better define the most effective therapeutic options for PPD.ResultsOf the 842 studies identified, 83 met the a priori criteria for inclusion. Based on the systematic reviews with the highest methodological quality, we found that use of antidepressants and telemedicine were the most effective treatments for PPD. Symptoms of PPD were also improved by traditional herbal medicine and aromatherapy. Current evidence for physical exercise and cognitive behavioural therapy in treating PPD remains equivocal. A significant, but weak relationship between AMSTAR score and journal impact factor was observed (p = 0.03, r = 0.24; 95% CI, 0.02 to 0.43) whilst no relationship was found between the number of total citations (p = 0.27, r = 0.12; 95% CI, - 0.09 to 0.34), or source of funding (p = 0.19).ConclusionOverall the systematic reviews on interventions for PPD are of low-moderate quality and are not improving over time. Antidepressants and telemedicine were the most effective therapeutic interventions for PPD treatment.
| S-EPMC7789727 | biostudies-literature