Dataset Information

Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study.

ABSTRACT: Background: The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool's application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people. Methods: Interventions were assessed independently by two authors using the six core iCAT_SR dimensions: (1) 'Target organisational levels/categories'; (2) 'Target behaviour/actions'; (3) 'Active intervention components'; (4) 'Degree of tailoring'; (5) 'Level of skill required by intervention deliverers'; (6) 'Level of skill required by intervention recipients'. Attempts were made to apply four optional dimensions: 'Interaction between intervention components'; 'Context/setting'; 'Recipient/provider factors'; 'Nature of causal pathway'. Inter-rater reliability was assessed using Cohen's Kappa coefficient. Disagreements were resolved by consensus discussion. The findings are presented narratively. Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Interventions were delivered across various organisational levels and categories (i.e. healthcare professionals and patients) and typically comprised multiple components. Intermediate skill levels were required by those delivering and receiving the interventions across all studies. A lack of detail in study reports precluded application of the iCAT_SR's optional dimensions. The inter-rater reliability was substantial (Cohen's Kappa = 0.75) Conclusions: This study describes the application of the iCAT_SR to studies included in a Cochrane systematic review. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic understanding of intervention complexity and facilitate replication in other settings. The experience gained has helped to refine the original guidance document relating to the application of iCAT_SR.

SUBMITTER: Cadogan CA

PROVIDER: S-EPMC7309054 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study.

Cadogan Cathal A CA Rankin Audrey A Lewin Simon S Hughes Carmel M CM

HRB open research 20200601

<b>Background:</b> The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool's application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people. <b>Methods:</b> Interventi ...[more]

PMID: 32596632

Similar Datasets

Project description:BACKGROUND:There is growing interest in evaluating differences in healthcare interventions across routinely collected demographic characteristics. However, individual subgroup analyses in randomized controlled trials are often not prespecified, adjusted for multiple testing, or conducted using the appropriate statistical test for interaction, and therefore frequently lack credibility. Meta-analyses can be used to examine the validity of potential subgroup differences by collating evidence across trials. Here, we characterize the conduct and clinical translation of age-treatment subgroup analyses in Cochrane reviews. METHODS:For a random sample of 928 Cochrane intervention reviews of randomized trials, we determined how often subgroup analyses of age are reported, how often these analyses have a P?<?0.05 from formal interaction testing, how frequently subgroup differences first observed in an individual trial are later corroborated by other trials in the same meta-analysis, and how often statistically significant results are included in commonly used clinical management resources (BMJ Best Practice, UpToDate, Cochrane Clinical Answers, Google Scholar, and Google search). RESULTS:Among 928 Cochrane intervention reviews, 189 (20.4%) included plans to conduct age-treatment subgroup analyses. The vast majority (162 of 189, 85.7%) of the planned analyses were not conducted, commonly because of insufficient trial data. There were 22 reviews that conducted their planned age-treatment subgroup analyses, and another 3 reviews appeared to perform unplanned age-treatment subgroup analyses. These 25 (25 of 928, 2.7%) reviews conducted a total of 97 age-treatment subgroup analyses, of which 65 analyses (in 20 reviews) had non-overlapping subgroup levels. Among the 65 age-treatment subgroup analyses, 14 (21.5%) did not report any formal interaction testing. Seven (10.8%) reported P?<?0.05 from formal age-treatment interaction testing; however, none of these seven analyses were in reviews that discussed the potential biological rationale or clinical significance of the subgroup findings or had results that were included in common clinical practice resources. CONCLUSION:Age-treatment subgroup analyses in Cochrane intervention reviews were frequently planned but rarely conducted, and implications of detected interactions were not discussed in the reviews or mentioned in common clinical resources. When subgroup analyses are performed, authors should report the findings, compare the results to previous studies, and outline any potential impact on clinical care.

Project description:BackgroundHealth interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews.MethodsWe developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool.ResultsThe tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions.ConclusionsThe iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

Project description:ObjectiveTo compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.DesignMethodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.Data sourcesCancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.Eligibility criteria for selecting studiesWe included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.ResultsWe included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95%?CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95%?CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95%?CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95%?CI 0.02 to 0.09), use individual patient data (RR 0.03; 95%?CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95%?CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0-143); high-impact journal SRs: 74.45 (0-652)).ConclusionsWhen comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.

Project description:ObjectivesHigh-quality reporting of treatment details can aid replication of study results in real-world clinical practice. The Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) is a reporting guideline for key elements of acupuncture interventions in clinical trials. This study used STRICTA to investigate whether Cochrane reviews of acupuncture adequately report important treatment details.DesignSystematic reviewMethodsCochrane reviews of acupuncture were identified from The Cochrane Library (issue 7, 2012). Randomised controlled trials (RCTs) included in the reviews and published after 2005 were obtained. Using STRICTA, we extracted acupuncture-related information from the Cochrane reviews and the RCTs. The characteristics of the included studies' table were the major source of intervention information from Cochrane reviews. Reporting quality of acupuncture interventions in Cochrane reviews was assessed and compared with the respective RCTs.Results25 Cochrane reviews of acupuncture and 92 RCTs met the selection criteria. Cochrane reviews were 16% less likely to report the acupuncture-related items of STRICTA than RCTs (risk ratio 0.84, 95% CI 0.79 to 0.88, I(2)=8%). Information was significantly better reported for 10 of the 15 treatment-group items of STRICTA in RCTs than in Cochrane reviews (p<0.05), while four items did so without statistical significance. One item related to practitioner background was significantly better reported in Cochrane reviews.ConclusionsReporting quality of treatment details in Cochrane reviews of acupuncture was insufficient with regard to STRICTA, even though such information was readily reported in RCTs. The overall quality of reporting of the RCTs, while better than the reviews, was also often suboptimal. Use of STRICTA guideline during the review process is recommended to adequately report the key treatment components in Cochrane reviews of acupuncture. The potential impact of STRICTA to the replicability and utilisation of reviews in future research and practice needs to be investigated.

Dataset Information

Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study.

Publications

Application of the intervention Complexity Assessment Tool for Systematic Reviews within a Cochrane review: an illustrative case study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets