Dataset Information

Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR).

ABSTRACT:

Background

Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews.

Methods

We developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool.

Results

The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions.

Conclusions

The iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

SUBMITTER: Lewin S

PROVIDER: S-EPMC5406941 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background: The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews. Worked examples of the tool's application are needed to promote its use and refinement. The aim of this case study was to apply the iCAT_SR to a subset of 20 studies included in a Cochrane review of interventions aimed at improving appropriate polypharmacy in older people. Methods: Interventions were assessed independently by two authors using the six core iCAT_SR dimensions: (1) 'Target organisational levels/categories'; (2) 'Target behaviour/actions'; (3) 'Active intervention components'; (4) 'Degree of tailoring'; (5) 'Level of skill required by intervention deliverers'; (6) 'Level of skill required by intervention recipients'. Attempts were made to apply four optional dimensions: 'Interaction between intervention components'; 'Context/setting'; 'Recipient/provider factors'; 'Nature of causal pathway'. Inter-rater reliability was assessed using Cohen's Kappa coefficient. Disagreements were resolved by consensus discussion. The findings are presented narratively. Results: Assessments involving the core iCAT_SR dimensions showed limited consistency in intervention complexity across included studies, even when categorised according to clinical setting. Interventions were delivered across various organisational levels and categories (i.e. healthcare professionals and patients) and typically comprised multiple components. Intermediate skill levels were required by those delivering and receiving the interventions across all studies. A lack of detail in study reports precluded application of the iCAT_SR's optional dimensions. The inter-rater reliability was substantial (Cohen's Kappa = 0.75) Conclusions: This study describes the application of the iCAT_SR to studies included in a Cochrane systematic review. Future intervention studies need to ensure more detailed reporting of interventions, context and the causal pathways underlying intervention effects to allow a more holistic understanding of intervention complexity and facilitate replication in other settings. The experience gained has helped to refine the original guidance document relating to the application of iCAT_SR.

Project description:Introduction: Amphetamine-type stimulants (ATS) use is a global concern due to increased usage and the harm to physical, mental, and social well-being. The objective of this overview of systematic reviews is to summarise trial results of psychosocial interventions and describe their efficacy and safety. Methods: We searched seven bibliographic databases to November 2020 for systematic reviews examining ATS misuse treatment by psychosocial interventions. Given the apparent incompleteness of the included reviews, we undertook a supplemental meta-analysis of all eligible primary studies. Results: We included 11 systematic reviews of moderate to high quality and 39 primary studies which assessed the outcomes of psychosocial interventions on people who use ATS. The key findings include: (1) There were conflicting results about the effectiveness of psychosocial interventions among reviews, which may confuse decision-makers in selecting treatment. (2) In the supplemental meta-analysis, relative to usual care (only counselling or self-help materials), membership of a psychological intervention group was associated with an important reduction in drug usage [risk ratio (RR) 0.80, 95% CI: 0.75 to 0.85]. Patients in psychological interventions used injectables substantially less [odds ratio (OR) 0.35, 95% CI: 0.24 to 0.49]. The risk of unsafe sex in the psychosocial intervention group was lower than in the control group (RR 0.49, 95% CI: 0.34 to 0.71). The combination of therapies reduced 1.51 day using drugs in the preceding 30 days (95% CI: -2.36 to -0.67) compared to cognitive behavioural therapy intervention alone. (3) Compared to usual care, cognitive behavioural therapy was less likely to be retained at follow-up (RR 0.89, 95% CI: 0.82 to 0.97; high-quality evidence). However, the additional of contingency management strategy can make an important improvement upon retention (RR 1.42, 95%CI: 1.25 to 1.62). Authors' Conclusions: Integrated models are more effective than a single-treatment strategy. Comprehensive and sustained psychosocial interventions can help to reduce use of ATS and other drugs, risk behaviours and mental disorders, and significantly improve treatment adherence.

Project description:BackgroundThe introduction of new medical technologies such as sensors has accelerated the process of collecting patient data for relevant clinical decisions, which has led to the introduction of a new technology known as digital biomarkers.ObjectiveThis study aims to assess the methodological quality and quality of evidence from meta-analyses of digital biomarker-based interventions.MethodsThis study follows the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline for reporting systematic reviews, including original English publications of systematic reviews reporting meta-analyses of clinical outcomes (efficacy and safety endpoints) of digital biomarker-based interventions compared with alternative interventions without digital biomarkers. Imaging or other technologies that do not measure objective physiological or behavioral data were excluded from this study. A literature search of PubMed and the Cochrane Library was conducted, limited to 2019-2020. The quality of the methodology and evidence synthesis of the meta-analyses were assessed using AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews 2) and GRADE (Grading of Recommendations, Assessment, Development, and Evaluations), respectively. This study was funded by the National Research, Development and Innovation Fund of Hungary.ResultsA total of 25 studies with 91 reported outcomes were included in the final analysis; 1 (4%), 1 (4%), and 23 (92%) studies had high, low, and critically low methodologic quality, respectively. As many as 6 clinical outcomes (7%) had high-quality evidence and 80 outcomes (88%) had moderate-quality evidence; 5 outcomes (5%) were rated with a low level of certainty, mainly due to risk of bias (85/91, 93%), inconsistency (27/91, 30%), and imprecision (27/91, 30%). There is high-quality evidence of improvements in mortality, transplant risk, cardiac arrhythmia detection, and stroke incidence with cardiac devices, albeit with low reporting quality. High-quality reviews of pedometers reported moderate-quality evidence, including effects on physical activity and BMI. No reports with high-quality evidence and high methodological quality were found.ConclusionsResearchers in this field should consider the AMSTAR-2 criteria and GRADE to produce high-quality studies in the future. In addition, patients, clinicians, and policymakers are advised to consider the results of this study before making clinical decisions regarding digital biomarkers to be informed of the degree of certainty of the various interventions investigated in this study. The results of this study should be considered with its limitations, such as the narrow time frame.International registered report identifier (irrid)RR2-10.2196/28204.

Project description:PurposeTo systematically summarize evidence from multiple systematic reviews (SRs) examining interventions addressing medication nonadherence and to discern differences in effectiveness by intervention, patient, and study characteristics.SummaryMEDLINE, the Cochrane Database of Systematic Reviews, and the Database of Abstracts of Reviews of Effects were searched for papers published from January 2004 to February 2017. English-language SRs examining benefits of medication adherence interventions were eligible. Inclusion was limited to adult patients prescribed medication for 1 of the following disease conditions: diabetes and prediabetes, heart conditions, hypertension and prehypertension, stroke, and cognitive impairment. Non-disease-specific SRs that considered medication adherence interventions for older adults, adults with chronic illness, and adults with known medication adherence problems were also included. Two researchers independently screened titles, abstracts, and full-text articles. They then extracted key variables from eligible SRs, reconciling discrepancies via discussion. A MeaSurement Tool to Assess systematic Reviews (AMSTAR) was used to assess SRs; those with scores below 8 were excluded. Conclusions regarding intervention effectiveness were extracted. Grades of Recommendation, Assessment, Development and Evaluation (GRADE) methodology was applied to assess evidence quality.ResultsOf 390 SRs, 25 met the inclusion criteria and assessed adherence as a primary outcome. Intervention types most consistently found to be effective were dose simplification, patient education, electronic reminders to patients, and reduced patient cost sharing or incentives. Of 50 conclusions drawn by the SRs, the underlying evidence was low or very low quality for 45 SRs.ConclusionDespite an abundance of primary studies and despite only examining high-quality SRs, the vast majority of primary studies supporting SR authors' conclusions were of low or very low quality. Nonetheless, health system leaders seeking to improve medication adherence should prioritize interventions that have been studied and found to be effective at improving patient adherence, including dose simplification, education, reminders, and financial incentives.

Project description:PURPOSE:To systematically evaluate and summarize evidence across multiple systematic reviews (SRs) examining interventions addressing polypharmacy. SUMMARY:MEDLINE, the Cochrane Database of Systematic Reviews, and the Database of Abstracts of Reviews of Effects (DARE) were searched for SRs evaluating interventions addressing polypharmacy in adults published from January 2004 to February 2017. Two authors independently screened, appraised, and extracted information. SRs with Assessment of Multiple Systematic Reviews (AMSTAR) scores below 8 were excluded. After extraction of relevant conclusions from each SR, evidence was summarized and conclusions compared. Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology was used to assess evidence quality. Six SRs met the inclusion criteria, 4 of which used meta-analytic pooling. Five SRs focused on older adults. Four were not restricted to any specific disease type, whereas 1 focused on proton pump inhibitors and another focused on patients with severe dementia. Care settings and measured outcomes varied widely. SRs examining the impact on patient-centered outcomes, including morbidity, mortality, patient satisfaction, and utilization, found inconsistent evidence regarding the benefit of polypharmacy interventions, but most concluded that interventions had either null or uncertain impact. Two SRs assessing medication appropriateness found very low-quality evidence of modest improvements with polypharmacy interventions. CONCLUSION:An overview of SRs of interventions to address polypharmacy found 6 recent and high-quality SRs, mostly focused on older adults, in which both process and outcome measures were used to evaluate interventions. Despite the low quality of evidence in the underlying primary studies, both SRs that assessed medication appropriateness found evidence that polypharmacy interventions improved it. However, there was no consistent evidence of any impact on downstream patient-centered outcomes such as healthcare utilization, morbidity, or mortality.

Project description:BackgroundMobile phone usage has been rapidly increasing worldwide. mHealth could efficiently deliver high-quality health care, but the evidence supporting its current effectiveness is still mixed.ObjectiveWe performed a systematic review of systematic reviews to assess the impact or effectiveness of mobile health (mHealth) interventions in different health conditions and in the processes of health care service delivery.MethodsWe used a common search strategy of five major scientific databases, restricting the search by publication date, language, and parameters in methodology and content. Methodological quality was evaluated using the Measurement Tool to Assess Systematic Reviews (AMSTAR) checklist.ResultsThe searches resulted in a total of 10,689 articles. Of these, 23 systematic reviews (371 studies; more than 79,665 patients) were included. Seventeen reviews included studies performed in low- and middle-income countries. The studies used diverse mHealth interventions, most frequently text messaging (short message service, SMS) applied to different purposes (reminder, alert, education, motivation, prevention). Ten reviews were rated as low quality (AMSTAR score 0-4), seven were rated as moderate quality (AMSTAR score 5-8), and six were categorized as high quality (AMSTAR score 9-11). A beneficial impact of mHealth was observed in chronic disease management, showing improvement in symptoms and peak flow variability in asthma patients, reducing hospitalizations and improving forced expiratory volume in 1 second; improving chronic pulmonary diseases symptoms; improving heart failure symptoms, reducing deaths and hospitalization; improving glycemic control in diabetes patients; improving blood pressure in hypertensive patients; and reducing weight in overweight and obese patients. Studies also showed a positive impact of SMS reminders in improving attendance rates, with a similar impact to phone call reminders at reduced cost, and improved adherence to tuberculosis and human immunodeficiency virus therapy in some scenarios, with evidence of decrease of viral load.ConclusionsAlthough mHealth is growing in popularity, the evidence for efficacy is still limited. In general, the methodological quality of the studies included in the systematic reviews is low. For some fields, its impact is not evident, the results are mixed, or no long-term studies exist. Exceptions include the moderate quality evidence of improvement in asthma patients, attendance rates, and increased smoking abstinence rates. Most studies were performed in high-income countries, implying that mHealth is still at an early stage of development in low-income countries.