Dataset Information

Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal.

ABSTRACT: OBJECTIVE:Psychotherapy research may suffer from factors such as a researcher's own therapy allegiance. The aim of this study was to evaluate if researcher allegiance (RA) was reported in meta-analyses and randomised controlled trials (RCTs) of psychotherapeutic treatments. DESIGN:Systematic approach using meta-analyses of different types of psychotherapies. DATA SOURCES:Medline, PsycINFO and Cochrane Database of Systematic Reviews. METHODS:We evaluated meta-analyses of RCTs regarding various types of psychotherapies. Meta-analyses were eligible if they included at least one RCT with RA and they were published in journals in Medline, PsycINFO and Cochrane Database of Systematic Reviews with an impact factor larger than 5. RESULTS:We identified 146 eligible meta-analyses that synthesised data from a total of 1198 unique RCTs. Only 25 of the meta-analyses (17.2%) reported allegiance and only 6 (4.1%) used a proper method to control its effect. Of the 1198 eligible primary RCTs, 793 (66.3%) were allegiant. Authors in 25 of these 793 RCTs (3.2%) reported their allegiance while only one study (0.2%) controlled for its effect. CONCLUSIONS:The vast majority among a group of published meta-analyses and RCTs of psychotherapeutic treatments seldom reported and evaluated the allegiance effect. The results of the present study highlight a major lack of this information in meta-analyses and their included studies, though meta-analyses perform slightly better than RCTs. Stringent guidelines should be adopted by journals in order to improve reporting and attenuate possible effects of RA in future research.

SUBMITTER: Dragioti E

PROVIDER: S-EPMC4458582 | biostudies-literature | 2015 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal.

Dragioti Elena E Dimoliatis Ioannis I Evangelou Evangelos E

BMJ open 20150601 6

<h4>Objective</h4>Psychotherapy research may suffer from factors such as a researcher's own therapy allegiance. The aim of this study was to evaluate if researcher allegiance (RA) was reported in meta-analyses and randomised controlled trials (RCTs) of psychotherapeutic treatments.<h4>Design</h4>Systematic approach using meta-analyses of different types of psychotherapies.<h4>Data sources</h4>Medline, PsycINFO and Cochrane Database of Systematic Reviews.<h4>Methods</h4>We evaluated meta-analyses ...[more]

PMID: 26033943

Similar Datasets

Project description:OBJECTIVE:Randomised controlled trials (RCT) are the gold standard to provide unbiased data. However, when patients have a treatment preference, randomisation may influence participation and outcomes (eg, external and internal validity). The aim of this study was to assess the influence of patients' preference in RCTs by analysing partially randomised patient preference trials (RPPT); an RCT and preference cohort combined. DESIGN:Systematic review and meta-analyses. DATA SOURCES:MEDLINE, Embase, PsycINFO and the Cochrane Library. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:RPPTs published between January 2005 and October 2018 reporting on allocation of patients to randomised and preference cohorts were included. DATA EXTRACTION AND SYNTHESIS:Two independent reviewers extracted data. The main outcomes were the difference in external validity (participation and baseline characteristics) and internal validity (lost to follow-up, crossover and the primary outcome) between the randomised and the preference cohort within each RPPT, compared in a meta-regression using a Wald test. Risk of bias was not assessed, as no quality assessment for RPPTs has yet been developed. RESULTS:In total, 117 of 3734 identified articles met screening criteria and 44 were eligible (24 873 patients). The participation rate in RPPTs was >95% in 14 trials (range: 48%-100%) and the randomisation refusal rate was >50% in 26 trials (range: 19%-99%). Higher education, female, older age, race and prior experience with one treatment arm were characteristics of patients declining randomisation. The lost to follow-up and cross-over rate were significantly higher in the randomised cohort compared with the preference cohort. Following the meta-analysis, the reported primary outcomes were comparable between both cohorts of the RPPTs, mean difference 0.093 (95% CI -0.178 to 0.364, p=0.502). CONCLUSIONS:Patients' preference led to a substantial proportion of a specific patient group refusing randomisation, while it did not influence the primary outcome within an RPPT. Therefore, RPPTs could increase external validity without compromising the internal validity compared with RCTs. PROSPERO REGISTRATION NUMBER:CRD42019094438.

Project description:IntroductionOne potential source of bias in randomised clinical trials of psychological interventions is researcher allegiance (RA). The operationalisation of RA differs strongly across studies, and there is not a generally accepted method of operationalising or measuring it. Furthermore, it remains unclear as to how RA affects the outcomes of trials and if it results in better outcomes for a preferred intervention. The aim of this project is to develop and validate a scale that accurately identifies RA, contribute to the understanding of the impact that RA has in a research setting and to make recommendations for addressing RA in practice.Methods and analysisA scale will first be developed and validated to measure RA in psychotherapy trials. The scale will be validated by surveying authors of psychotherapy trials to assess their opinions, beliefs and preferences of psychotherapy interventions. Furthermore, the scale will be validated for use outside the field of psychotherapy. The validated checklist will then be used to examine two potential mechanisms of how RA may affect outcomes of interventions: publication bias (by assessing grants) and risk of bias (RoB). Finally, recommendations will be developed, and a feasibility study will be conducted at a national mental health agency in The Netherlands. Main analyses comprise inter-rater reliability of checklist items, correlations to examine the relationship between checklist items and author survey (convergent validity) as well as checklist items and trial outcomes and multivariate meta-regression techniques to assess potential mechanisms of how allegiance affects trial outcomes (publication bias and RoB).Ethics and disseminationThis study has been reviewed and approved by the Scientific and Ethical Review Board (VCWE) at the Vrije Universiteit Amsterdam. Study result and advancements will also be published on the Open Science Framework. Furthermore, main findings will be disseminated through articles in international peer-reviewed open access journals. Results and recommendations will be communicated to the Cochrane Collaboration, the Campbell Collaboration and other funding agencies.

Project description:ObjectivePositive psychotherapy (PPT) aims at increasing positive affect, meaning and engagement. We aimed to synthesise the available evidence on PPT efficacy.DesignWe conducted a preregistered systematic literature search and meta-analysis of randomised controlled trials examining the efficacy of PPT for increasing positive (eg, satisfaction with life) or decreasing negative psychological outcomes (eg, depression).Data sourcesMedline, PsycINFO and Web of Science from 2006 (ie, inception of PPT) to February 2020 as well as related systematic reviews and meta-analyses.ResultsWe included 20 randomised controlled trial with a total of 1360 participants. Moderate effect sizes were found for increasing positive outcomes (g=-0.72, 95% CI: -1.31 to -0.14, k=10, numbers needed to treat (NNT)=2.55) and reducing negative outcomes (g=0.48, 95% CI: 0.18 to 0.78, k=8, NNT=3.76) when PPT was compared with waitlist control conditions at post-treatment assessment. When compared with active control conditions, PPT yielded large effect sizes for increasing positive outcomes (g=-0.92, 95% CI: -1.74 to -0.11, k=6, NNT=2.05) and reducing depression (g=0.94, 95% CI: 0.18 to 1.70, k=6, NNT=2.03) at post-treatment assessment. No significant differences in efficacy were found when compared with established treatments such as cognitive-behavioural therapy. Evidence was found to support an association between trial quality and effect sizes. For positive outcomes, higher trial quality was related to larger effect size. Whereas higher trial quality was associated with smaller effect size for depression. Follow-up assessments remained too scarce for most planned analyses.ConclusionsOur findings support the short-term efficacy of PPT. However, results are to be regarded with due caution in the light of low number of trials. More high-quality trials that assess efficacy at follow-ups are needed to draw firmer conclusions on the long-term efficacy of PPT.Prospero registration numberCRD42020173567.

Project description:ObjectivesTo investigate whether an authorship effect is found that leads to better performance in studies conducted by the original developers of the Patient Health Questionnaire (PHQ-9) (allegiant studies).DesignSystematic review with random effects bivariate diagnostic meta-analysis. Search strategies included electronic databases, examination of reference lists and forward citation searches.Inclusion criteriaIncluded studies provided sufficient data to calculate the diagnostic accuracy of the PHQ-9 against a gold standard diagnosis of major depression using the algorithm or the summed item scoring method at cut-off point 10.Data extractionDescriptive information, methodological quality criteria and 2×2?contingency tables.ResultsSeven allegiant and 20 independent studies reported the diagnostic performance of the PHQ-9 using the algorithm scoring method. Pooled diagnostic OR (DOR) for the allegiant group was 64.40, and 15.05 for non-allegiant studies group. The allegiance status was a significant predictor of DOR variation (p<0.0001).Five allegiant studies and 26 non-allegiant studies reported the performance of the PHQ-9 at recommended cut-off point of 10. Pooled DOR for the allegiant group was 49.31, and 24.96 for the non-allegiant studies. The allegiance status was a significant predictor of DOR variation (p=0.015).Some potential alternative explanations for the observed authorship effect including differences in study characteristics and quality were found, although it is not clear how some of them account for the observed differences.ConclusionsAllegiant studies reported better performance of the PHQ-9. Allegiance status was predictive of variation in the DOR. Based on the observed differences between independent and non-independent studies, we were unable to conclude or exclude that allegiance effects are present in studies examining the diagnostic performance of the PHQ-9. This study highlights the need for future meta-analyses of diagnostic validation studies of psychological measures to evaluate the impact of researcher allegiance in the primary studies.

Project description:OBJECTIVE:To assess the association between intake of non-sugar sweeteners (NSS) and important health outcomes in generally healthy or overweight/obese adults and children. DESIGN:Systematic review following standard Cochrane review methodology. DATA SOURCES:Medline (Ovid), Embase, Cochrane CENTRAL, WHO International Clinical Trials Registry Platform, Clinicaltrials.gov, and reference lists of relevant publications. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:Studies including generally healthy adults or children with or without overweight or obesity were eligible. Included study designs allowed for a direct comparison of no intake or lower intake of NSS with higher NSS intake. NSSs had to be clearly named, the dose had to be within the acceptable daily intake, and the intervention duration had to be at least seven days. MAIN OUTCOME MEASURES:Body weight or body mass index, glycaemic control, oral health, eating behaviour, preference for sweet taste, cancer, cardiovascular disease, kidney disease, mood, behaviour, neurocognition, and adverse effects. RESULTS:The search resulted in 13?941 unique records. Of 56 individual studies that provided data for this review, 35 were observational studies. In adults, evidence of very low and low certainty from a limited number of small studies indicated a small beneficial effect of NSSs on body mass index (mean difference -0.6, 95% confidence interval -1.19 to -0.01; two studies, n=174) and fasting blood glucose (-0.16 mmol/L, -0.26 to -0.06; two, n=52). Lower doses of NSSs were associated with lower weight gain (-0.09 kg, -0.13 to -0.05; one, n=17?934) compared with higher doses of NSSs (very low certainty of evidence). For all other outcomes, no differences were detected between the use and non-use of NSSs, or between different doses of NSSs. No evidence of any effect of NSSs was seen on overweight or obese adults or children actively trying to lose weight (very low to moderate certainty). In children, a smaller increase in body mass index z score was observed with NSS intake compared with sugar intake (-0.15, -0.17 to -0.12; two, n=528, moderate certainty of evidence), but no significant differences were observed in body weight (-0.60 kg, -1.33 to 0.14; two, n=467, low certainty of evidence), or between different doses of NSSs (very low to moderate certainty). CONCLUSIONS:Most health outcomes did not seem to have differences between the NSS exposed and unexposed groups. Of the few studies identified for each outcome, most had few participants, were of short duration, and their methodological and reporting quality was limited; therefore, confidence in the reported results is limited. Future studies should assess the effects of NSSs with an appropriate intervention duration. Detailed descriptions of interventions, comparators, and outcomes should be included in all reports. SYSTEMATIC REVIEW REGISTRATION:Prospero CRD42017047668.

Project description:BackgroundMeta-analysis is a growing approach to evidence synthesis and network meta-analysis in particular represents an important and developing method within Health Technology Assessment (HTA). Meta-analysis of survival data is usually performed using the individual summary statistic-the hazard ratio (HR) from each randomised controlled trial (RCT).ObjectivesThe objectives of this study are to: (i) review the methods and reporting of survival analyses in oncology RCTs; and (ii) assess the suitability and relevance of survival data reported in RCTs for inclusion into meta-analysis.MethodsFive oncology journals were searched to identify Phase III RCTs published between April and July 2015. Eligible studies included those that analysed a survival outcome.ResultsThirty-two RCTs reporting survival outcomes in cancer populations were identified. None of the publications reported details relating to a strategy for statistical model building, the goodness of fit of the final model, or final model validation for the analysis of survival outcomes. The majority of studies (88%) reported the use of Cox proportional hazards (PH) regression to analyse survival endpoints. However, most publications failed to report the validation of the statistical models in terms of the PH assumption.ConclusionsThis review highlights deficiencies in terms of reporting the methods and validity of survival analyses within oncology RCTs. We support previous recommendations to encourage authors to improve the reporting of survival analyses in journal publications. We also recommend that the final choice of a statistical model for survival should be informed by goodness of model fit to a given dataset, and that model assumptions are validated. The failure of trial investigators and statisticians to investigate the PH for RCT survival data is likely to result in clinical decisions based on inappropriate methods. The development of alternative approaches for the meta-analysis of survival outcomes when the PH assumption is implausible is required if valid clinical decisions are to be made.

Project description:BackgroundThere is considerable debate as to the relative merits of using randomised controlled trial (RCT) data as opposed to observational data in systematic reviews of adverse effects. This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of RCTs as compared to meta-analysis of observational studies.Methods and findingsSearches were carried out in ten databases in addition to reference checking, contacting experts, citation searches, and hand-searching key journals, conference proceedings, and Web sites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from RCTs could be directly compared, using the ratio of odds ratios, with the pooled estimate for the same adverse effect arising from observational studies. Nineteen studies, yielding 58 meta-analyses, were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% confidence interval 0.93-1.15). There was less discrepancy with larger studies. The symmetric funnel plot suggests that there is no consistent difference between risk estimates from meta-analysis of RCT data and those from meta-analysis of observational studies. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped (54/58, 93%). In terms of statistical significance, in nearly two-thirds (37/58, 64%), the results agreed (both studies showing a significant increase or significant decrease or both showing no significant difference). In only one meta-analysis about one adverse effect was there opposing statistical significance.ConclusionsEmpirical evidence from this overview indicates that there is no difference on average in the risk estimate of adverse effects of an intervention derived from meta-analyses of RCTs and meta-analyses of observational studies. This suggests that systematic reviews of adverse effects should not be restricted to specific study types. Please see later in the article for the Editors' Summary.

Dataset Information

Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal.

Publications

Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets