Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications.
Ontology highlight
ABSTRACT: Objective
To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications.Design
Cohort of protocols of randomised controlled trial and subsequent full journal publications.Setting
Six research ethics committees in Switzerland, Germany, and Canada.Data sources
894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications.Results
Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis.Conclusions
Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.
SUBMITTER: Kasenda B
PROVIDER: S-EPMC4100616 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:OBJECTIVE:To determine (i) the difference in the frequency of serious adverse events (SAEs) reported in trial registrations and their respective primary publications and (ii) the effect of adding SAE data from registries to a network meta-analysis (NMA) in changing the surface under the cumulative ranking (SUCRA) curve values of interventions. DESIGN:Secondary analysis of primary publications from two NMAs. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:We included randomised trials published in English after 2005 that were included in two NMAs of pharmacological interventions for Alzheimer's disease and chronic obstructive pulmonary disease. DATA EXTRACTION:Two reviewers independently searched multiple international trial registries for registration status and abstracted data from the included study publications and ClinicalTrials.gov. RESULTS:Of the 203 randomised trials included, 140 (69.0%) were registered with a trial registry and 72 (35.5%) posted results in the registry. The proportion of registered trials increased over time (38.5% in 2005 vs 78.6% in 2014). Of the publications with results posted in a trial registry, 14 (19.4%) had inconsistent reporting of overall SAEs; 7 (10.4%) studies did not report SAEs in the publication but did in the registry. In the 134 randomised trials with a prespecified primary outcome in the registry, 19 studies (9.4%) had a change in the primary outcome in the publication. Adding SAEs reported in registries to the NMAs did not affect the ranking of interventions. CONCLUSION:We identified inconsistent reporting of SAEs in randomised trials that were included in two NMAs. Findings highlight the importance of including trial registries in the grey literature search and verifying safety data before incorporating it into NMAs. STUDY REGISTRATION:osf.io/mk6dr.
| S-EPMC6731894 | biostudies-other
Project description:ObjectiveTo investigate to what extent evidence from previous similar trials or systematic reviews was considered before conducting new trials.DesignCohort study of contemporary protocols for trials with ethical approval.MethodsAll protocols for randomised trials approved by the five ethical committees in Denmark between January 2012 and March 2013 were screened for eligibility. Included protocols were read in full to determine whether a systematic search had been conducted and references were checked to evaluate whether trial rationale and design could be challenged for not adequately considering previous evidence. To investigate whether protocols cited relevant trials, we used simple search strategies that could easily be conducted by researchers without experience with literature searches.ResultsSixty-seven protocols were included. Only two (3%) of the protocols explicitly stated to have conducted a literature search and only one (1%) provided information that allowed the search to be replicated. Eleven (16%) of the protocols described trials where we found the information insufficient to judge if the trial was ethically justified, either due to a comparator that was not supported by the presented evidence (six protocols), because they did not present a rationale for conducting the trial (two protocols), or for both reasons (three protocols). For eight (12%) of the protocols, our search identified trials that could have been relevant to cite as justification.ConclusionsWhile most protocols seem to adequately consider existing evidence, a substantial minority of trials might lack a sufficient evidence base. Very few trials seemed to have been based on a literature search which makes it impossible to know whether all relevant previous trials had been considered. Rules for ethical approval should include requirements for systematic literature searches to ensure that research participants are not exposed to sub-optimal treatments or unnecessary harms as well as to reduce research waste.
| S-EPMC6858261 | biostudies-literature
Project description:ObjectiveTo investigate the impact of industry funding on reporting of subgroup analyses in randomised controlled trials.DesignSystematic review.Data sourcesMedline.Study selectionRandomised controlled trials published in 118 core clinical journals (defined by the National Library of Medicine) in 2007. 1140 study reports in a 1:1 ratio by high (five general medicine journals with largest number of total citations in 2007) versus lower impact journals, were randomly sampled. Two reviewers, independently and in duplicate, used standardised, piloted forms to screen study reports for eligibility and to extract data. They also used explicit criteria to determine whether a randomised controlled trial reported subgroup analyses. Logistic regression was used to examine the association of prespecified study characteristics with reporting versus not reporting of subgroup analyses.Results469 randomised controlled trials were included, of which 207 (44%) reported subgroup analyses. High impact journals (adjusted odds ratio 2.64, 95% confidence interval 1.62 to 4.33), non-surgical (versus surgical) trials (2.10, 1.26 to 3.50), and larger sample size (3.38, 1.64 to 6.99) were associated with more frequent reporting of subgroup analyses. The strength of association between trial funding and reporting of subgroups differed in trials with and without statistically significant primary outcomes (interaction P=0.02). In trials without statistically significant results for the primary outcome, industry funded trials were more likely to report subgroup analyses (2.29, 1.30 to 4.72) than non-industry funded trials. This was not true for trials with a statistically significant primary outcome (0.79, 0.46 to 1.36). Industry funded trials were associated with less frequent prespecification of subgroup hypotheses (31.3% v 38.0%, adjusted odds ratio 0.49, 0.26 to 0.94), and less use of the interaction test for analyses of subgroup effects (41.4% v 49.1%, 0.52, 0.28 to 0.97) than non-industry funded trials.ConclusionIndustry funded randomised controlled trials, in the absence of statistically significant primary outcomes, are more likely to report subgroup analyses than non-industry funded trials. Industry funded trials less frequently prespecify subgroup hypotheses and less frequently test for interaction than non-industry funded trials. Subgroup analyses from industry funded trials with negative results for the primary outcome should be viewed with caution.
| S-EPMC6173170 | biostudies-literature
Project description:ObjectiveTo improve communication of harm in publications of randomised controlled trials via the development of recommendations for visually presenting harm outcomes.DesignConsensus study.Setting15 clinical trials units registered with the UK Clinical Research Collaboration, an academic population health department, Roche Products, and The BMJ.ParticipantsExperts in clinical trials: 20 academic statisticians, one industry statistician, one academic health economist, one data graphics designer, and two clinicians.Main outcomemeasures A methodological review of statistical methods identified visualisations along with those recommended by consensus group members. Consensus on visual recommendations was achieved (at least 60% of the available votes) over a series of three meetings with participants. The participants reviewed and critically appraised candidate visualisations against an agreed framework and voted on whether to endorse each visualisation. Scores marginally below this threshold (50-60%) were revisited for further discussions and votes retaken until consensus was reached.Results28 visualisations were considered, of which 10 are recommended for researchers to consider in publications of main research findings. The choice of visualisations to present will depend on outcome type (eg, binary, count, time-to-event, or continuous), and the scenario (eg, summarising multiple emerging events or one event of interest). A decision tree is presented to assist trialists in deciding which visualisations to use. Examples are provided of each endorsed visualisation, along with an example interpretation, potential limitations, and signposting to code for implementation across a range of standard statistical software. Clinician feedback was incorporated into the explanatory information provided in the recommendations to aid understanding and interpretation.ConclusionsVisualisations provide a powerful tool to communicate harms in clinical trials, offering an alternative perspective to the traditional frequency tables. Increasing the use of visualisations for harm outcomes in clinical trial manuscripts and reports will provide clearer presentation of information and enable more informative interpretations. The limitations of each visualisation are discussed and examples of where their use would be inappropriate are given. Although the decision tree aids the choice of visualisation, the statistician and clinical trial team must ultimately decide the most appropriate visualisations for their data and objectives. Trialists should continue to examine crude numbers alongside visualisations to fully understand harm profiles.
| S-EPMC9108928 | biostudies-literature
Project description:OBJECTIVES:To assess whether a journal club model could improve comprehension and written and spoken medical English in a population of Chinese medical professionals. SETTING AND PARTICIPANTS:The study population consisted of 52 medical professionals who were residents or postgraduate master or PhD students in the Department of Obstetrics and Gynecology, Heilongjiang University of Chinese Medicine, China. INTERVENTION:After a three-part baseline examination to assess medical English comprehension, participants were randomised to either (1) an intensive journal club treatment arm or (2) a self-study group. At the conclusion of the 8-week intervention participants (n=52) were re-tested with new questions. OUTCOME MEASURES:The primary outcome was the change in score on a multiple choice examination. Secondary outcomes included change in scores on written and oral examinations which were modelled on the Test of English as a Foreign Language (TOEFL). RESULTS:Both groups had improved scores on the multiple choice examination without a statistically significant difference between them (90% power). However, there was a statistically significant difference between the groups in mean improvement in scores for both written (95% CI 1.1 to 5.0; p=0.003) and spoken English (95% CI 0.06 to 3.7; p=0.04) favouring the journal club intervention. CONCLUSIONS:Interacting with colleagues and an English-speaking facilitator in a journal club improved both written and spoken medical English in Chinese medical professionals. Journal clubs may be suitable for use as a self-sustainable teaching model to improve fluency in medical English in foreign medical professionals. TRIAL REGISTRATION NUMBER:NCT01844609.
| S-EPMC4735128 | biostudies-literature
Project description:BackgroundClinical trials are one of the most important sources of evidence for guiding evidence-based practice and the design of new trials. However, most of this information is available only in free text - e.g., in journal publications - which is labour intensive to process for systematic reviews, meta-analyses, and other evidence synthesis studies. This paper presents an automatic information extraction system, called ExaCT, that assists users with locating and extracting key trial characteristics (e.g., eligibility criteria, sample size, drug dosage, primary outcomes) from full-text journal articles reporting on randomized controlled trials (RCTs).MethodsExaCT consists of two parts: an information extraction (IE) engine that searches the article for text fragments that best describe the trial characteristics, and a web browser-based user interface that allows human reviewers to assess and modify the suggested selections. The IE engine uses a statistical text classifier to locate those sentences that have the highest probability of describing a trial characteristic. Then, the IE engine's second stage applies simple rules to these sentences to extract text fragments containing the target answer. The same approach is used for all 21 trial characteristics selected for this study.ResultsWe evaluated ExaCT using 50 previously unseen articles describing RCTs. The text classifier (first stage) was able to recover 88% of relevant sentences among its top five candidates (top5 recall) with the topmost candidate being relevant in 80% of cases (top1 precision). Precision and recall of the extraction rules (second stage) were 93% and 91%, respectively. Together, the two stages of the extraction engine were able to provide (partially) correct solutions in 992 out of 1050 test tasks (94%), with a majority of these (696) representing fully correct and complete answers.ConclusionsOur experiments confirmed the applicability and efficacy of ExaCT. Furthermore, they demonstrated that combining a statistical method with 'weak' extraction rules can identify a variety of study characteristics. The system is flexible and can be extended to handle other characteristics and document types (e.g., study protocols).
| S-EPMC2954855 | biostudies-literature
Project description:OBJECTIVE:Randomised controlled trials (RCT) are the gold standard to provide unbiased data. However, when patients have a treatment preference, randomisation may influence participation and outcomes (eg, external and internal validity). The aim of this study was to assess the influence of patients' preference in RCTs by analysing partially randomised patient preference trials (RPPT); an RCT and preference cohort combined. DESIGN:Systematic review and meta-analyses. DATA SOURCES:MEDLINE, Embase, PsycINFO and the Cochrane Library. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:RPPTs published between January 2005 and October 2018 reporting on allocation of patients to randomised and preference cohorts were included. DATA EXTRACTION AND SYNTHESIS:Two independent reviewers extracted data. The main outcomes were the difference in external validity (participation and baseline characteristics) and internal validity (lost to follow-up, crossover and the primary outcome) between the randomised and the preference cohort within each RPPT, compared in a meta-regression using a Wald test. Risk of bias was not assessed, as no quality assessment for RPPTs has yet been developed. RESULTS:In total, 117 of 3734 identified articles met screening criteria and 44 were eligible (24 873 patients). The participation rate in RPPTs was >95% in 14 trials (range: 48%-100%) and the randomisation refusal rate was >50% in 26 trials (range: 19%-99%). Higher education, female, older age, race and prior experience with one treatment arm were characteristics of patients declining randomisation. The lost to follow-up and cross-over rate were significantly higher in the randomised cohort compared with the preference cohort. Following the meta-analysis, the reported primary outcomes were comparable between both cohorts of the RPPTs, mean difference 0.093 (95% CI -0.178 to 0.364, p=0.502). CONCLUSIONS:Patients' preference led to a substantial proportion of a specific patient group refusing randomisation, while it did not influence the primary outcome within an RPPT. Therefore, RPPTs could increase external validity without compromising the internal validity compared with RCTs. PROSPERO REGISTRATION NUMBER:CRD42019094438.
| S-EPMC6797441 | biostudies-other
Project description:BackgroundThe number of publications on clinical trials is unknown as well as the countries publishing most trial reports. To try to examine these questions we performed an ecological study.MethodsWe searched the 454,449 records on publications in The Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, Issue 3, 2005 (CD-ROM version) for possible country of origin. We inspected a random sample of 906 records for information on country and type of trial.ResultsThere was an exponential growth of publications on randomised controlled trials and controlled clinical trials since 1946, but the growth seems to have seized since 2000. We identified the possible country of origin of 210,974 publications (46.4%). The USA is leading with about 46,789 publications followed by UK, Germany, Italy, the Netherlands, Canada, and France. Sweden becomes the leader with 891 publications per million inhabitants during the last 60 years followed by Denmark (n = 864), New Zealand (n = 791), Finland (n = 781), the Netherlands (n = 570), Switzerland (n = 547), and Norway (n = 543). In depth assessment of the random sample backed these findings.ConclusionMany records lacked country of origin, even after the additional scrutiny. The number of publications on clinical trials increased exponentially until the turn of the century. Rather small, democratic, and wealthy countries take the lead when the number of publications on clinical trials is calculated based on million inhabitants. If all countries produced the same number of trials as these countries, this could mean thousands of new effective treatments during the next 60 years.
| S-EPMC1808475 | biostudies-literature
Project description:BackgroundSubgroup analyses are frequently used to assess heterogeneity of treatment effects in randomised clinical trials. Inconsistent, improper and incomplete implementation, reporting and interpretation have been identified as ongoing challenges. Further, subgroup analyses were frequently criticised because of unreliable or potentially misleading results. More recently, recommendations and guidelines have been provided to improve the reporting of data in this regard.MethodsThis systematic review was based on a literature search within the digital archives of three selected medical journals, The New England Journal of Medicine, The Lancet and Circulation. We reviewed articles of randomised clinical trials in the domain of cardiovascular disease which were published in 2015 and 2016. We screened and evaluated the selected articles for the mode of implementation and reporting of subgroup analyses.ResultsWe were able to identify a total of 130 eligible publications of randomised clinical trials. In 89/130 (68%) articles, results of at least one subgroup analysis were presented. This was dependent on the considered journal (p < 0.001), the number of included patients (p < 0.001) and the lack of statistical significance of a trial's primary analysis (p < 0.001). The number of reported subgroup analyses ranged from 1 to 101 (median = 13). We were able to comprehend the specification time of reported subgroup analyses for 71/89 (80%) articles, with 55/89 (62%) articles presenting exclusively pre-specified analyses. This information was not always traceable on the basis of provided trial protocols and often did not include the pre-definition of cut-off values for the categorization of subgroups. The use of interaction tests was reported in 84/89 (94%) articles, with 36/89 (40%) articles reporting heterogeneity of the treatment effect for at least one primary or secondary trial outcome. Subgroup analyses were reported more frequently for larger randomised clinical trials, and if primary analyses did not reach statistical significance. Information about the implementation of subgroup analyses was reported most consistently for articles from The New England Journal of Medicine, since it was also traceable on the basis of provided trial protocols. We were able to comprehend whether subgroup analyses were pre-specified in a majority of the reviewed publications. Even though results of multiple subgroup analyses were reported for most published trials, a corresponding adjustment for multiple testing was rarely considered.ConclusionCompared to previous reviews in this context, we observed improvements in the reporting of subgroup analyses of cardiovascular randomised clinical trials. Nonetheless, critical shortcomings, such as inconsistent reporting of the implementation and insufficient pre-specification, persist.
| S-EPMC8174013 | biostudies-literature
Project description:AimsThis study aimed to investigate the reporting of subgroup analyses in heart failure (HF) randomized controlled trials (RCTs) and to determine the strength and credibility of subgroup claims.Methods and resultsAll primary HF RCTs published in nine high-impact journals from 1 January 2008 to 31 December 2017 were included. Multivariable regression analysis was used to identify factors that may favour the reporting of results in specific subgroups. Strength of the subgroup effect claimed was classified into (i) strong, (ii) likely, or (iii) suggestive. Credibility of subgroup claim was scored using a pre-specified 10 pointer criteria. Of the 261 HF RCTs studied, 107 (41%) reported subgroup analyses. Twenty-five (23%) RCTs claimed a subgroup effect for the primary outcome of which six (24%) made a strong claim, eight (32%) claimed a likely effect, and 11 (44%) suggested a possible subgroup effect. Seven of the 25 RCTs did not employ interaction testing for subgroup claims of the primary outcome. Three out of 10 pre-specified credibility criteria were satisfied by half of the trials. Fourteen trials justified the choice of subgroups, and 10 explicitly stated they were underpowered to detect differences within subgroups. Source of funding did not influence the frequency of reporting subgroup analyses (OR 0.53, 95% CI 0.78-3.62, P = 0.52).ConclusionsAppropriate credibility criteria were rarely met even by HF RCTs that held strong subgroup claims. Subgroup analyses should be pre-specified, be adequately powered, present interaction terms, and be replicated in independent data before being integrated into clinical decision making.
| S-EPMC7835611 | biostudies-literature