Dataset Information

Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study.

ABSTRACT:

Objectives

To compare how allocation concealment is described in publications of randomised clinical trials and corresponding protocols, and to estimate how often trial publications with unclear allocation concealment have adequate concealment according to the protocol.

Design

Cohort study of 102 sets of trial protocols and corresponding publications.

Setting

Protocols of randomised trials approved by the scientific and ethical committees for Copenhagen and Frederiksberg, 1994 and 1995.

Main outcome measures

Frequency of adequate, unclear, and inadequate allocation concealment and sequence generation in trial publications compared with protocols, and the proportion of protocols where methods were reported to be adequate but descriptions were unclear in the trial publications.

Results

96 of the 102 trials had unclear allocation concealment according to the trial publication. According to the protocols, 15 of these 96 trials had adequate allocation concealment (16%, 95% confidence interval 9% to 24%), 80 had unclear concealment (83%, 74% to 90%), and one had inadequate concealment. When retrospectively defined loose criteria for concealment were applied, 83 of the 102 trial publications had unclear concealment. According to their protocol, 33 of these 83 trials had adequate allocation concealment (40%, 29% to 51%), 49 had unclear concealment (59%, 48% to 70%), and one had inadequate concealment.

Conclusions

Most randomised clinical trials have unclear allocation concealment on the basis of the trial publication alone. Most of these trials also have unclear allocation concealment according to their protocol.

SUBMITTER: Pildal J

PROVIDER: S-EPMC557221 | biostudies-literature | 2005 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study.

Pildal Julie J Chan An-Wen AW Hróbjartsson Asbjørn A Forfang Elisabeth E Altman Douglas G DG Gøtzsche Peter C PC

BMJ (Clinical research ed.) 20050407 7499

<h4>Objectives</h4>To compare how allocation concealment is described in publications of randomised clinical trials and corresponding protocols, and to estimate how often trial publications with unclear allocation concealment have adequate concealment according to the protocol.<h4>Design</h4>Cohort study of 102 sets of trial protocols and corresponding publications.<h4>Setting</h4>Protocols of randomised trials approved by the scientific and ethical committees for Copenhagen and Frederiksberg, 1 ...[more]

PMID: 15817527

Similar Datasets

Project description:BackgroundLittle is known about how adverse events are summarised and reported in trials, as detailed information is usually considered confidential. We have acquired clinical study reports (CSRs) from the European Medicines Agency through the Freedom of Information Act. The CSRs describe the results of studies conducted as part of the application for marketing authorisation for the slimming pill orlistat. The purpose of this study was to study how adverse events were summarised and reported in study protocols, CSRs, and published papers of orlistat trials.Methods and findingsWe received the CSRs from seven randomised placebo controlled orlistat trials (4,225 participants) submitted by Roche. The CSRs consisted of 8,716 pages and included protocols. Two researchers independently extracted data on adverse events from protocols and CSRs. Corresponding published papers were identified on PubMed and adverse event data were extracted from this source as well. All three sources were compared. Individual adverse events from one trial were summed and compared to the totals in the summary report. None of the protocols or CSRs contained instructions for investigators on how to question participants about adverse events. In CSRs, gastrointestinal adverse events were only coded if the participant reported that they were "bothersome," a condition that was not specified in the protocol for two of the trials. Serious adverse events were assessed for relationship to the drug by the sponsor, and all adverse events were coded by the sponsor using a glossary that could be updated by the sponsor. The criteria for withdrawal due to adverse events were in one case related to efficacy (high fasting glucose led to withdrawal), which meant that one trial had more withdrawals due to adverse events in the placebo group. Finally, only between 3% and 33% of the total number of investigator-reported adverse events from the trials were reported in the publications because of post hoc filters, though six of seven papers stated that "all adverse events were recorded." For one trial, we identified an additional 1,318 adverse events that were not listed or mentioned in the CSR itself but could be identified through manually counting individual adverse events reported in an appendix. We discovered that the majority of patients had multiple episodes of the same adverse event that were only counted once, though this was not described in the CSRs. We also discovered that participants treated with orlistat experienced twice as many days with adverse events as participants treated with placebo (22.7 d versus 14.9 d, p-value < 0.0001, Student's t test). Furthermore, compared with the placebo group, adverse events in the orlistat group were more severe. None of this was stated in the CSR or in the published paper. Our analysis was restricted to one drug tested in the mid-1990s; our results might therefore not be applicable for newer drugs.ConclusionsIn the orlistat trials, we identified important disparities in the reporting of adverse events between protocols, clinical study reports, and published papers. Reports of these trials seemed to have systematically understated adverse events. Based on these findings, systematic reviews of drugs might be improved by including protocols and CSRs in addition to published articles.

Project description:BackgroundClinicians, patients, and policy-makers rely on published evidence from clinical trials to help inform decision-making. A lack of complete and transparent reporting of the investigated trial outcomes limits reproducibility of results and knowledge synthesis efforts, and contributes to outcome switching and other reporting biases. Outcome-specific extensions for the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT-Outcomes) and Consolidated Standards of Reporting Trials (CONSORT-Outcomes) reporting guidelines are under development to facilitate harmonized reporting of outcomes in trial protocols and reports. The aim of this review was to identify and synthesize existing guidance for trial outcome reporting to inform extension development.MethodsWe searched for documents published in the last 10 years that provided guidance on trial outcome reporting using: an electronic bibliographic database search (MEDLINE and the Cochrane Methodology Register); a grey literature search; and solicitation of colleagues using a snowballing approach. Two reviewers completed title and abstract screening, full-text screening, and data charting after training. Extracted trial outcome reporting guidance was compared with candidate reporting items to support, refute, or refine the items and to assess the need for the development of additional items.ResultsIn total, 1758 trial outcome reporting recommendations were identified within 244 eligible documents. The majority of documents were published by academic journals (72%). Comparison of each recommendation with the initial list of 70 candidate items led to the development of an additional 62 items, producing 132 candidate items. The items encompassed outcome selection, definition, measurement, analysis, interpretation, and reporting of modifications between trial documents. The total number of documents supporting each candidate item ranged widely (median 5, range 0-84 documents per item), illustrating heterogeneity in the recommendations currently available for outcome reporting across a large and diverse sample of sources.ConclusionsOutcome reporting guidance for clinical trial protocols and reports lacks consistency and is spread across a large number of sources that may be challenging to access and implement in practice. Evidence and consensus-based guidance, currently in development (SPIRIT-Outcomes and CONSORT-Outcomes), may help authors adequately describe trial outcomes in protocols and reports transparently and completely to help reduce avoidable research waste.

Project description:ObjectivesTo determine the reporting quality of published randomised controlled trial (RCT) protocols before and after the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement (2013), and any association with author, trial or journal factors.DesignMethodological study.Data sourcesMEDLINE, Embase and CENTRAL were electronically searched using optimised search strategies.Eligibility criteriaProtocols written for an RCT of living humans, published in full text in a peer-reviewed journal and published in the English language.Main outcomePrimary outcome was the overall proportion of checklist items which were adequately reported in RCT protocols published before and after the SPIRIT statement.Results300 RCT protocols were retrieved; 150 from the period immediately before the SPIRIT statement (9 July 2012 to 28 December 2012) and 150 from a recent period after the SPIRIT statement (25 January 2019 to 20 March 2019). 47.9% (95% CI, 46.5% to 49.3%) of checklist items were adequately reported in RCT protocols before the SPIRIT statement and 56.7% (95% CI, 54.9% to 58.5%) after the SPIRIT statement. This represents an 8.8% (95% CI, 6.6% to 11.1%; p<0.0001) mean improvement in the overall proportion of checklist items adequately reported since the SPIRIT statement. While 40% of individual checklist items had a significant improvement in adequate reporting after the SPIRIT statement, 11.3% had a significant deterioration and there were no RCT protocols in which all individual checklist items were complete. The factors associated with higher reporting quality of RCT protocols in multiple regression analysis were author expertise or experience in epidemiology or statistics, multicentre trials, longer protocol word length and publicly reported journal policy of compliance with the SPIRIT statement.ConclusionThe overall reporting quality of RCT protocols has significantly improved since the SPIRIT statement, although a substantial proportion of individual checklist items remain poorly reported. Continued and concerted efforts are required by journals, editors, reviewers and investigators to improve the completeness and transparency of RCT protocols.

Project description:BackgroundInconsistencies between a protocol and its umbrella review (UR) may mislead readers about the importance of findings or lead to false-positive results. Furthermore, not documenting and explaining inconsistencies in the UR could reduce its transparency. To our knowledge, no study has examined the methodological consistency of the protocols with their URs and assessed the transparency of the URs when generating evidence.ObjectiveThis study aimed to investigate the inconsistency of protocols with their URs in the methodology and assess the transparency of the URs.MethodsWe searched medical-related electronic databases from their inception to January 1, 2022. We investigated inconsistencies between protocols and their publications and transparencies in the search strategy, inclusion criteria, methods of screening and data extraction, quality assessment, and statistical analysis.ResultsWe included 31 protocols and 35 publications. For the search strategy, 39 inconsistencies between the protocols and their publications were found in 26 of the 35 (74%) URs, and 16 of these inconsistencies were indicated and explained. There were 84 inconsistencies between the protocols and their URs regarding the inclusion criteria in 31 of the 35 (89%) URs, and 29 of the inconsistencies were indicated and explained. Deviations from their protocols were found in 12 of the 32 (38%) URs reporting the methods of screening, 14 of the 30 (47%) URs reporting the methods of data extraction, and 11 of the 32 (34%) URs reporting the methods for quality assessment. Of the 35 URs, 6 (17%) were inconsistent with their protocols in terms of the tools for quality assessment; one-half (3/6, 50%) of them indicated and explained the deviations. As for the statistical analysis, 31 of the 35 (89%) URs generated 61 inconsistencies between the publications and their protocols, and 16 inconsistencies were indicated and explained.ConclusionsThere was a high prevalence of inconsistencies between protocols and publications of URs, and more than one-half of the inconsistencies were not indicated and explained in the publications. Therefore, how to promote the transparency of URs will be a major part of future work.

Project description:BackgroundMesenchymal stem or stromal cells are the most widely used cell therapy to date. They are heterogeneous, with variations in growth potential, differentiation capacity and protein expression profile depending on tissue source and production process. Nomenclature and defining characteristics have been debated for almost 20 years, yet the generic term 'MSC' is used to cover a wide range of cellular phenotypes. Against a documented lack of definition of cellular populations used in clinical trials, our study evaluated the extent of characterisation of the cellular population or study drug.MethodsA literature search of clinical trials involving mesenchymal stem/stromal cells was refined to 84 papers upon application of pre-defined inclusion/exclusion criteria. Data were extracted covering background trial information including location, phase, indication, tissue source and details of clinical cell population characterisation (expression of surface markers, viability, differentiation assays and potency/functionality assays). Descriptive statistics were applied, and tests of association between groups were explored using Fisher's exact test for count data with simulated p value.ResultsTwenty-eight studies (33.3%) include no characterisation data. Forty-five (53.6%) reported average values per marker for all cell lots used in the trial, and 11 (13.1%) studies included individual values per cell lot. Viability was reported in 57% of studies. Differentiation was discussed: osteogenesis (29% of papers), adipogenesis (27%), and chondrogenesis (20%) and other functional assays arose in 7 papers (8%). The extent of characterisation was not related to the clinical phase of development. Assessment of functionality was very limited and did not always relate to the likely mechanism of action.ConclusionsThe extent of characterisation was poor and variable. Our findings concur with those in other fields including bone marrow aspirate and platelet-rich plasma therapy. We discuss the potential implications of these findings for the use of mesenchymal stem or stromal cells in regenerative medicine, and the importance of characterisation for transparency and comparability of literature.

Project description:ImportancePublishing study protocols might reduce research waste because of unclear methods or incomplete reporting; on the other hand, there might be few additional benefits of publishing protocols for registered trials that are never completed or published. No study has investigated the proportion of published protocols associated with published results.ObjectiveTo estimate the proportion of published trial protocols for which there are not associated published results.Design, setting, and participantsThis cross-sectional study used stratified random sampling to identify registered clinical trials with protocols published between January 2011 and August 2022 and indexed in PubMed Central. Ongoing studies and those within 1 year of the primary completion date on ClinicalTrials.gov were excluded. Published results were sought from August 2022 to March 2023 by searching ClinicalTrials.gov, emailing authors, and using an automated tool, as well as through incidental discovery.Main outcomes and measuresThe primary outcome was a weighted estimate of the proportion of registered trials with published protocols that also had published main results. The proportion of trials with unpublished results was estimated using a weighted mean.ResultsFrom 1500 citations that were screened, 308 clinical trial protocols were included, and it was found that 87 trials had not published their main results. Most included trials were investigator-initiated evaluations of nonregulated products. When published, results appeared a mean (SD) of 3.4 (2.0) years after protocol publications. With the use of a weighted mean, an estimated 4754 (95% CI, 4296-5226) eligible clinical trial protocols were published and indexed in PubMed Central between 2011 and 2022. In the weighted analysis, 1708 of those protocols (36%; 95% CI, 31%-41%) were not associated with publication of main results. In a sensitivity analysis excluding protocols published after 2019, an estimated 25% (95% CI, 20%-30%) of 3670 (95% CI, 3310-4032) protocol publications were not associated with publication of main results.Conclusions and relevanceThis cross-sectional study of clinical trial protocols published on PubMed Central between 2011 and 2022 suggests that many protocols were not associated with subsequent publication of results. The overall benefits of publishing study protocols might outweigh the research waste caused by unnecessary protocol publications.

Project description:ObjectiveTo investigate to what extent evidence from previous similar trials or systematic reviews was considered before conducting new trials.DesignCohort study of contemporary protocols for trials with ethical approval.MethodsAll protocols for randomised trials approved by the five ethical committees in Denmark between January 2012 and March 2013 were screened for eligibility. Included protocols were read in full to determine whether a systematic search had been conducted and references were checked to evaluate whether trial rationale and design could be challenged for not adequately considering previous evidence. To investigate whether protocols cited relevant trials, we used simple search strategies that could easily be conducted by researchers without experience with literature searches.ResultsSixty-seven protocols were included. Only two (3%) of the protocols explicitly stated to have conducted a literature search and only one (1%) provided information that allowed the search to be replicated. Eleven (16%) of the protocols described trials where we found the information insufficient to judge if the trial was ethically justified, either due to a comparator that was not supported by the presented evidence (six protocols), because they did not present a rationale for conducting the trial (two protocols), or for both reasons (three protocols). For eight (12%) of the protocols, our search identified trials that could have been relevant to cite as justification.ConclusionsWhile most protocols seem to adequately consider existing evidence, a substantial minority of trials might lack a sufficient evidence base. Very few trials seemed to have been based on a literature search which makes it impossible to know whether all relevant previous trials had been considered. Rules for ethical approval should include requirements for systematic literature searches to ensure that research participants are not exposed to sub-optimal treatments or unnecessary harms as well as to reduce research waste.

Project description:BackgroundAs production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured.ObjectivesTo compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs.MethodsJP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented.ResultsWe identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups.ConclusionsThis comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice.Study registrationThe full protocol is available at: https://osf.io/29xvk/.

Dataset Information

Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study.

Objectives

Design

Setting

Main outcome measures

Results

Conclusions

Publications

Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets