Dataset Information

The use of interim data and Data Monitoring Committee recommendations in randomized controlled trial reports: frequency, implications and potential sources of bias.

ABSTRACT:

Background

Interim analysis of accumulating trial data is important to protect participant safety during randomized controlled trials (RCTs). Data Monitoring Committees (DMCs) often undertake such analyses, but their widening role may lead to extended use of interim analysis or recommendations that could potentially bias trial results.

Methods

Systematic search of eight major publications: Annals of Internal Medicine, BMJ, Circulation, CID, JAMA, JCO, Lancet and NEJM, including all randomised controlled trials (RCTs) between June 2000 and May 2005 to identify RCTs that reported use of interim analysis, with or without DMC involvement. Recommendations made by the DMC or based on interim analysis were identified and potential sources of bias assessed. Independent double data extraction was performed on all included trials.

Results

We identified 1772 RCTs, of which 470 (27%; 470/1772) reported the use of a DMC and a further 116 (7%; 116/1772) trials reported some form of interim analysis without explicit mention of a DMC. There were 28 trials (24 with a formal DMC), randomizing a total of 79396 participants, identified as recommending changes to the trial that may have lead to biased results. In most of these, some form of sample size re-estimation was recommended with four trials also reporting changes to trial endpoints. The review relied on information reported in the primary publications and methods papers relating to the trials, higher rates of use may have occurred but not been reported.

Conclusion

The reported use of interim analysis and DMCs in clinical trials has been increasing in recent years. It is reassuring that in most cases recommendations were made in the interest of participant safety. However, in practice, recommendations that may lead to potentially biased trial results are being made.

SUBMITTER: Tharmanathan P

PROVIDER: S-EPMC2279143 | biostudies-literature | 2008 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The use of interim data and Data Monitoring Committee recommendations in randomized controlled trial reports: frequency, implications and potential sources of bias.

Tharmanathan Puvan P Calvert Melanie M Hampton John J Freemantle Nick N

BMC medical research methodology 20080320

<h4>Background</h4>Interim analysis of accumulating trial data is important to protect participant safety during randomized controlled trials (RCTs). Data Monitoring Committees (DMCs) often undertake such analyses, but their widening role may lead to extended use of interim analysis or recommendations that could potentially bias trial results.<h4>Methods</h4>Systematic search of eight major publications: Annals of Internal Medicine, BMJ, Circulation, CID, JAMA, JCO, Lancet and NEJM, including al ...[more]

PMID: 18366697

Similar Datasets

Project description:ObjectiveTo investigate the association between conflicts of interest and favourable recommendations in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews.DesignSystematic review.Eligibility criteriaStudies that compared the association between conflicts of interest and favourable recommendations of drugs or devices (eg, recommending a drug) in clinical guidelines, advisory committee reports, opinion pieces (eg, editorials), or narrative reviews.Data sourcesPubMed, Embase, Cochrane Methodology Register (from inception to February 2020), reference lists, Web of Science, and grey literature.Data extraction and analysisTwo authors independently extracted data and assessed the methodological quality of the studies. Pooled relative risks and 95% confidence intervals were estimated using random effects models (relative risk >1 indicates that documents with conflicts of interest more often had favourable recommendations than documents with no conflicts of interest). Financial and non-financial conflicts of interest were analysed separately, and the four types of documents were analysed separately (preplanned) and combined (post hoc).Results21 studies that analysed 106 clinical guidelines, 1809 advisory committee reports, 340 opinion pieces, and 497 narrative reviews were included. Unpublished data were received for 11 studies (eight full datasets and three summary datasets). 15 studies showed risk of confounding because the compared documents could differ in factors other than conflicts of interest (eg, different drugs used for different populations). The relative risk for associations between financial conflicts of interest and favourable recommendations for clinical guidelines was 1.26 (95% confidence interval 0.93 to 1.69; four studies of 86 clinical guidelines), for advisory committee reports was 1.20 (0.99 to 1.45; four studies of 629 advisory committee reports), for opinion pieces was 2.62 (0.91 to 7.55; four studies of 284 opinion pieces), and for narrative reviews was 1.20 (0.97 to 1.49; four studies of 457 narrative reviews). An analysis of all four types of documents combined supported these findings (1.26, 1.09 to 1.44). In one study that investigated specialty interests, the association between including radiologists as authors of guidelines and recommending routine breast cancer was: relative risk 2.10, 95% confidence interval 0.92 to 4.77; 12 clinical guidelines).ConclusionsWe interpret our findings to indicate that financial conflicts of interest are associated with favourable recommendations of drugs and devices in clinical guidelines, advisory committee reports, opinion pieces, and narrative reviews. Limitations of this review were risk of confounding in the included studies and the statistical imprecision of individual analyses of each document type. It is not certain whether non-financial conflicts of interest influence recommendations.Systematic review registrationCochrane Methodology Review Protocol MR000040.

Project description:BackgroundThere is an abundance of guidance for the interim monitoring of individually randomised trials. While methodological literature exists on how to extend these methods to cluster randomised trials, there is little guidance on practical implementation. Cluster trials have many features which make their monitoring needs different. We outline the methodological and practical challenges of interim monitoring of cluster trials; and apply these considerations to a case study.Case studyThe E-MOTIVE study is an 80-cluster randomised trial of a bundle of interventions to treat postpartum haemorrhage. The proposed data monitoring plan includes (1) monitor sample size assumptions, (2) monitor for evidence of selection bias, and (3) an interim assessment of the primary outcome, as well as monitoring data completeness. The timing of the sample size monitoring is chosen with both consideration of statistical precision and to allow time to recruit more clusters. Monitoring for selection bias involves comparing individual-level characteristics and numbers recruited between study arms to identify any post-randomisation participant identification bias. An interim analysis of outcomes presented with 99.9% confidence intervals using the Haybittle-Peto approach should mitigate any concern regarding the inflation of type-I error. The pragmatic nature of the trial means monitoring for adherence is not relevant, as it is built into a process evaluation.ConclusionsThe interim analyses of cluster trials have a number of important differences to monitoring individually randomised trials. In cluster trials, there will often be a greater need to monitor nuisance parameters, yet there will often be considerable uncertainty in their estimation. This means the utility of sample size re-estimation can be questionable particularly when there are practical or funding difficulties associated with making any changes to planned sample sizes. Perhaps most importantly interim monitoring has the potential to identify selection bias, particularly in trials with post-randomisation identification or recruitment. Finally, the pragmatic nature of cluster trials might mean that the utility of methods to allow for interim monitoring of outcomes based on statistical testing, or monitoring for adherence to study interventions, are less relevant. Our intention is to facilitate the planning of future cluster randomised trials and to promote discussion and debate to improve monitoring of these studies.

Project description:Background/aims Use of data monitoring committees to oversee clinical trials was first proposed nearly 50 years ago. Since then, data monitoring committee use in clinical trials has increased and evolved. Nonetheless, there are no well-defined criteria for determining the need for a data monitoring committee, and considerable variability exists in data monitoring committee composition and conduct. To understand and describe the role and function of data monitoring committees, and establish best practices for data monitoring committee trial oversight, the Clinical Trials Transformation Initiative-a public-private partnership to improve clinical trials-launched a multi-stakeholder project. Methods The data monitoring committee project team included 16 individuals charged with (1) clarifying the purpose of data monitoring committees, (2) identifying best practices for independent data monitoring committee conduct, (3) describing effective communication practices, and (4) developing strategies for training data monitoring committee members. Evidence gathering included a survey, a series of focus group discussions, and a 2-day expert meeting aimed at achieving consensus opinions that form the foundation of our data monitoring committee recommendations. Results We define the role of the data monitoring committee as an advisor to the research sponsor on whether to continue, modify, or terminate a trial based on periodic assessment of trial data. Data monitoring committees should remain independent from the sponsor and be composed of members with no relevant conflicts of interest. Representation on a data monitoring committee generally should include at least one clinician with expertise in the therapeutic area being studied, a biostatistician, and a designated chairperson who has experience with clinical trials and data monitoring. Data monitoring committee meetings are held periodically to evaluate the unmasked data from ongoing trials, but the content and conduct of meetings may vary depending on specific goals or topics for deliberation. To guide data monitoring committee conduct and communication plans, a charter consistent with the protocol's research design and statistical analysis plan should be developed and agreed upon by the sponsor and the data monitoring committee prior to patient enrollment. We recommend concise and flexible charters that explain roles, responsibilities, operational issues, and how data monitoring committee recommendations are generated and communicated. The demand for data monitoring committee members appears to exceed the current pool of qualified individuals. To prepare a new generation of trained data monitoring committee members, we encourage a combination of didactic educational programs, practical experience, and skill development through apprenticeships and mentoring by experienced data monitoring committee members. Conclusion Our recommendations address data monitoring committee use, conduct, communication practices, and member preparation and training. Furthermore recommendations form the foundation for ongoing efforts to improve clinical trial oversight and enhance the safety and integrity of clinical research. These recommendations serve as a call to action for implementation of best practices that benefit study participants, study sponsors, and society.

Project description:BackgroundOutcomes for coronary artery bypass surgery are of broadening interest, but the impact of data type on quality reporting has not been fully examined. We compared the performance of administrative and clinical data-based risk adjustment models at a tertiary-quaternary care hospital.MethodsWe used a prospective study design to test two risk adjustment models, one from administrative (Canadian Institute for Health Information [CIHI] Cardiac Care Quality Indicator) and one from clinical data (Society of Thoracic Surgeons), on cardiac surgical procedures performed between 2013 and 2016 (n = 1635). Our primary outcome was in-hospital mortality within 30 days of surgery. Model performance was established by comparing predicted and observed mortality, model calibration and handling of critical covariates.ResultsObserved mortality was 1.96%, which was the same as that predicted by the Society of Thoracic Surgeons model (1.96%), but significantly higher than that predicted by the CIHI model (1.03%). Despite both models having similar C statistics (0.756 CIHI; 0.758 Society of Thoracic Surgeons), the CIHI model showed significant underestimation of mortality among patients at higher risk. There was significant miscalibration of risk associated with 7 covariates: New York Heart Association class IV, congestive heart failure, ejection fraction less than 20%, atrial fibrillation, acute coronary insufficiency, cardiac compromise (shock, myocardial infarction < 24 h, intra-aortic balloon pump, cardiac resuscitation or preprocedure circulatory support) and creatinine concentration of 100 mg/dL or more. Together, these factors accounted for 84% of the difference in predicted mortality between the administrative and clinical models.InterpretationRisk prediction using administrative data underestimated risk of death, potentially inflating observed-to-predicted mortality ratios at hospitals with patients who are more ill. Caution is warranted when hospital reports of cardiac surgery outcomes are based on administrative data alone.

Dataset Information

The use of interim data and Data Monitoring Committee recommendations in randomized controlled trial reports: frequency, implications and potential sources of bias.

Background

Methods

Results

Conclusion

Publications

The use of interim data and Data Monitoring Committee recommendations in randomized controlled trial reports: frequency, implications and potential sources of bias.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets