Interpretation of CIs in clinical trials with non-significant results: systematic review and recommendations.
Ontology highlight
ABSTRACT: Objectives
Interpretation of CIs in randomised clinical trials (RCTs) with treatment effects that are not statistically significant can distinguish between results that are 'negative' (the data are not consistent with a clinically meaningful treatment effect) or 'inconclusive' (the data remain consistent with the possibility of a clinically meaningful treatment effect). This interpretation is important to ensure that potentially beneficial treatments are not prematurely abandoned in future research or clinical practice based on invalid conclusions.Design
Systematic review of RCT reports published in 2014 in Annals of Internal Medicine, New England Journal of Medicine, JAMA, JAMA Internal Medicine and The Lancet (n=247).Results
85 of 99 articles with statistically non-significant results reported CIs for the treatment effect. Only 17 of those 99 articles interpreted the CI. Of the 22 articles in which CIs indicated an inconclusive result, only four acknowledged that the study could not rule out a clinically meaningful treatment effect.Conclusions
Interpretation of CIs is important but occurs infrequently in study reports of trials with treatment effects that are not statistically significant. Increased author interpretation of CIs could improve application of RCT results. Reporting recommendations are provided.
SUBMITTER: Gewandter JS
PROVIDER: S-EPMC5726092 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Non-inferiority (NI) randomized clinical trials (RCTs) commonly evaluate efficacy of new antiretroviral (ARV) drugs in human immunodeficiency virus (HIV) patients. Their reporting and interpretation have not been systematically evaluated. We evaluated the reporting of NI RCTs in HIV patients according to the CONSORT statement and assessed the degree of misinterpretation of RCTs when NI was inconclusive or not established.Systematic review.PubMed, Web of Science, and Scopus were reviewed until December 2011. Selection and extraction was performed independently by three reviewers.Of the 42 RCTs (n = 21,919; range 41-3,316) selected, 23 were in ARV-naïve and 19 in ARV-experienced patients. Twenty-seven (64%) RCTs provided information about prior RCTs of the active comparator, and 37 (88%) used 2-sided CIs. Two thirds of trials used a NI margin between 10 and 12%, although only 12 explained the method to determine it. Blinding was used in 9 studies only. The main conclusion was based on both intention-to-treat (ITT) and per protocol (PP) analyses in 5 trials, on PP analysis only in 4 studies, and on ITT only in 31 studies. Eleven of 16 studies with NI inconclusive or not established highlighted NI or equivalence, and distracted readers with positive secondary results.There is poor reporting and interpretation of NI RCTs performed in HIV patients. Maximizing the reporting of the method of NI margin determination, use of blinding and both ITT and PP analyses, and interpreting negative NI according to actual primary findings will improve the understanding of results and their translation into clinical practice.
| S-EPMC3643946 | biostudies-literature
Project description:ObjectiveTo study how composite outcomes, which have combined several components into a single measure, are defined, reported, and interpreted.DesignSystematic review of parallel group randomised clinical trials published in 2008 reporting a binary composite outcome. Two independent observers extracted the data using a standardised data sheet, and two other observers, blinded to the results, selected the most important component.ResultsOf 40 included trials, 29 (73%) were about cardiovascular topics and 24 (60%) were entirely or partly industry funded. Composite outcomes had a median of three components (range 2-9). Death or cardiovascular death was the most important component in 33 trials (83%). Only one trial provided a good rationale for the choice of components. We judged that the components were not of similar importance in 28 trials (70%); in 20 of these, death was combined with hospital admission. Other major problems were change in the definition of the composite outcome between the abstract, methods, and results sections (13 trials); missing, ambiguous, or uninterpretable data (9 trials); and post hoc construction of composite outcomes (4 trials). Only 24 trials (60%) provided reliable estimates for both the composite and its components, and only six trials (15%) had components of similar, or possibly similar, clinical importance and provided reliable estimates. In 11 of 16 trials with a statistically significant composite, the abstract conclusion falsely implied that the effect applied also to the most important component.ConclusionsThe use of composite outcomes in trials is problematic. Components are often unreasonably combined, inconsistently defined, and inadequately reported. These problems will leave many readers confused, often with an exaggerated perception of how well interventions work.
| S-EPMC2923692 | biostudies-literature
Project description:ImportanceSpin, the inaccurate reporting of randomized clinical trials (RCTs) with results that are not statistically significant for the primary end point, distorts interpretation of results and leads to misinterpretation. However, the prevalence of spin and related factors in noninferiority cancer RCTs remains unclear.ObjectiveTo examine misleading reporting, or spin, and the associated factors in noninferiority cancer RCTs through a systematic review.Data sourcesA systematic search of the PubMed database was performed for articles published between January 1, 2010, and December 31, 2019, using the Cochrane Highly Sensitive Search Strategy.Study selectionTwo investigators independently selected studies using the inclusion criteria of noninferiority parallel-group RCTs aiming to confirm effects to cancer treatments published between January 1, 2010, and December 31, 2019, reporting results that were not statistically significant for the primary end points.Data extraction and synthesisStandardized data abstraction was used to extract information concerning the trial characteristics and spin based on a prespecified definition. The main investigator extracted the trial characteristics while both readers independently evaluated the spin. The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline was followed.Main outcomes and measuresThe main outcome was spin prevalence in any section of the report. Spin was defined as use of specific reporting strategies, from whatever motive, to highlight that the experimental treatment is beneficial, despite no statistically significant difference for the primary outcome, or to distract the reader from results that are not statistically significant. The associations (prevalence difference and odds ratios [ORs]) between spin and trial characteristics were also evaluated.ResultsThe analysis included 52 of 2752 reports identified in the PubMed search. Spin was identified in 39 reports (75.0%; 95% CI, 61.6%-84.9%), including the abstract (34 reports [65.4%; 95% CI, 51.1%-76.9%]) and the main text (38 reports [73.1%; 95% CI, 59.7%-83.3%]). Univariate analysis found that the spin prevalence was higher in reports with data managers (prevalence difference, 27%; 95% CI, 1.1%-50.3%), reports without funding from for-profit sources (prevalence difference, 31.2%; 95% CI, 4.8%-53.8%), and reports of novel experimental treatments (prevalence difference, 37.5%; 95% CI, 5.8%-64.7%). Multivariable analysis found that novel experimental treatment (OR, 4.64; 95% CI, 0.98-22.02) and funding only from nonprofit sources only (OR, 5.20; 95% CI, 1.21-22.29) were associated with spin.Conclusions and relevanceIn this systematic review, most noninferiority RCTs reporting results that were not statistically significant for the primary end points showed distorted interpretation and inaccurate reporting. The novelty of an experimental treatment and funding only from nonprofit sources were associated with spin.
| S-EPMC8652604 | biostudies-literature
Project description:Spin in the reporting of randomized controlled trials, where authors report research in a way that potentially misrepresents results and mislead readers, has been demonstrated in the broader medical literature. We investigated spin in wound care trials with (a) no statistically significant result for the primary outcome and (b) no clearly specified primary outcome.We searched the Cochrane Wounds Group Specialised Register of Trials for randomized controlled trials (RCTs). Eligible studies were: Parallel-group RCTs of interventions for foot, leg or pressure ulcers published in 2004 to 2009 (inclusive) with either a clearly identified primary outcome for which there was a statistically non-significant result (Cohort A) or studies that had no clear primary outcome (Cohort B).We extracted general study details. For both Cohorts A and B we then assessed for the presence of spin. For Cohort A we used a pre-defined process to assess reports for spin. For Cohort B we aimed to assess spin by recording the number of positive treatment effect claims made. We also compared the number of statistically significant and non-significant results reported in the main text and the abstract looking specifically for spin in the form of selective outcome reporting.Of the 71 eligible studies, 28 were eligible for Cohort A; of these, 71% (20/28) contained spin. Cohort B contained 43 studies; of these, 86% (37/43) had abstracts that claimed a favorable treatment claim. Whilst 74% (32/43) of main text results in Cohort B included at least one statistically non-significant result, this was not reflected in the abstract where only 28% contained (12/43) at least one statistically non-significant result.Spin is a frequent phenomenon in reports of RCTs of wound treatments. Studies without statistically significant results for the primary outcome used spin in 71% of cases. Furthermore, 33% (43/132) of reports of wound RCTs did not specify a primary outcome and there was evidence of spin and selective outcome reporting in the abstracts of these. Readers should be wary of only reading the abstracts of reports of RCTs of wound treatments since they are frequently misleading regarding treatment effects.
| S-EPMC3832286 | biostudies-other
Project description:Invasive fungal infections cause excessive morbidity and mortality in premature neonates and severely ill infants.Safety and efficacy outcomes of micafungin were compared between prematurely and non-prematurely born infants <2 years of age. Data were obtained from all completed phase I-III clinical trials with micafungin that had enrolled infants (<2 years of age) that were listed in the Astellas Clinical Study Database. Demographics, adverse events, hepatic function tests and treatment success data were extracted and validated by the Astellas biostatistical group for all micafungin-treated patients, <2 years of age, using the unique patient identifier.One-hundred and sixteen patients included in 9 clinical trials, 48% premature [birth weight (BW) <2500 g and/or gestational age <37 weeks], 52% non-premature, received ? 1 dose of micafungin. Among premature patients, 14.5% were low BW (1500-2499 g), 36.4% very low BW (1000-1499 g) and 49.1% extremely low BW (<1000 g). Ninety patients (78%) completed the studies; 13 [11% (4 premature)] died. Significantly more non-premature than premature patients discontinued treatment (P = 0.003). Treatment-related adverse events were recorded in 23% of patients with no difference between groups. More extremely low BW (n = 4, 15%) and very low BW (n = 8, 40%) infants experienced treatment-related adverse events than low BW (n = 0) and there was no relation to micafungin dose or duration. For a subgroup of 30 patients with invasive candidiasis, treatment success was achieved in 73% in both premature and non-premature groups. Prophylaxis was successful in 4/5 non-premature hematopoietic stem cell transplant patients.Micafungin has a safe profile in premature and non-premature infants with substantial efficacy.
| S-EPMC4196786 | biostudies-other
Project description:The SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013 Statement was developed to provide guidance for inclusion of key methodological components in clinical trial protocols. However, these standards do not include guidance specific to pathology input in clinical trials. This systematic review aims to synthesise existing recommendations specific to pathology practice in clinical trials for implementation in trial protocol design. Articles were identified from database searches and deemed eligible for inclusion if they contained: (1) guidance and/or a checklist, which was (2) pathology-related, with (3) content relevant to clinical trial protocols or could influence a clinical trial protocol design from a pathology perspective and (4) were published in 1996 or later. The quality of individual papers was assessed using the AGREE-GRS (Appraisal of Guidelines for REsearch & Evaluation - Global Rating Scale) tool, and the confidence in cumulative evidence was evaluated using the GRADE-CERQual (Grading of Recommendations Assessment, Development and Evaluation-Confidence in Evidence from Reviews of Qualitative research) approach. Extracted recommendations were synthesised using the best fit framework method, which includes thematic analysis followed by a meta-aggregative approach to synthesis within the framework. Of the 10 184 records screened and 199 full-text articles reviewed, only 40 guidance resources met the eligibility criteria for inclusion. Recommendations extracted from 22 guidance documents were generalisable enough for data synthesis. Seven recommendation statements were synthesised as follows: (1) multidisciplinary collaboration in trial design with early involvement of pathologists, particularly with respect to the use of biospecimens and associated biomarker/analytical assays and in the evaluation of pathology-related parameters; (2) funding and training for personnel undertaking trial work; (3) selection of an accredited laboratory with suitable facilities to undertake scheduled work; (4) quality assurance of pathology-related parameters; (5) transparent reporting of pathology-related parameters; (6) policies regarding informatics and tracking biospecimens across trial sites; and (7) informed consent for specimen collection and retention for future research.
| S-EPMC8073003 | biostudies-literature
Project description:BackgroundIt is unclear how to disseminate the results of randomised controlled trials effectively to health professionals and policymakers to improve treatment, care or prevention through changing policy and practice. This systematic review examined the effectiveness of different methods of dissemination of clinical research results to professional audiences.MethodsWe systematically reviewed the published and grey literature from 2000 to 2022 for studies assessing different approaches for disseminating clinical study results to professional audiences (health professionals, policymakers and guideline developers). Two reviewers assessed potentially relevant full texts for inclusion. We grouped studies by intervention type, synthesising findings using effect direction plots. Outcomes were grouped into out-takes (e.g. awareness, knowledge, understanding), outcomes (e.g. attitude changes) and impact (changes in policy/practice). The quality of evidence was assessed using GRADE.ResultsOur search identified 13,264 unique records, of which 416 full texts were assessed for eligibility. Of 60 studies that were identified as eligible for inclusion, 20 evaluated the effectiveness of interventions to disseminate clinical research results (13 RCTs, 2 observational studies, 3 pre- and post-intervention surveys and 2 cross-sectional surveys). Studies were grouped by intervention: 7 studies that involved face-to-face meetings between the target audience and trained educators were classified as 'outreach interventions'; 5 studies that provided a summary format for systematic review findings (e.g. summary of findings tables) were grouped together. There was high certainty evidence of a small beneficial impact of outreach interventions on health and moderate certainty evidence of impact on practice (mostly prescribing). There was no evidence of impact on policy and very low certainty around benefits on outcomes and out-takes. We found no consistent benefits of summary formats for systematic review results on outcomes or out-takes (moderate quality evidence). Other interventions with less evidence are reported in the Additional Materials.ConclusionsOutreach interventions to disseminate clinical research results can lead to changes in practice and improvements in health. However, these interventions can be resource-intensive. Investment is vital to identify and implement effective and cost-effective ways to disseminate results, so that the potential benefits of trials to patients can be realised.Trial registrationInternational Prospective Register of Systematic Reviews (PROSPERO), CRD42019137364.
| S-EPMC10835915 | biostudies-literature
Project description:As the nontherapeutic use of prescription medications escalates, serious associated consequences have also increased. This makes it essential to estimate misuse, abuse, and related events (MAREs) in the development and postmarketing adverse event surveillance and monitoring of prescription drugs accurately. However, classifications and definitions to describe prescription drug MAREs differ depending on the purpose of the classification system, may apply to single events or ongoing patterns of inappropriate use, and are not standardized or systematically employed, thereby complicating the ability to assess MARE occurrence adequately. In a systematic review of existing prescription drug MARE terminology and definitions from consensus efforts, review articles, and major institutions and agencies, MARE terms were often defined inconsistently or idiosyncratically, or had definitions that overlapped with other MARE terms. The Analgesic, Anesthetic, and Addiction Clinical Trials, Translations, Innovations, Opportunities, and Networks (ACTTION) public-private partnership convened an expert panel to develop mutually exclusive and exhaustive consensus classifications and definitions of MAREs occurring in clinical trials of analgesic medications to increase accuracy and consistency in characterizing their occurrence and prevalence in clinical trials. The proposed ACTTION classifications and definitions are designed as a first step in a system to adjudicate MAREs that occur in analgesic clinical trials and postmarketing adverse event surveillance and monitoring, which can be used in conjunction with other methods of assessing a treatment's abuse potential.
| S-EPMC5460151 | biostudies-literature
Project description:PurposePatient-Reported Outcomes (PROs) and its measures (PROMs) are key to outcome assessment in Fibromyalgia (FM) trials. The aim of this review was to investigate which domains and instruments were assessed in recent FM trials and to compare them to recommendations by the Outcome Measures in Rheumatology (OMERACT) initiative. In addition, we investigated the overlap with a generic health assessment approach, i.e. eight domains suggested by the Patient-Reported Outcome Measurement Information System® (PROMIS®).MethodsIn compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, a systematic literature search in scientific databases including PubMed, PsycInfo, and Embase was conducted to identify studies that assessed at least two dimensions of health-related quality of life (HRQoL) from 2015 to June 2022. Non-randomized and randomized controlled trials were included in the analysis. We extracted PROs and PROMs used in each study.ResultsFrom 1845 identified records, 107 records out of 105 studies met the inclusion criteria. Studies investigated 50 PROs using 126 different PROMs. Most frequently assessed domains were pain, depression, fatigue, and anxiety (> 95% of the studies). The disease-specific FIQ was the most frequently applied PROM (82%). Overall, only 9% of the studies covered all domains deemed mandatory by OMERACT. Very few studies covered all eight generic health domains suggested by PROMIS.ConclusionThe majority of trials covered most OMERACT domains or generic PROMIS health domains. There was, however, great variability in the instruments used to assess the domains, which points at a limited degree of standardization in the field.
| S-EPMC10172242 | biostudies-literature
Project description:Recent data show an interaction between COVID-19 and nicotine and indicate the need for an assessment of transdermal nicotine use in non-smokers. Assessments have been conducted into the short-term cognitive effects of nicotine and into diseases such as Parkinson's, Tourette syndrome, ADHD or ulcerative colitis.MethodsAnalyses of nicotine administration protocols and safety were conducted after reviewing Medline and Science Direct databases performing a search using the words [transdermal nicotine] AND [non-smoker] AND selected diseases.ResultsAmong 298 articles identified, there were 35 reviewed publications reporting on 33 studies of non-smokers receiving transdermal nicotine for >48hours. In the 16 randomized trials, 7 crossover, 1 case/control and 9 open studies patients received an initial nicotine dose of between 2.5mg and 15mg/day. In 22 studies, daily doses increased by 2 to 7 steps in 3 to 96 days until the dose was between 5mg and 105mg/day. The target nicotine dose was 19.06±20.89mg/day. The 987 non-smokers (534 never-smokers, 326 ex-smokers and 127 classified as "non-smokers") received or did not receive nicotine. The most common side-effects were nausea and skin itching. Forty-three (7.1%) non-smokers stopped treatment because of an adverse event of nicotine. No hospitalization related to nicotine side-effects were reported.ConclusionDespite a relatively safe tolerance profile, transdermal nicotine therapy in non-smokers can only be used in clinical trials. There is a lack of formal assessment of the potential risk of developing a tobacco addiction. This review offers baseline data to set a transdermal nicotine protocol for non-smokers with a new purpose.
| S-EPMC8183099 | biostudies-literature