Dataset Information

Can Oncologists Predict the Efficacy of Treatments in Randomized Trials?

ABSTRACT:

Background

Decisions about trial funding, ethical approval, or clinical practice guideline recommendations require expert judgments about the potential efficacy of new treatments. We tested whether individual and aggregated expert opinion of oncologists could predict reliably the efficacy of cancer treatments tested in randomized controlled trials.

Materials and methods

An international sample of 137 oncologists specializing in genitourinary, lung, and colorectal cancer provided forecasts on primary outcome attainment for five active randomized cancer trials within their subspecialty; skill was assessed using Brier scores (BS), which measure the average squared deviation between forecasts and outcomes.

Results

A total of 40% of trials in our sample reported positive primary outcomes. Experts generally anticipated this overall frequency (mean forecast, 34%). Individual experts on average outperformed random predictions (mean BS = 0.29 [95% confidence interval (CI), 0.28-0.33] vs. 0.33) but underperformed prediction algorithms that always guessed 50% (BS = 0.25) or that were trained on base rates (BS = 0.19). Aggregating forecasts improved accuracy (BS = 0.25; 95% CI, 0.16-0.36]). Neither individual experts nor aggregated predictions showed appreciable discrimination between positive and nonpositive trials (area under the curve of a receiver operating characteristic curve, 0.52 and 0.43, respectively).

Conclusion

These findings are based on a limited sample of trials. However, they reinforce the importance of basing research and policy decisions on the results of randomized trials rather than expert opinion or low-level evidence.

Implications for practice

Predictions of oncologists, either individually or in the aggregate, did not anticipate reliably outcomes for randomized trials in cancer. These findings suggest that pooled expert opinion about treatment efficacy is no substitute for randomized trials. They also underscore the challenges of using expert opinion to prioritize interventions for clinical trials or to make recommendations in clinical practice guidelines.

SUBMITTER: Benjamin DM

PROVIDER: S-EPMC7794184 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Can Oncologists Predict the Efficacy of Treatments in Randomized Trials?

Benjamin Daniel M DM Mandel David R DR Barnes Tristan T Krzyzanowska Monika K MK Leighl Natasha N Tannock Ian F IF Kimmelman Jonathan J

The oncologist 20200826 1

<h4>Background</h4>Decisions about trial funding, ethical approval, or clinical practice guideline recommendations require expert judgments about the potential efficacy of new treatments. We tested whether individual and aggregated expert opinion of oncologists could predict reliably the efficacy of cancer treatments tested in randomized controlled trials.<h4>Materials and methods</h4>An international sample of 137 oncologists specializing in genitourinary, lung, and colorectal cancer provided f ...[more]

PMID: 32936509

Similar Datasets

Project description:Background: We provided a comprehensive evaluation of efficacy of available treatments for coronavirus disease 2019 (COVID-19). Methods: We searched for candidate COVID-19 studies in WHO COVID-19 Global Research Database up to August 19, 2021. Randomized controlled trials for suspected or confirmed COVID-19 patients published on peer-reviewed journals were included, regardless of demographic characteristics. Outcome measures included mortality, mechanical ventilation, hospital discharge and viral clearance. Bayesian network meta-analysis with fixed effects was conducted to estimate the effect sizes using posterior means and 95% equal-tailed credible intervals (CrIs). Odds ratio (OR) was used as the summary measure for treatment effect. Bayesian hierarchical models were used to estimate effect sizes of treatments grouped by the treatment classifications. Results: We identified 222 eligible studies with a total of 102,950 patients. Compared with the standard of care, imatinib, intravenous immunoglobulin and tocilizumab led to lower risk of death; baricitinib plus remdesivir, colchicine, dexamethasone, recombinant human granulocyte colony stimulating factor and tocilizumab indicated lower occurrence of mechanical ventilation; tofacitinib, sarilumab, remdesivir, tocilizumab and baricitinib plus remdesivir increased the hospital discharge rate; convalescent plasma, ivermectin, ivermectin plus doxycycline, hydroxychloroquine, nitazoxanide and proxalutamide resulted in better viral clearance. From the treatment class level, we found that the use of antineoplastic agents was associated with fewer mortality cases, immunostimulants could reduce the risk of mechanical ventilation and immunosuppressants led to higher discharge rates. Conclusions: This network meta-analysis identified superiority of several COVID-19 treatments over the standard of care in terms of mortality, mechanical ventilation, hospital discharge and viral clearance. Tocilizumab showed its superiority compared with SOC on preventing severe outcomes such as death and mechanical ventilation as well as increasing the discharge rate, which might be an appropriate treatment for patients with severe or mild/moderate illness. We also found the clinical efficacy of antineoplastic agents, immunostimulants and immunosuppressants with respect to the endpoints of mortality, mechanical ventilation and discharge, which provides valuable information for the discovery of potential COVID-19 treatments.

Project description:Background: Melasma is an acquired pigmentation disorder with challenges in treatment because of its refractory nature and high risk of recurrence. Objectives: This study aimed to compare the efficacy and side effects of 14 common therapies for melasma using a systematic review and network meta-analysis (NMA). Methods: The PubMed, Embase, and Cochrane Library databases were searched till December 2020 using the melasma area and severity index as a therapeutic index. A total of 59 randomized controlled trials (RCTs) met the inclusion criteria and were selected. Results: The ranking of relative efficacy compared with placebo in descending order was Q-switched Nd:Yag 1,064-nm laser (QSND), intense pulsed light, ablative fractional laser (AFL), triple combined cream (TCC), topical vitamin C, oral tranexamic acid (oTA), peeling, azelaic acid, microneedles (MNs), topical tranexamic acid (tTA), tretinoin, picosecond laser, hydroquinone (HQ), and non-AFL. Moreover, QSND was more effective than HQ and tTA against melasma. The ranking of percentage (%) of side effects in ascending order for each of 14 therapies with more than 80 participants was tretinoin (10.1%), oTA (17.6%), HQ (18.2%), AFL (20.0%), QSND (21.5%), TCC (25.7%), tTA (36.75%), peeling (38.0%), and MN (52.3%). Taking both efficacy and safety into consideration, TCC was found to be the most favorable selection among the topical drugs for melasma. QSND and AFL were still the best ways to treat melasma among photoelectric devices. oTA as system administration was a promising way recommended for melasma. Among 31 studies, 87% (27/31) studies showed that the efficacy of combination therapies is superior to that of single therapy. The quality of evidence in this study was generally high because of nearly 50% of split-face RCTs. Conclusions: Based on the published studies, this NMA indicated that QSND, AFL, TCC, and oTA would be the preferred ways to treat melasma for dermatologists. However, more attention should be paid to the efficacy and safety simultaneously during the clinical application. Most of the results were in line with those of the previous studies, but a large number of RCTs should be included for validation or update. Systematic Review Registration: identifier: CRD42021239203.

Project description:BackgroundSystemic treatments, namely, either monotherapy or combination therapy, are commonly administered to patients with advanced or metastatic colorectal cancer (CRC). This study aimed to provide the complete efficacy and safety profiles and ranking of systemic therapies for the treatment of unresectable advanced or metastatic CRC.MethodsWe searched PubMed, Embase, the Cochrane Library, and ClinicalTrials.gov from inception until June 30, 2021, and also the bibliographies of relevant studies. Randomized controlled trials comparing two or more treatments, namely, at least capecitabine, 5-fluorouracil, leucovorin, irinotecan, bevacizumab, cetuximab, oxaliplatin, or panitumumab were investigated. A network meta-analysis using the Bayesian approach was performed to compare the efficacy and safety of treatments. The surface under the cumulative ranking curve (SUCRA) was calculated for the probability of each treatment as the most effective. The overall response rate (ORR), disease control rate (DCR), overall survival (OS), progression-free survival (PFS), adverse events (AEs) grade ≥3, and serious adverse events (SAEs) were evaluated.ResultsOne hundred two publications with 36,147 participants were assigned to 39 different treatments. Among 11 treatments with full information on six outcomes, FOLFIRI/FOLFOX/FOLFOXIRI + bevacizumab significantly improved both the ORR and DCR, compared to FOLFIRI. Although FOLFOX and FOLFIRI/FOLFOX + cetuximab significantly prolonged both OS and PFS, treatments were comparable in terms of AEs grade ≥3 and SAEs. The top highest SUCRA values were observed in the FOLFOXIRI + panitumumab group for ORR (96%) and DCR (99%), FOLFIRI + bevacizumab + panitumumab group for OS (62%) and PFS (54%), and FOLFOXIRI + bevacizumab group for AEs grade ≥3 (59%) and SAEs (59%) outcomes.ConclusionsThese findings suggest an available range of systemic treatment therapies with different efficacy and safety profiles with patients. Further investigations of the side effects and mutation status are required to confirm our findings.Systematic review registrationhttps://www.crd.york.ac.uk/prospero/, identifier CRD42019127772.

Project description:BackgroundEndpoint choice for randomized controlled trials of treatments for novel coronavirus-induced disease (COVID-19) is complex. Trials must start rapidly to identify treatments that can be used as part of the outbreak response, in the midst of considerable uncertainty and limited information. COVID-19 presentation is heterogeneous, ranging from mild disease that improves within days to critical disease that can last weeks to over a month and can end in death. While improvement in mortality would provide unquestionable evidence about the clinical significance of a treatment, sample sizes for a study evaluating mortality are large and may be impractical, particularly given a multitude of putative therapies to evaluate. Furthermore, patient states in between "cure" and "death" represent meaningful distinctions. Clinical severity scores have been proposed as an alternative. However, the appropriate summary measure for severity scores has been the subject of debate, particularly given the variable time course of COVID-19. Outcomes measured at fixed time points, such as a comparison of severity scores between treatment and control at day 14, may risk missing the time of clinical benefit. An endpoint such as time to improvement (or recovery) avoids the timing problem. However, some have argued that power losses will result from reducing the ordinal scale to a binary state of "recovered" versus "not recovered."MethodsWe evaluate statistical power for possible trial endpoints for COVID-19 treatment trials using simulation models and data from two recent COVID-19 treatment trials.ResultsPower for fixed time-point methods depends heavily on the time selected for evaluation. Time-to-event approaches have reasonable statistical power, even when compared with a fixed time-point method evaluated at the optimal time.DiscussionTime-to-event analysis methods have advantages in the COVID-19 setting, unless the optimal time for evaluating treatment effect is known in advance. Even when the optimal time is known, a time-to-event approach may increase power for interim analyses.

Dataset Information

Can Oncologists Predict the Efficacy of Treatments in Randomized Trials?

Background

Materials and methods

Results

Conclusion

Implications for practice

Publications

Can Oncologists Predict the Efficacy of Treatments in Randomized Trials?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets