Dataset Information

Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors.

ABSTRACT:

Importance

To date, oncologist and model prognostic performance have been assessed independently and mostly retrospectively; however, how model prognostic performance compares with oncologist prognostic performance prospectively remains unknown.

Objective

To compare oncologist performance with a model in predicting 3-month mortality for patients with metastatic solid tumors in an outpatient setting.

Design, setting, and participants

This prognostic study evaluated prospective predictions for a cohort of patients with metastatic solid tumors seen in outpatient oncology clinics at a National Cancer Institute-designated cancer center and associated satellites between December 6, 2019, and August 6, 2021. Oncologists (57 physicians and 17 advanced practice clinicians) answered a 3-month surprise question (3MSQ) within clinical pathways. A model was trained with electronic health record data from January 1, 2013, to April 24, 2019, to identify patients at high risk of 3-month mortality and deployed silently in October 2019. Analysis was limited to oncologist prognostications with a model prediction within the preceding 30 days.

Exposures

Three-month surprise question and gradient-boosting binary classifier.

Main outcomes and measures

The primary outcome was performance comparison between oncologists and the model to predict 3-month mortality. The primary performance metric was the positive predictive value (PPV) at the sensitivity achieved by the medical oncologists with their 3MSQ answers.

Results

A total of 74 oncologists answered 3099 3MSQs for 2041 patients with advanced cancer (median age, 62.6 [range, 18-96] years; 1271 women [62.3%]). In this cohort with a 15% prevalence of 3-month mortality and 30% sensitivity for both oncologists and the model, the PPV of oncologists was 34.8% (95% CI, 30.1%-39.5%) and the PPV of the model was 60.0% (95% CI, 53.6%-66.3%). Area under the receiver operating characteristic curve for the model was 81.2% (95% CI, 79.1%-83.3%). The model significantly outperformed the oncologists in short-term mortality.

Conclusions and relevance

In this prognostic study, the model outperformed oncologists overall and within the breast and gastrointestinal cancer cohorts in predicting 3-month mortality for patients with advanced cancer. These findings suggest that further studies may be useful to examine how model predictions could improve oncologists' prognostic confidence and patient-centered goal-concordant care at the end of life.

SUBMITTER: Zachariah FJ

PROVIDER: S-EPMC9157269 | biostudies-literature | 2022 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors.

Zachariah Finly J FJ Rossi Lorenzo A LA Roberts Laura M LM Bosserman Linda D LD

JAMA network open 20220502 5

<h4>Importance</h4>To date, oncologist and model prognostic performance have been assessed independently and mostly retrospectively; however, how model prognostic performance compares with oncologist prognostic performance prospectively remains unknown.<h4>Objective</h4>To compare oncologist performance with a model in predicting 3-month mortality for patients with metastatic solid tumors in an outpatient setting.<h4>Design, setting, and participants</h4>This prognostic study evaluated prospecti ...[more]

PMID: 35639380

Similar Datasets

Project description:Importance:Machine learning algorithms could identify patients with cancer who are at risk of short-term mortality. However, it is unclear how different machine learning algorithms compare and whether they could prompt clinicians to have timely conversations about treatment and end-of-life preferences. Objectives:To develop, validate, and compare machine learning algorithms that use structured electronic health record data before a clinic visit to predict mortality among patients with cancer. Design, Setting, and Participants:Cohort study of 26 525 adult patients who had outpatient oncology or hematology/oncology encounters at a large academic cancer center and 10 affiliated community practices between February 1, 2016, and July 1, 2016. Patients were not required to receive cancer-directed treatment. Patients were observed for up to 500 days after the encounter. Data analysis took place between October 1, 2018, and September 1, 2019. Exposures:Logistic regression, gradient boosting, and random forest algorithms. Main Outcomes and Measures:Primary outcome was 180-day mortality from the index encounter; secondary outcome was 500-day mortality from the index encounter. Results:Among 26 525 patients in the analysis, 1065 (4.0%) died within 180 days of the index encounter. Among those who died, the mean age was 67.3 (95% CI, 66.5-68.0) years, and 500 (47.0%) were women. Among those who were alive at 180 days, the mean age was 61.3 (95% CI, 61.1-61.5) years, and 15 922 (62.5%) were women. The population was randomly partitioned into training (18 567 [70.0%]) and validation (7958 [30.0%]) cohorts at the patient level, and a randomly selected encounter was included in either the training or validation set. At a prespecified alert rate of 0.02, positive predictive values were higher for the random forest (51.3%) and gradient boosting (49.4%) algorithms compared with the logistic regression algorithm (44.7%). There was no significant difference in discrimination among the random forest (area under the receiver operating characteristic curve [AUC], 0.88; 95% CI, 0.86-0.89), gradient boosting (AUC, 0.87; 95% CI, 0.85-0.89), and logistic regression (AUC, 0.86; 95% CI, 0.84-0.88) models (P for comparison = .02). In the random forest model, observed 180-day mortality was 51.3% (95% CI, 43.6%-58.8%) in the high-risk group vs 3.4% (95% CI, 3.0%-3.8%) in the low-risk group; at 500 days, observed mortality was 64.4% (95% CI, 56.7%-71.4%) in the high-risk group and 7.6% (7.0%-8.2%) in the low-risk group. In a survey of 15 oncology clinicians with a 52.1% response rate, 100 of 171 patients (58.8%) who had been flagged as having high risk by the gradient boosting algorithm were deemed appropriate for a conversation about treatment and end-of-life preferences in the upcoming week. Conclusions and Relevance:In this cohort study, machine learning algorithms based on structured electronic health record data accurately identified patients with cancer at risk of short-term mortality. When the gradient boosting algorithm was applied in real time, clinicians believed that most patients who had been identified as having high risk were appropriate for a timely conversation about treatment and end-of-life preferences.

Project description:BackgroundTP53 is the most commonly mutated gene across all cancer types. R175H mutation was considered structural mutation where the mutation causes misfolding of the protein and leads to a significant conformational alterations within p53's DNA binding domain. The aim of this study was to explain the reason why R175H worse the response to immunotherapy by analyzing tumor immune microenvironment through the expression of immune cells and PD-1.Materials and methodsPatients diagnosed with metastatic carcinoma, including colorectal cancer (CRC), breast cancer (BRCA), gastric cancer (GC), non-small cell lung cancer (NSCLC), and 20 other cancer types, treated in a palliative setting at Samsung Medical Center between October 2019 and April 2021, were enrolled. Of these patients, those who underwent TDS analysis (TruSight™ Oncology 500 assay [TSO 500]) were finally analyzed.ResultsOf 1770 patients, 1012 (57.2%) harbored genetic alterations in TP53. All mutations were single nucleotide variants (SNVs), and the most frequent SNV was R175H (n = 84, 7.5%) which was known as one of the most common hotspot TP53 mutation. The overall survival of patients with TP53 R175H mutations was significantly worse following chemotherapy (606 vs. 456 days, p < 0.001) or immunotherapy (822 vs. 350 days, p < 0.001) compared to those with TP53 mutation in other loci. RNA sequencing indicated that the immune response-related pathways were downregulated in tumors harboring TP53 R175H mutation. Moreover, the expression of CD8(+) T cells PD-1 were lowered in R175H mutation tumors. In the analysis of TP53 structural domain, compared to those having TP53 mutation in other domain, patients with mutations occurring in the nuclear exporter signal (NES) and E4F1-binding domains had significantly worse overall survival following chemotherapy (NES: 606 vs. 451 days, p = 0.043; E4F1: 606 vs. 469 days, p = 0.046) and immunotherapy (NES: 822 vs. 403 days, p < 0.001; E4F1: 822 vs. 413 days, p < 0.001). In addition, tumors with TP53 mutation and co-existing copy number amplification of CCND1, FGF4, and FGF19 in chromosome 11 conferred worse prognosis than those with only TP53 mutation (p < 0.050).DiscussionEach TP53 mutations indicated differential treatment outcomes following chemotherapy or immunotherapy in patients with metastatic cancer. Functional analysis including RNASeq suggested that TP53 mutation downregulated immune response.ConclusionOverall, we found each TP53 mutation to indicate different prognoses in patients with metastatic tumors undergoing chemotherapy and ICI treatment. Further validations, including a prospective cohort study or a functional study, would be particularly valuable in advancing the knowledge on this aspect and developing improved prognostic parameters.

Project description:BackgroundTissue from research biopsies provides access to insights into tumor biology. We aimed to determine medical oncologists' (MOs') attitudes toward research biopsies in patients with metastatic breast cancer (MBC).Materials and methodsA total of 309 breast MOs from National Cancer Institute (NCI)-designated cancer centers were invited to complete a self-administered survey about their attitudes toward approaching patients for research purpose-only biopsies (RPOBs), performed as a standalone procedure, or additional biopsies, performed with a clinically indicated biopsy. The MOs were asked to predict what proportion of their MBC patients would consider undergoing research biopsies.ResultsOf the 309 MOs, 221 (72%) responded. Of these 221 MOs, 30 were ineligible, leaving 191 eligible responders. Nearly all the MOs reported they were comfortable approaching patients regarding research biopsies of blood or skin. One fifth of MOs were uncomfortable approaching patients for RPOBs of the breast. One half of MOs were uncomfortable approaching patients for RPOBs of the liver. A significant variation was found in the perceptions by MOs of their patients' willingness to undergo research biopsies. The factors associated with increased comfort in approaching patients for research biopsies included fewer years in practice, caring for patients who had undergone recent research biopsies, and the predicted willingness of patients to consent to biopsies. The risk of a biopsy and biopsy-related pain were the most common reasons for reluctance to refer patients for research biopsies.ConclusionSignificant variation exists, even at NCI centers, in the comfort level of MOs in approaching MBC patients for research biopsies. MOs' attitudes toward research biopsies might be a modifiable factor in increasing tissue collection for research.Implications for practiceTissue-based research is critical in advancing our understanding of cancer biology, and obtaining tissue from a research biopsy provides an essential resource. This study demonstrates the variability of oncologists' attitudes toward research biopsies and elicits factors associated with increased comfort levels with approaching patients for research biopsies. Biopsy risk and biopsy-related pain were commonly cited reasons not to refer patients for research biopsies. If the risk of a research biopsy is deemed sufficiently low enough to be acceptable, oncologists' attitudes might be a potential target for education and change, which may assist in increasing the availability of tissue for cancer research.

Dataset Information

Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors.

Importance

Objective

Design, setting, and participants

Exposures

Main outcomes and measures

Results

Conclusions and relevance

Publications

Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets