Dataset Information

Expected clinical utility of automatable prediction models for improving palliative and end-of-life care outcomes: Toward routine decision analysis before implementation.

ABSTRACT:

Objective

The study sought to evaluate the expected clinical utility of automatable prediction models for increasing goals-of-care discussions (GOCDs) among hospitalized patients at the end of life (EOL).

Materials and methods

We built a decision model from the perspective of clinicians who aim to increase GOCDs at the EOL using an automated alert system. The alternative strategies were 4 prediction models-3 random forest models and the Modified Hospital One-year Mortality Risk model-to generate alerts for patients at a high risk of 1-year mortality. They were trained on admissions from 2011 to 2016 (70 788 patients) and tested with admissions from 2017-2018 (16 490 patients). GOCDs occurring in usual care were measured with code status orders. We calculated the expected risk difference (beneficial outcomes with alerts minus beneficial outcomes without alerts among those at the EOL), the number needed to benefit (number of alerts needed to increase benefit over usual care by 1 outcome), and the net benefit (benefit minus cost) of each strategy.

Results

Models had a C-statistic between 0.79 and 0.86. A code status order occurred during 2599 of 3773 (69%) hospitalizations at the EOL. At a risk threshold corresponding to an alert prevalence of 10%, the expected risk difference ranged from 5.4% to 10.7% and the number needed to benefit ranged from 5.4 to 10.9 alerts. Using revealed preferences, only 2 models improved net benefit over usual care. A random forest model with diagnostic predictors had the highest expected value, including in sensitivity analyses.

Discussion

Prediction models with acceptable predictive validity differed meaningfully in their ability to improve over usual decision making.

Conclusions

An evaluation of clinical utility, such as by using decision curve analysis, is recommended after validating a prediction model because metrics of model predictiveness, such as the C-statistic, are not informative of clinical value.

SUBMITTER: Taseen R

PROVIDER: S-EPMC8510333 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Introduction: Improving end-of-life (EOL) care is a priority worldwide as this population experiences poor outcomes and accounts disproportionately for costs. In clinical practice, physician judgement is the core method of identifying EOL care needs but has important limitations. Machine learning (ML) is a subset of artificial intelligence advancing capacity to identify patterns and make predictions using large datasets. ML approaches have the potential to improve clinical decision-making and policy design, but there has been no systematic assembly of current evidence. Methods: We conducted a rapid review, searching systematically seven databases from inception to December 31st, 2018: EMBASE, MEDLINE, Cochrane Library, PsycINFO, WOS, SCOPUS and ECONLIT. We included peer-reviewed studies that used ML approaches on routine data to improve palliative and EOL care for adults. Our specified outcomes were survival, quality of life (QoL), place of death, costs, and receipt of high-intensity treatment near end of life. We did not search grey literature and excluded material that was not a peer-reviewed article. Results: The database search identified 426 citations. We discarded 162 duplicates and screened 264 unique title/abstracts, of which 22 were forwarded for full text review. Three papers were included, 18 papers were excluded and one full text was sought but unobtainable. One paper predicted six-month mortality, one paper predicted 12-month mortality and one paper cross-referenced predicted 12-month mortality with healthcare spending. ML-informed models outperformed logistic regression in predicting mortality but poor prognosis is a weak driver of costs. Models using only routine administrative data had limited benefit from ML methods. Conclusion: While ML can in principle help to identify those at risk of adverse outcomes and inappropriate treatment near EOL, applications to policy and practice are formative. Future research must not only expand scope to other outcomes and longer timeframes, but also engage with individual preferences and ethical challenges.

Project description:BackgroundEarly provision of palliative care, at least 3-4 months before death, can improve patient quality of life and reduce burdensome treatments and financial costs. However, there is wide variation in the duration of palliative care received before death reported across the research literature. This study aims to determine the duration of time from initiation of palliative care to death for adults receiving palliative care across the international literature.MethodsWe conducted a systematic review and meta-analysis that was registered with PROSPERO (CRD42018094718). Six databases were searched for articles published between Jan 1, 2013, and Dec 31, 2018: MEDLINE, Embase, CINAHL, Global Health, Web of Science and The Cochrane Library, as well undertaking citation list searches. Following PRISMA guidelines, articles were screened using inclusion (any study design reporting duration from initiation to death in adults palliative care services) and exclusion (paediatric/non-English language studies, trials influencing the timing of palliative care) criteria. Quality appraisal was completed using Hawker's criteria and the main outcome was the duration of palliative care (median/mean days from initiation to death).ResultsOne hundred sixty-nine studies from 23 countries were included, involving 11,996,479 patients. Prior to death, the median duration from initiation of palliative care to death was 18.9 days (IQR 0.1), weighted by the number of participants. Significant differences between duration were found by disease type (15 days for cancer vs 6 days for non-cancer conditions), service type (19 days for specialist palliative care unit, 20 days for community/home care, and 6 days for general hospital ward) and development index of countries (18.91 days for very high development vs 34 days for all other levels of development). Forty-three per cent of studies were rated as 'good' quality. Limitations include a preponderance of data from high-income countries, with unclear implications for low- and middle-income countries.ConclusionsDuration of palliative care is much shorter than the 3-4 months of input by a multidisciplinary team necessary in order for the full benefits of palliative care to be realised. Furthermore, the findings highlight inequity in access across patient, service and country characteristics. We welcome more consistent terminology and methodology in the assessment of duration of palliative care from all countries, alongside increased reporting from less-developed settings, to inform benchmarking, service evaluation and quality improvement.

Project description:Characteristics of the physician influence the essential decision-making in end-of-life care. However, the effect of special education in palliative medicine on different aspects of decision-making in end-of-life care remains unknown. The aim of this study was to explore the decision-making in end-of-life care among physicians with or without special competency in palliative medicine (cPM).A questionnaire including an advanced lung cancer patient-scenario with multiple decision options in end-of-life care situation was sent to 1327 Finnish physicians. Decisions to withdraw or withhold ten life-prolonging interventions were asked on a scale from 1 (definitely would not) to 5 (definitely would) - first, without additional information and then after the family's request for aggressive treatment and the availability of an advance directive. Values from chronological original scenario, family's appeal and advance directive were clustered by trajectory analysis.We received 699 (53%) responses. The mean values of the ten answers in the original scenario were 4.1 in physicians with cPM, 3.4 in general practitioners, 3.4 in surgeons, 3.5 in internists and 3.8 in oncologists (p?<?0.05 for physicians with cPM vs. oncologists and p?<?0.001 for physicians with cPM vs. others). Younger age and not being an oncologist or not having cPM increased aggressive treatment decisions in multivariable logistic regression analysis. The less aggressive approach of physicians with cPM differed between therapies, being most striking concerning intravenous hydration, nasogastric tube and blood transfusions. The aggressive approach increased by the family's request (p?<?0.001) and decreased by an advance directive (p?<?0.001) in all physicians, regardless of special education in palliative medicine.Physicians with special education in palliative medicine make less aggressive decisions in end-of-life care. The impact of specialty on decision-making varies among treatment options. Education in end-of-life care decision-making should be mandatory for young physicians and those in specialty training.

Project description:ObjectivesOperators performing fetal growth scans are usually aware of the gestational age of the pregnancy, which may lead to expected-value bias when performing biometric measurements. We aimed to evaluate the incidence of expected-value bias in routine fetal growth scans and assess its impact on standard biometric measurements.MethodsWe collected prospectively full-length video recordings of routine ultrasound growth scans coupled with operator eye tracking. Expected value was defined as the gestational age at the time of the scan, based on the estimated due date that was established at the dating scan. Expected-value bias was defined as occurring when the operator looked at the measurement box on the screen during the process of caliper adjustment before saving a measurement. We studied the three standard biometric planes on which measurements of head circumference (HC), abdominal circumference (AC) and femur length (FL) are obtained. We evaluated the incidence of expected-value bias and quantified the impact of biased measurements.ResultsWe analyzed 272 third-trimester growth scans, performed by 16 operators, during which a total of 1409 measurements (354 HC, 703 AC and 352 FL; including repeat measurements) were obtained. Expected-value bias occurred in 91.4% of the saved standard biometric plane measurements (85.0% for HC, 92.9% for AC and 94.9% for FL). The operators were more likely to adjust the measurements towards the expected value than away from it (47.7% vs 19.7% of measurements; P?<?0.001). On average, measurements were corrected by 2.3?±?5.6, 2.4?±?10.4 and 3.2?±?10.4?days of gestation towards the expected gestational age for the HC, AC, and FL measurements, respectively. Additionally, we noted a statistically significant reduction in measurement variance once the operator was biased (P?=?0.026). Comparing the lowest and highest possible estimated fetal weight (using the smallest and largest biased HC, AC and FL measurements), we noted that the discordance, in percentage terms, was 10.1%?±?6.5%, and that in 17% (95%?CI, 12-21%) of the scans, the fetus could be considered as small-for-gestational age or appropriate-for-gestational age if using the smallest or largest possible measurements, respectively. Similarly, in 13% (95%?CI, 9-16%) of scans, the fetus could be considered as large-for-gestational age or appropriate-for-gestational age if using the largest or smallest possible measurements, respectively.ConclusionsDuring routine third-trimester growth scans, expected-value bias frequently occurs and significantly changes standard biometric measurements obtained. © 2019 the Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

Project description:ContextMagnetic resonance imaging (MRI) is currently the most accurate imaging modality to assess local prostate cancer stage. Despite a growing body of evidence, incorporation of MRI images into decision-making process concerning surgical template of radical prostatectomy, is complex and still poorly understood.ObjectiveWe sought to determine the value of MRI in preoperative planning before radical prostatectomy.Materials and methodsSystematic search through electronic PubMed, EMBASE, and Cochrane databases from 2000 up to April 2018 was performed. Only studies that used preoperative MRI in decision-making process regarding extension of resection in patients with prostate cancer, in whom radical prostatectomy was an initial form of treatment were included into analysis. Their quality was scored by Risk Of Bias In Non-Randomized Studies of Interventions system. Meta-analysis was performed to calculate the weighted summary proportion under the fixed or random effects model as appropriate and pooled effects were depicted on forest plots.ResultsThe results showed that the preoperative MRI led to the modification of initial surgical template in one third of cases (35%). This occurred increasingly with the rising prostate cancer-risk category: 28%, 33%, 52% in low-, intermediate- and high-risk group, respectively. Modification of neurovascular bundle-sparing surgery based on MRI appeared to have no impact on the positive surgical margin rate. The decision based on MRI was correct on average in 77% of cases and differed across prostate cancer-risk categories: 63%, 75% and 91% in low-, intermediate- and high-risk group, accordingly.ConclusionsIn summary, MRI has a considerable impact on the decision-making process regarding the extent of resection during radical prostatectomy. Adaptation of MRI images by operating surgeons has at worst no significant impact on surgical margin status, however its ability to decrease the positive surgical margin rates remains unconfirmed.