Is Artificial Intelligence Better Than Human Clinicians in Predicting Patient Outcomes?
Ontology highlight
ABSTRACT:
SUBMITTER: Lee J
PROVIDER: S-EPMC7481865 | biostudies-literature | 2020 Aug
REPOSITORIES: biostudies-literature
Publications
Is Artificial Intelligence Better Than Human Clinicians in Predicting Patient Outcomes?
Journal of medical Internet research 20200826 8
In contrast with medical imaging diagnostics powered by artificial intelligence (AI), in which deep learning has led to breakthroughs in recent years, patient outcome prediction poses an inherently challenging problem because it focuses on events that have not yet occurred. Interestingly, the performance of machine learning-based patient outcome prediction models has rarely been compared with that of human clinicians in the literature. Human intuition and insight may be sources of underused pred ...[more]
Similar Datasets
Project description:BackgroundArtificial intelligence (AI) has been extensively used in a range of medical fields to promote therapeutic development. The development of diverse AI techniques has also contributed to early detections, disease diagnoses, and referral management. However, concerns about the value of advanced AI in disease diagnosis have been raised by health care professionals, medical service providers, and health policy decision makers.ObjectiveThis review aimed to systematically examine the literature, in particular, focusing on the performance comparison between advanced AI and human clinicians to provide an up-to-date summary regarding the extent of the application of AI to disease diagnoses. By doing so, this review discussed the relationship between the current advanced AI development and clinicians with respect to disease diagnosis and thus therapeutic development in the long run.MethodsWe systematically searched articles published between January 2000 and March 2019 following the Preferred Reporting Items for Systematic reviews and Meta-Analysis in the following databases: Scopus, PubMed, CINAHL, Web of Science, and the Cochrane Library. According to the preset inclusion and exclusion criteria, only articles comparing the medical performance between advanced AI and human experts were considered.ResultsA total of 9 articles were identified. A convolutional neural network was the commonly applied advanced AI technology. Owing to the variation in medical fields, there is a distinction between individual studies in terms of classification, labeling, training process, dataset size, and algorithm validation of AI. Performance indices reported in articles included diagnostic accuracy, weighted errors, false-positive rate, sensitivity, specificity, and the area under the receiver operating characteristic curve. The results showed that the performance of AI was at par with that of clinicians and exceeded that of clinicians with less experience.ConclusionsCurrent AI development has a diagnostic performance that is comparable with medical experts, especially in image recognition-related fields. Further studies can be extended to other types of medical imaging such as magnetic resonance imaging and other medical practices unrelated to images. With the continued development of AI-assisted technologies, the clinical implications underpinned by clinicians' experience and guided by patient-centered health care principle should be constantly considered in future AI-related and other technology-based medical research.
| S-EPMC6716335 | biostudies-literature
Project description:Hospital systems, payers, and regulators have focused on reducing length of stay (LOS) and early readmission, with uncertain benefit. Interpretable machine learning (ML) may assist in transparently identifying the risk of important outcomes. We conducted a retrospective cohort study of hospitalizations at a tertiary academic medical center and its branches from January 2011 to May 2018. A consecutive sample of all hospitalizations in the study period were included. Algorithms were trained on medical, sociodemographic, and institutional variables to predict readmission, length of stay (LOS), and death within 48-72 h. Prediction performance was measured by area under the receiver operator characteristic curve (AUC), Brier score loss (BSL), which measures how well predicted probability matches observed probability, and other metrics. Interpretations were generated using multiple feature extraction algorithms. The study cohort included 1,485,880 hospitalizations for 708,089 unique patients (median age of 59 years, first and third quartiles (QI) [39, 73]; 55.6% female; 71% white). There were 211,022 30-day readmissions for an overall readmission rate of 14% (for patients ≥65 years: 16%). Median LOS, including observation and labor and delivery patients, was 2.94 days (QI [1.67, 5.34]), or, if these patients are excluded, 3.71 days (QI [2.15, 6.51]). Predictive performance was as follows: 30-day readmission (AUC 0.76/BSL 0.11); LOS > 5 days (AUC 0.84/BSL 0.15); death within 48-72 h (AUC 0.91/BSL 0.001). Explanatory diagrams showed factors that impacted each prediction.
| S-EPMC7125114 | biostudies-literature
Project description:Failure to attend scheduled hospital appointments disrupts clinical management and consumes resource estimated at £1 billion annually in the United Kingdom National Health Service alone. Accurate stratification of absence risk can maximize the yield of preventative interventions. The wide multiplicity of potential causes, and the poor performance of systems based on simple, linear, low-dimensional models, suggests complex predictive models of attendance are needed. Here, we quantify the effect of using complex, non-linear, high-dimensional models enabled by machine learning. Models systematically varying in complexity based on logistic regression, support vector machines, random forests, AdaBoost, or gradient boosting machines were trained and evaluated on an unselected set of 22,318 consecutive scheduled magnetic resonance imaging appointments at two UCL hospitals. High-dimensional Gradient Boosting Machine-based models achieved the best performance reported in the literature, exhibiting an area under the receiver operating characteristic curve of 0.852 and average precision of 0.511. Optimal predictive performance required 81 variables. Simulations showed net potential benefit across a wide range of attendance characteristics, peaking at £3.15 per appointment at current prevalence and call efficiency. Optimal attendance prediction requires more complex models than have hitherto been applied in the field, reflecting the complex interplay of patient, environmental, and operational causal factors. Far from an exotic luxury, high-dimensional models based on machine learning are likely essential to optimal scheduling amongst other operational aspects of hospital care. High predictive performance is achievable with data from a single institution, obviating the need for aggregating large-scale sensitive data across governance boundaries.
| S-EPMC6550247 | biostudies-literature
Project description:BackgroundCardiac arrest is a life-threatening cessation of activity in the heart. Early prediction of cardiac arrest is important, as it allows for the necessary measures to be taken to prevent or intervene during the onset. Artificial intelligence (AI) technologies and big data have been increasingly used to enhance the ability to predict and prepare for the patients at risk.ObjectiveThis study aims to explore the use of AI technology in predicting cardiac arrest as reported in the literature.MethodsA scoping review was conducted in line with the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping reviews. Scopus, ScienceDirect, Embase, the Institute of Electrical and Electronics Engineers, and Google Scholar were searched to identify relevant studies. Backward reference list checks of the included studies were also conducted. Study selection and data extraction were independently conducted by 2 reviewers. Data extracted from the included studies were synthesized narratively.ResultsOut of 697 citations retrieved, 41 studies were included in the review, and 6 were added after backward citation checking. The included studies reported the use of AI in the prediction of cardiac arrest. Of the 47 studies, we were able to classify the approaches taken by the studies into 3 different categories: 26 (55%) studies predicted cardiac arrest by analyzing specific parameters or variables of the patients, whereas 16 (34%) studies developed an AI-based warning system. The remaining 11% (5/47) of studies focused on distinguishing patients at high risk of cardiac arrest from patients who were not at risk. Two studies focused on the pediatric population, and the rest focused on adults (45/47, 96%). Most of the studies used data sets with a size of <10,000 samples (32/47, 68%). Machine learning models were the most prominent branch of AI used in the prediction of cardiac arrest in the studies (38/47, 81%), and the most used algorithm was the neural network (23/47, 49%). K-fold cross-validation was the most used algorithm evaluation tool reported in the studies (24/47, 51%).ConclusionsAI is extensively used to predict cardiac arrest in different patient settings. Technology is expected to play an integral role in improving cardiac medicine. There is a need for more reviews to learn the obstacles to the implementation of AI technologies in clinical settings. Moreover, research focusing on how to best provide clinicians with support to understand, adapt, and implement this technology in their practice is also necessary.
| S-EPMC8726033 | biostudies-literature
Project description:Acute kidney injury (AKI) is a common complication of hospitalization that greatly and negatively affects the short-term and long-term outcomes of patients. Current guidelines use serum creatinine level and urine output rate for defining AKI and as the staging criteria of AKI. However, because they are not sensitive or specific markers of AKI, clinicians find it difficult to predict the occurrence of AKI and prescribe timely treatment. Advances in computing technology have led to the recent use of machine learning and artificial intelligence in AKI prediction, recent research reported that by using electronic health records (EHR) the AKI prediction via machine-learning models can reach AUROC over 0.80, in some studies even reach 0.93. Our review begins with the background and history of the definition of AKI, and the evolution of AKI risk factors and prediction models is also appraised. Then, we summarize the current evidence regarding the application of e-alert systems and machine-learning models in AKI prediction.
| S-EPMC8701097 | biostudies-literature
Project description:Background In-hospital cardiac arrest is a major burden in health care. Although several track-and-trigger systems are used to predict cardiac arrest, they often have unsatisfactory performances. We hypothesized that a deep-learning-based artificial intelligence algorithm (DLA) could effectively predict cardiac arrest using electrocardiography (ECG). We developed and validated a DLA for predicting cardiac arrest using ECG. Methods We conducted a retrospective study that included 47,505 ECGs of 25,672 adult patients admitted to two hospitals, who underwent at least one ECG from October 2016 to September 2019. The endpoint was occurrence of cardiac arrest within 24?h from ECG. Using subgroup analyses in patients who were initially classified as non-event, we confirmed the delayed occurrence of cardiac arrest and unexpected intensive care unit transfer over 14?days. Results We used 32,294 ECGs of 10,461 patients and 4483 ECGs of 4483 patients from a hospital were used as development and internal validation data, respectively. Additionally, 10,728 ECGs of 10,728 patients from another hospital were used as external validation data, which confirmed the robustness of the developed DLA. During internal and external validation, the areas under the receiver operating characteristic curves of the DLA in predicting cardiac arrest within 24?h were 0.913 and 0.948, respectively. The high risk group of the DLA showed a significantly higher hazard for delayed cardiac arrest (5.74% vs. 0.33%, P?<?0.001) and unexpected intensive care unit transfer (4.23% vs. 0.82%, P?<?0.001). A sensitivity map of the DLA displayed the ECG regions used to predict cardiac arrest, with the DLA focused most on the QRS complex. Conclusions Our DLA successfully predicted cardiac arrest using diverse formats of ECG. The results indicate that cardiac arrest could be screened and predicted not only with a conventional 12-lead ECG, but also with a single-lead ECG using a wearable device that employs our DLA.
| S-EPMC7541213 | biostudies-literature
Project description:BACKGROUND: All polypeptide backbones have the potential to form amyloid fibrils, which are associated with a number of degenerative disorders. However, the likelihood that amyloidosis would actually occur under physiological conditions depends largely on the amino acid composition of a protein. We explore using a naive Bayesian classifier and a weighted decision tree for predicting the amyloidogenicity of immunoglobulin sequences. RESULTS: The average accuracy based on leave-one-out (LOO) cross validation of a Bayesian classifier generated from 143 amyloidogenic sequences is 60.84%. This is consistent with the average accuracy of 61.15% for a holdout test set comprised of 103 AM and 28 non-amyloidogenic sequences. The LOO cross validation accuracy increases to 81.08% when the training set is augmented by the holdout test set. In comparison, the average classification accuracy for the holdout test set obtained using a decision tree is 78.64%. Non-amyloidogenic sequences are predicted with average LOO cross validation accuracies between 74.05% and 77.24% using the Bayesian classifier, depending on the training set size. The accuracy for the holdout test set was 89%. For the decision tree, the non-amyloidogenic prediction accuracy is 75.00%. CONCLUSIONS: This exploratory study indicates that both classification methods may be promising in providing straightforward predictions on the amyloidogenicity of a sequence. Nevertheless, the number of available sequences that satisfy the premises of this study are limited, and are consequently smaller than the ideal training set size. Increasing the size of the training set clearly increases the accuracy, and the expansion of the training set to include not only more derivatives, but more alignments, would make the method more sound. The accuracy of the classifiers may also be improved when additional factors, such as structural and physico-chemical data, are considered. The development of this type of classifier has significant applications in evaluating engineered antibodies, and may be adapted for evaluating engineered proteins in general.
| S-EPMC3098112 | biostudies-literature
Project description:BackgroundCoronary artery stenosis increases hospital mortality and leads to poor neurological recovery in cardiac arrest (CA) patients. However, electrocardiography (ECG) cannot fully predict the presence of coronary artery stenosis in CA patients. Hence, we aimed to determine whether regional wall motion abnormality (RWMA), as observed by two-dimensional echocardiography (2DE), predicted patient survival outcomes with greater accuracy than did ST segment elevation (STE) on ECG in CA patients who underwent coronary angiography (CAG) after return of spontaneous circulation.MethodsThis was a retrospective observational study of adult patients with CA of presumed cardiac etiology who underwent CAG at a single tertiary care hospital. We investigated whether RWMA observed on 2DE predicted patient outcomes more accurately than did STE observed on ECG. The primary outcome was incidence of hospital mortality. The secondary outcomes were Glasgow-Pittsburgh Cerebral Performance Category scores measured 6 months after discharge and significant coronary artery stenosis on CAG.ResultsAmong the 145 patients, 36 (24.8%) experienced in-hospital death. In multivariable analysis of survival outcomes, only total arrest time (P=0.011) and STE (P=0.035) were significant. The odds ratio (OR) and 95% confidence interval (CI), which were obtained by adjusting the total arrest time for survival outcomes, were significant only for STE (OR, 0.40; 95% CI, 0.17-0.94). The presence of RWMA was not a significant factor.ConclusionsWhile STE predicted survival outcomes in adult CA patients, RWMA did not. The decision to perform CAG after CA should include ECG under existing guidelines. The use of RWMA has limited benefits in treatment of this population.
| S-EPMC7940108 | biostudies-literature
Project description:We describe a novel collaboration between academia and industry, an in-house data science and artificial intelligence challenge held by Novartis to develop machine-learning models for predicting drug-development outcomes, building upon research at MIT using data from Informa as the starting point. With over 50 cross-functional teams from 25 Novartis offices around the world participating in the challenge, the domain expertise of these Novartis researchers was leveraged to create predictive models with greater sophistication. Ultimately, two winning teams developed models that outperformed the baseline MIT model-areas under the curve of 0.88 and 0.84 versus 0.78, respectively-through state-of-the-art machine-learning algorithms and the use of newly incorporated features and data. In addition to validating the variables shown to be associated with drug approval in the earlier MIT study, the challenge also provided new insights into the drivers of drug-development success and failure.
| S-EPMC8369231 | biostudies-literature
Project description:Early diagnosis is the most important determinant of oral and oropharyngeal squamous cell carcinoma (OPSCC) outcomes, yet most of these cancers are detected late, when outcomes are poor. Typically, nonspecialists such as dentists screen for oral cancer risk, and then they refer high-risk patients to specialists for biopsy-based diagnosis. Because the clinical appearance of oral mucosal lesions is not an adequate indicator of their diagnosis, status, or risk level, this initial triage process is inaccurate, with poor sensitivity and specificity. The objective of this study is to provide an overview of emerging optical imaging modalities and novel artificial intelligence-based approaches, as well as to evaluate their individual and combined utility and implications for improving oral cancer detection and outcomes. The principles of image-based approaches to detecting oral cancer are placed within the context of clinical needs and parameters. A brief overview of artificial intelligence approaches and algorithms is presented, and studies that use these 2 approaches singly and together are cited and evaluated. In recent years, a range of novel imaging modalities has been investigated for their applicability to improving oral cancer outcomes, yet none of them have found widespread adoption or significantly affected clinical practice or outcomes. Artificial intelligence approaches are beginning to have considerable impact in improving diagnostic accuracy in some fields of medicine, but to date, only limited studies apply to oral cancer. These studies demonstrate that artificial intelligence approaches combined with imaging can have considerable impact on oral cancer outcomes, with applications ranging from low-cost screening with smartphone-based probes to algorithm-guided detection of oral lesion heterogeneity and margins using optical coherence tomography. Combined imaging and artificial intelligence approaches can improve oral cancer outcomes through improved detection and diagnosis.
| S-EPMC7036512 | biostudies-literature