Dataset Information

Artificial intelligence outperforms standard blood-based scores in identifying liver fibrosis patients in primary care.

ABSTRACT: For years, hepatologists have been seeking non-invasive methods able to detect significant liver fibrosis. However, no previous algorithm using routine blood markers has proven to be clinically appropriate in primary care. We present a novel approach based on artificial intelligence, able to predict significant liver fibrosis in low-prevalence populations using routinely available patient data. We built six ensemble learning models (LiverAID) with different complexities using a prospective screening cohort of 3352 asymptomatic subjects. 463 patients were at a significant risk that justified performing a liver biopsy. Using an unseen hold-out dataset, we conducted a head-to-head comparison with conventional methods: standard blood-based indices (FIB-4, Forns and APRI) and transient elastography (TE). LiverAID models appropriately identified patients with significant liver stiffness (> 8 kPa) (AUC of 0.86, 0.89, 0.91, 0.92, 0.92 and 0.94, and NPV ≥ 0.98), and had a significantly superior discriminative ability (p < 0.01) than conventional blood-based indices (AUC = 0.60-0.76). Compared to TE, LiverAID models showed a good ability to rule out significant biopsy-assessed fibrosis stages. Given the ready availability of the required data and the relatively high performance, our artificial intelligence-based models are valuable screening tools that could be used clinically for early identification of patients with asymptomatic chronic liver diseases in primary care.

SUBMITTER: Blanes-Vidal V

PROVIDER: S-EPMC8861108 | biostudies-literature | 2022 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Artificial intelligence outperforms standard blood-based scores in identifying liver fibrosis patients in primary care.

Blanes-Vidal Victoria V Lindvig Katrine P KP Thiele Maja M Nadimi Esmaeil S ES Krag Aleksander A

Scientific reports 20220221 1

For years, hepatologists have been seeking non-invasive methods able to detect significant liver fibrosis. However, no previous algorithm using routine blood markers has proven to be clinically appropriate in primary care. We present a novel approach based on artificial intelligence, able to predict significant liver fibrosis in low-prevalence populations using routinely available patient data. We built six ensemble learning models (LiverAID) with different complexities using a prospective scree ...[more]

PMID: 35190650

Similar Datasets

Project description:ImportanceEarly warning decision support tools to identify clinical deterioration in the hospital are widely used, but there is little information on their comparative performance.ObjectiveTo compare 3 proprietary artificial intelligence (AI) early warning scores and 3 publicly available simple aggregated weighted scores.Design, setting, and participantsThis retrospective cohort study was performed at 7 hospitals in the Yale New Haven Health System. All consecutive adult medical-surgical ward hospital encounters between March 9, 2019, and November 9, 2023, were included.ExposuresSimultaneous Epic Deterioration Index (EDI), Rothman Index (RI), eCARTv5 (eCART), Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and NEWS2 scores.Main outcomes and measuresClinical deterioration, defined as a transfer from ward to intensive care unit or death within 24 hours of an observation.ResultsOf the 362 926 patient encounters (median patient age, 64 [IQR, 47-77] years; 200 642 [55.3%] female), 16 693 (4.6%) experienced a clinical deterioration event. eCART had the highest area under the receiver operating characteristic curve at 0.895 (95% CI, 0.891-0.900), followed by NEWS2 at 0.831 (95% CI, 0.826-0.836), NEWS at 0.829 (95% CI, 0.824-0.835), RI at 0.828 (95% CI, 0.823-0.834), EDI at 0.808 (95% CI, 0.802-0.812), and MEWS at 0.757 (95% CI, 0.750-0.764). After matching scores at the moderate-risk sensitivity level for a NEWS score of 5, overall positive predictive values (PPVs) ranged from a low of 6.3% (95% CI, 6.1%-6.4%) for an EDI score of 41 to a high of 17.3% (95% CI, 16.9%-17.8%) for an eCART score of 94. Matching scores at the high-risk specificity of a NEWS score of 7 yielded overall PPVs ranging from a low of 14.5% (95% CI, 14.0%-15.2%) for an EDI score of 54 to a high of 23.3% (95% CI, 22.7%-24.2%) for an eCART score of 97. The moderate-risk thresholds provided a median of at least 20 hours of lead time for all the scores. Median lead time at the high-risk threshold was 11 (IQR, 0-69) hours for eCART, 8 (IQR, 0-63) hours for NEWS, 6 (IQR, 0-62) hours for NEWS2, 5 (IQR, 0-56) hours for MEWS, 1 (IQR, 0-39) hour for EDI, and 0 (IQR, 0-42) hours for RI.Conclusions and relevanceIn this cohort study of inpatient encounters, eCART outperformed the other AI and non-AI scores, identifying more deteriorating patients with fewer false alarms and sufficient time to intervene. NEWS, a non-AI, publicly available early warning score, significantly outperformed EDI. Given the wide variation in accuracy, additional transparency and oversight of early warning tools may be warranted.

Project description:Introduction: Prognostic scores are important tools in oncology to facilitate clinical decision-making based on patient characteristics. To date, classic survival analysis using Cox proportional hazards regression has been employed in the development of these prognostic scores. With the advance of analytical models, this study aimed to determine if more complex machine-learning algorithms could outperform classical survival analysis methods. Methods: In this benchmarking study, two datasets were used to develop and compare different prognostic models for overall survival in pan-cancer populations: a nationwide EHR-derived de-identified database for training and in-sample testing and the OAK (phase III clinical trial) dataset for out-of-sample testing. A real-world database comprised 136K first-line treated cancer patients across multiple cancer types and was split into a 90% training and 10% testing dataset, respectively. The OAK dataset comprised 1,187 patients diagnosed with non-small cell lung cancer. To assess the effect of the covariate number on prognostic performance, we formed three feature sets with 27, 44 and 88 covariates. In terms of methods, we benchmarked ROPRO, a prognostic score based on the Cox model, against eight complex machine-learning models: regularized Cox, Random Survival Forests (RSF), Gradient Boosting (GB), DeepSurv (DS), Autoencoder (AE) and Super Learner (SL). The C-index was used as the performance metric to compare different models. Results: For in-sample testing on the real-world database the resulting C-index [95% CI] values for RSF 0.720 [0.716, 0.725], GB 0.722 [0.718, 0.727], DS 0.721 [0.717, 0.726] and lastly, SL 0.723 [0.718, 0.728] showed significantly better performance as compared to ROPRO 0.701 [0.696, 0.706]. Similar results were derived across all feature sets. However, for the out-of-sample validation on OAK, the stronger performance of the more complex models was not apparent anymore. Consistently, the increase in the number of prognostic covariates did not lead to an increase in model performance. Discussion: The stronger performance of the more complex models did not generalize when applied to an out-of-sample dataset. We hypothesize that future research may benefit by adding multimodal data to exploit advantages of more complex models.

Project description:BackgroundThe global prevalence of non-alcoholic fatty liver disease (NAFLD) continues to rise. Non-invasive diagnostic modalities including ultrasonography and clinical scoring systems have been proposed as alternatives to liver biopsy but with limited performance. Artificial intelligence (AI) is currently being integrated with conventional diagnostic methods in the hopes of performance improvements. We aimed to estimate the performance of AI-assisted systems for diagnosing NAFLD, non-alcoholic steatohepatitis (NASH), and liver fibrosis.MethodsA systematic review was performed to identify studies integrating AI in the diagnosis of NAFLD, NASH, and liver fibrosis. Pooled sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and summary receiver operating characteristic curves were calculated.ResultsTwenty-five studies were included in the systematic review. Meta-analysis of 13 studies showed that AI significantly improved the diagnosis of NAFLD, NASH and liver fibrosis. AI-assisted ultrasonography had excellent performance for diagnosing NAFLD, with a sensitivity, specificity, PPV, NPV of 0.97 (95% confidence interval (CI): 0.91-0.99), 0.98 (95% CI: 0.89-1.00), 0.98 (95% CI: 0.93-1.00), and 0.95 (95% CI: 0.88-0.98), respectively. The performance of AI-assisted ultrasonography was better than AI-assisted clinical data sets for the identification of NAFLD, which provided a sensitivity, specificity, PPV, NPV of 0.75 (95% CI: 0.66-0.82), 0.82 (95% CI: 0.74-0.88), 0.75 (95% CI: 0.60-0.86), and 0.82 (0.74-0.87), respectively. The area under the curves were 0.98 and 0.85 for AI-assisted ultrasonography and AI-assisted clinical data sets, respectively. AI-integrated clinical data sets had a pooled sensitivity, specificity of 0.80 (95%CI: 0.75-0.85), 0.69 (95%CI: 0.53-0.82) for identifying NASH, as well as 0.99-1.00 and 0.76-1.00 for diagnosing liver fibrosis stage F1-F4, respectively.ConclusionAI-supported systems provide promising performance improvements for diagnosing NAFLD, NASH, and identifying liver fibrosis among NAFLD patients. Prospective trials with direct comparisons between AI-assisted modalities and conventional methods are warranted before real-world implementation.Protocol registrationPROSPERO (CRD42021230391).

Project description:ObjectivesThe continuous integration of artificial intelligence (AI) into clinical settings requires the development of up-to-date and robust guidelines and standard frameworks that consider the evolving challenges of AI implementation in medicine. This review evaluates the quality of these guideline and summarizes ethical frameworks, best practices, and recommendations.Materials and methodsThe Appraisal of Guidelines, Research, and Evaluation II tool was used to assess the quality of guidelines based on 6 domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. The protocol of this review including the eligibility criteria, the search strategy data extraction sheet and methods, was published prior to the actual review with International Registered Report Identifier of DERR1-10.2196/47105.ResultsThe initial search resulted in 4975 studies from 2 databases and 7 studies from manual search. Eleven articles were selected for data extraction based on the eligibility criteria. We found that while guidelines generally excel in scope, purpose, and editorial independence, there is significant variability in applicability and the rigor of guideline development. Well-established initiatives such as TRIPOD+AI, DECIDE-AI, SPIRIT-AI, and CONSORT-AI have shown high quality, particularly in terms of stakeholder involvement. However, applicability remains a prominent challenge among the guidelines. The result also showed that the reproducibility, ethical, and environmental aspects of AI in medicine still need attention from both medical and AI communities.DiscussionOur work highlights the need for working toward the development of integrated and comprehensive reporting guidelines that adhere to the principles of Findability, Accessibility, Interoperability and Reusability. This alignment is essential for fostering a cultural shift toward transparency and open science, which are pivotal milestone for sustainable digital health research.ConclusionThis review evaluates the current reporting guidelines, discussing their advantages as well as challenges and limitations.

Dataset Information

Artificial intelligence outperforms standard blood-based scores in identifying liver fibrosis patients in primary care.

Publications

Artificial intelligence outperforms standard blood-based scores in identifying liver fibrosis patients in primary care.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets