Diagnostic Accuracy of Machine Learning Models to Identify Congenital Heart Disease: A Meta-Analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Hoodbhoy Z
PROVIDER: S-EPMC8297386 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:In this meta-analysis, we aimed to estimate the diagnostic accuracy of machine learning models on digital mammograms and tomosynthesis in breast cancer classification and to assess the factors affecting its diagnostic accuracy. We searched for related studies in Web of Science, Scopus, PubMed, Google Scholar and Embase. The studies were screened in two stages to exclude the unrelated studies and duplicates. Finally, 36 studies containing 68 machine learning models were included in this meta-analysis. The area under the curve (AUC), hierarchical summary receiver operating characteristics (HSROC) curve, pooled sensitivity and pooled specificity were estimated using a bivariate Reitsma model. Overall AUC, pooled sensitivity and pooled specificity were 0.90 (95% CI: 0.85-0.90), 0.83 (95% CI: 0.78-0.87) and 0.84 (95% CI: 0.81-0.87), respectively. Additionally, the three significant covariates identified in this study were country (p = 0.003), source (p = 0.002) and classifier (p = 0.016). The type of data covariate was not statistically significant (p = 0.121). Additionally, Deeks' linear regression test indicated that there exists a publication bias in the included studies (p = 0.002). Thus, the results should be interpreted with caution.
| S-EPMC9320089 | biostudies-literature
Project description:Alzheimer's disease (AD) is a complex, and multifactorial neurodegenerative disease. Previous studies have revealed that oxidative stress, synaptic toxicity, autophagy, and neuroinflammation play crucial roles in the progress of AD, however, its pathogenesis is still unclear. Recent researches have indicated that ferroptosis, an iron-dependent programmed cell death, might be involved in the pathogenesis of AD. Therefore, we aim to screen correlative ferroptosis-related genes (FRGs) in the progress of AD to clarify insights into the diagnostic value. Interestingly, we identified eight FRGs were significantly differentially expressed in AD patients. 10,044 differentially expressed genes (DEGs) were finally identified by differential expression analysis. The following step was investigating the function of DEGs using gene set enrichment analysis (GSEA). Weight gene correlation analysis was performed to explore ten modules and 104 hub genes. Subsequently, based on machine learning algorithms, we constructed diagnostic classifiers to select characteristic genes. Through the multivariable logistic regression analysis, five features (RAF1, NFKBIA, MOV10L1, IQGAP1, FOXO1) were then validated, which composed a diagnostic model of AD. Thus, our findings not only developed genetic diagnostics strategy, but set a direction for further study of the disease pathogenesis and therapy targets.
| S-EPMC9575464 | biostudies-literature
Project description:Objective: Congenital heart diseases (CHDs) are associated with an extremely heavy global disease burden as the most common category of birth defects. Genetic and environmental factors have been identified as risk factors of CHDs previously. However, high volume clinical indicators have never been considered when predicting CHDs. This study aimed to predict the occurrence of CHDs by considering thousands of variables from self-reported questionnaires and routinely collected clinical laboratory data using machine learning algorithms. Methods: We conducted a birth cohort study at one of the largest cardiac centers in China from 2011 to 2017. All fetuses were screened for CHDs using ultrasound and cases were confirmed by at least two pediatric cardiologists using echocardiogram. A total of 1,127 potential predictors were included to predict CHDs. We used the Explainable Boosting Machine (EBM) for prediction and evaluated the model performance using area under the Receive Operating Characteristics (ROC) curves (AUC). The top predictors were selected according to their contributions and predictive values. Thresholds were calculated for the most significant predictors. Results: Overall, 5,390 mother-child pairs were recruited. Our prediction model achieved an AUC of 76% (69-83%) from out-of-sample predictions. Among the top 35 predictors of CHDs we identified, 34 were from clinical laboratory tests and only one was from the questionnaire (abortion history). Total accuracy, sensitivity, and specificity were 0.65, 0.74, and 0.65, respectively. Maternal serum uric acid (UA), glucose, and coagulation levels were the most consistent and significant predictors of CHDs. According to the thresholds of the predictors identified in our study, which did not reach the current clinical diagnosis criteria, elevated UA (>4.38 mg/dl), shortened activated partial thromboplastin time (<33.33 s), and elevated glucose levels were the most important predictors and were associated with ranges of 1.17-1.54 relative risks of CHDs. We have developed an online predictive tool for CHDs based on our findings that may help screening and prevention of CHDs. Conclusions: Maternal UA, glucose, and coagulation levels were the most consistent and significant predictors of CHDs. Thresholds below the current clinical definition of "abnormal" for these predictors could be used to help develop CHD screening and prevention strategies.
| S-EPMC8777022 | biostudies-literature
Project description:The aim of this paper is to summarize and evaluate results from existing studies on accuracy and clinical value of three-dimensional printed heart models (3DPHM) for determining whether 3D printing can significantly improve on how the congenital heart disease (CHD) is managed in current clinical practice. Proquest, Google Scholar, Scopus, PubMed, and Medline were searched for relevant studies until April 2019. Two independent reviewers performed manual data extraction and assessed the risk of bias of the studies using the tools published on National Institutes of Health (NIH) website. The following data were extracted from the studies: author, year of publication, study design, imaging modality, segmentation software, utility of 3DPHM, CHD types, and dimensional accuracy. R software was used for the meta-analysis. Twenty-four articles met the inclusion criteria and were included in the systematic review. However, only 7 studies met the statistical requirements and were eligible for meta-analysis. Cochran's Q test demonstrated significant variation among the studies for both of the meta-analyses of accuracy of 3DPHM and the utility of 3DPHM in medical education. Analysis of all included studies reported the mean deviation between the 3DPHM and the medical images is not significant, implying that 3DPHM are highly accurate. As for the utility of the 3DPHM, it is reported in all relevant studies that the 3DPHM improve the learning experience and satisfaction among the users, and play a critical role in facilitating surgical planning of complex CHD cases. 3DPHM have the potential to enhance communication in medical practice, however their clinical value remains debatable. More studies are required to yield a more meaningful meta-analysis.
| S-EPMC6780783 | biostudies-literature
Project description:PurposeEarly clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis.MethodsA systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance.ResultsAfter screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n = 15; 54%), followed by hospital wards (n = 7; 25%), the emergency department (ED, n = 4; 14%) and all of these settings (n = 2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance.ConclusionThis systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.
| S-EPMC7067741 | biostudies-literature
Project description:ObjectiveMachine learning (ML) diagnostic tools have significant potential to improve health care. However, methodological pitfalls may affect diagnostic test accuracy studies used to appraise such tools. We aimed to evaluate the prevalence and reporting of design characteristics within the literature. Further, we sought to empirically assess whether design features may be associated with different estimates of diagnostic accuracy.Materials and methodsWe systematically retrieved 2 × 2 tables (n = 281) describing the performance of ML diagnostic tools, derived from 114 publications in 38 meta-analyses, from PubMed. Data extracted included test performance, sample sizes, and design features. A mixed-effects metaregression was run to quantify the association between design features and diagnostic accuracy.ResultsParticipant ethnicity and blinding in test interpretation was unreported in 90% and 60% of studies, respectively. Reporting was occasionally lacking for rudimentary characteristics such as study design (28% unreported). Internal validation without appropriate safeguards was used in 44% of studies. Several design features were associated with larger estimates of accuracy, including having unreported (relative diagnostic odds ratio [RDOR], 2.11; 95% confidence interval [CI], 1.43-3.1) or case-control study designs (RDOR, 1.27; 95% CI, 0.97-1.66), and recruiting participants for the index test (RDOR, 1.67; 95% CI, 1.08-2.59).DiscussionSignificant underreporting of experimental details was present. Study design features may affect estimates of diagnostic performance in the ML diagnostic test accuracy literature.ConclusionsThe present study identifies pitfalls that threaten the validity, generalizability, and clinical value of ML diagnostic tools and provides recommendations for improvement.
| S-EPMC7647361 | biostudies-literature
Project description:The quality of treatment and prognosis after pediatric congenital heart surgery remains unsatisfactory. A reliable prediction model for postoperative complications of congenital heart surgery patients is essential to enable prompt initiation of therapy and improve the quality of prognosis. Here, we develop an interpretable machine-learning-based model that integrates patient demographics, surgery-specific features and intraoperative blood pressure data for accurately predicting complications after pediatric congenital heart surgery. We used blood pressure variability and the k-means algorithm combined with a smoothed formulation of dynamic time wrapping to extract features from time-series data. In addition, SHAP framework was used to provide explanations of the prediction. Our model achieved the best performance both in binary and multi-label classification compared with other consensus-based risk models. In addition, this explainable model explains why a prediction was made to help improve the clinical understanding of complication risk and generate actionable knowledge in practice. The combination of model performance and interpretability is easy for clinicians to trust and provide insight into how they should respond before the condition worsens after pediatric congenital heart surgery.
| S-EPMC8390484 | biostudies-literature
Project description:BackgroundKidney disease progression rates vary among patients. Rapid and accurate prediction of kidney disease outcomes is crucial for disease management. In recent years, various prediction models using Machine Learning (ML) algorithms have been established in nephrology. However, their accuracy have been inconsistent. Therefore, we conducted a systematic review and meta-analysis to investigate the diagnostic accuracy of ML algorithms for kidney disease progression.MethodsWe searched PubMed, EMBASE, Cochrane Central Register of Controlled Trials, the Chinese Biomedicine Literature Database, Chinese National Knowledge Infrastructure, Wanfang Database, and the VIP Database for diagnostic studies on ML algorithms' accuracy in predicting kidney disease prognosis, from the establishment of these databases until October 2020. Two investigators independently evaluate study quality by QUADAS-2 tool and extracted data from single ML algorithm for data synthesis using the bivariate model and the hierarchical summary receiver operating characteristic (HSROC) model.ResultsFifteen studies were left after screening, only 6 studies were eligible for data synthesis. The sample size of these 6 studies was 12,534, and the kidney disease types could be divided into chronic kidney disease (CKD) and Immunoglobulin A Nephropathy, with 5 articles using end-stage renal diseases occurrence as the primary outcome. The main results indicated that the area under curve (AUC) of the HSROC was 0.87 (0.84-0.90) and ML algorithm exhibited a strong specificity, 95% confidence interval and heterogeneity (I2) of (0.87, 0.84-0.90, [I2 99.0%]) and a weak sensitivity of (0.68, 0.58-0.77, [I2 99.7%]) in predicting kidney disease deterioration. And the the results of subgroup analysis indicated that ML algorithm's AUC for predicting CKD prognosis was 0.82 (0.79-0.85), with the pool sensitivity of (0.64, 0.49-0.77, [I2 99.20%]) and pool specificity of (0.84, 0.74-0.91, [I2 99.84%]). The ML algorithm's AUC for predicting IgA nephropathy prognosis was 0.78 (0.74-0.81), with the pool sensitivity of (0.74, 0.71-0.77, [I2 7.10%]) and pool specificity of (0.93, 0.91-0.95, [I2 83.92%]).ConclusionTaking advantage of big data, ML algorithm-based prediction models have high accuracy in predicting kidney disease progression, we recommend ML algorithms as an auxiliary tool for clinicians to determine proper treatment and disease management strategies.
| S-EPMC9341041 | biostudies-literature
Project description:IntroductionUnderstanding congenital heart disease (CHD) is vital for medical personnel and parents of affected children. While traditional 2D schematics serve as the typical approach used, several studies have shown these models to be limiting in understanding complex structures. Recent world-emphasis has shifted to 3D printed models as a complement to 2D imaging to bridge knowledge and create new opportunities for experiential learning. We sought to systematically compare 3D digital and physical models for medical personnel and parent education compared to traditional methods.Methods3D printed and digital models were made out of MRI and CT data for 20 common CHD. Fellows and nurse practitioners used these models to explore intra-cardiac pathologies following traditional teaching. The models were also used for parent education in outpatient settings after traditional education. The participants were then asked to fill out a Likert scale questionnaire to assess their understanding and satisfaction with different teaching techniques. These ratings were compared using paired t-tests and Pearson's correlation.ResultsTwenty-five medical personnel (18 fellows; 2 nurses; 4 nurse practitioners and one attending) and twenty parents participated in the study. The diagnosis varied from simple mitral valve pathology to complex single ventricle palliation. Parent and medical personnel perceived understanding with digital models was significantly higher than traditional (p = 0.01). Subjects also felt that physical models were overall more useful than digital ones (p = 0.001). Physicians using models for parent education also perceived the models to be useful, not significantly impacting their clinical workflow.Conclusions3D models, both digital and printed, enhance medical personnel and parental perceived understanding of CHD.
| S-EPMC8411549 | biostudies-literature
Project description:BackgroundThis systematic review and meta-analysis were conducted to objectively evaluate the evidence of machine learning (ML) in the patient diagnosis of Intracranial Hemorrhage (ICH) on computed tomography (CT) scans.MethodsUntil May 2023, systematic searches were conducted in ISI Web of Science, PubMed, Scopus, Cochrane Library, IEEE Xplore Digital Library, CINAHL, Science Direct, PROSPERO, and EMBASE for studies that evaluated the diagnostic precision of ML model-assisted ICH detection. Patients with and without ICH as the target condition who were receiving CT-Scan were eligible for the research, which used ML algorithms based on radiologists' reports as the gold reference standard. For meta-analysis, pooled sensitivities, specificities, and a summary receiver operating characteristics curve (SROC) were used.ResultsAt last, after screening the title, abstract, and full paper, twenty-six retrospective and three prospective, and two retrospective/prospective studies were included. The overall (Diagnostic Test Accuracy) DTA of retrospective studies with a pooled sensitivity was 0.917 (95% CI 0.88-0.943, I2 = 99%). The pooled specificity was 0.945 (95% CI 0.918-0.964, I2 = 100%). The pooled diagnostic odds ratio (DOR) was 219.47 (95% CI 104.78-459.66, I2 = 100%). These results were significant for the specificity of the different network architecture models (p-value = 0.0289). However, the results for sensitivity (p-value = 0.6417) and DOR (p-value = 0.2187) were not significant. The ResNet algorithm has higher pooled specificity than other algorithms with 0.935 (95% CI 0.854-0.973, I2 = 93%).ConclusionThis meta-analysis on DTA of ML algorithms for detecting ICH by assessing non-contrast CT-Scans shows the ML has an acceptable performance in diagnosing ICH. Using ResNet in ICH detection remains promising prediction was improved via training in an Architecture Learning Network (ALN).
| S-EPMC10694901 | biostudies-literature