Dataset Information

Validation of a Machine Learning Model to Predict Childhood Lead Poisoning.

ABSTRACT:

Importance

Childhood lead poisoning causes irreversible neurobehavioral deficits, but current practice is secondary prevention.

Objective

To validate a machine learning (random forest) prediction model of elevated blood lead levels (EBLLs) by comparison with a parsimonious logistic regression.

Design, setting, and participants

This prognostic study for temporal validation of multivariable prediction models used data from the Women, Infants, and Children (WIC) program of the Chicago Department of Public Health. Participants included a development cohort of children born from January 1, 2007, to December 31, 2012, and a validation WIC cohort born from January 1 to December 31, 2013. Blood lead levels were measured until December 31, 2018. Data were analyzed from January 1 to October 31, 2019.

Exposures

Blood lead level test results; lead investigation findings; housing characteristics, permits, and violations; and demographic variables.

Main outcomes and measures

Incident EBLL (≥6 μg/dL). Models were assessed using the area under the receiver operating characteristic curve (AUC) and confusion matrix metrics (positive predictive value, sensitivity, and specificity) at various thresholds.

Results

Among 6812 children in the WIC validation cohort, 3451 (50.7%) were female, 3057 (44.9%) were Hispanic, 2804 (41.2%) were non-Hispanic Black, 458 (6.7%) were non-Hispanic White, and 442 (6.5%) were Asian (mean [SD] age, 5.5 [0.3] years). The median year of housing construction was 1919 (interquartile range, 1903-1948). Random forest AUC was 0.69 compared with 0.64 for logistic regression (difference, 0.05; 95% CI, 0.02-0.08). When predicting the 5% of children at highest risk to have EBLLs, random forest and logistic regression models had positive predictive values of 15.5% and 7.8%, respectively (difference, 7.7%; 95% CI, 3.7%-11.3%), sensitivity of 16.2% and 8.1%, respectively (difference, 8.1%; 95% CI, 3.9%-11.7%), and specificity of 95.5% and 95.1% (difference, 0.4%; 95% CI, 0.0%-0.7%).

Conclusions and relevance

The machine learning model outperformed regression in predicting childhood lead poisoning, especially in identifying children at highest risk. Such a model could be used to target the allocation of lead poisoning prevention resources to these children.

SUBMITTER: Potash E

PROVIDER: S-EPMC7495240 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of a Machine Learning Model to Predict Childhood Lead Poisoning.

Potash Eric E Ghani Rayid R Walsh Joe J Jorgensen Emile E Lohff Cortland C Prachand Nik N Mansour Raed R

JAMA network open 20200901 9

<h4>Importance</h4>Childhood lead poisoning causes irreversible neurobehavioral deficits, but current practice is secondary prevention.<h4>Objective</h4>To validate a machine learning (random forest) prediction model of elevated blood lead levels (EBLLs) by comparison with a parsimonious logistic regression.<h4>Design, setting, and participants</h4>This prognostic study for temporal validation of multivariable prediction models used data from the Women, Infants, and Children (WIC) program of the ...[more]

PMID: 32936296

Similar Datasets

Project description:ImportanceMachine-learning algorithms offer better predictive accuracy than traditional prognostic models but are too complex and opaque for clinical use.ObjectiveTo compare different machine learning methods in predicting overall mortality in cirrhosis and to use machine learning to select easily scored clinical variables for a novel cirrhosis prognostic model.Design, setting, and participantsThis prognostic study used a retrospective cohort of adult patients with cirrhosis or its complications seen in 130 hospitals and affiliated ambulatory clinics in the integrated, national Veterans Affairs health care system from October 1, 2011, to September 30, 2015. Patients were followed up through December 31, 2018. Data were analyzed from October 1, 2017, to May 31, 2020.ExposuresPotential predictors included demographic characteristics; liver disease etiology, severity, and complications; use of health care resources; comorbid conditions; and comprehensive laboratory and medication data. Patients were randomly selected for model development (66.7%) and validation (33.3%). Three different statistical and machine learning methods were evaluated: gradient descent boosting, logistic regression with least absolute shrinkage and selection operator (LASSO) regularization, and logistic regression with LASSO constrained to select no more than 10 predictors (partial pathway model). Predictor inclusion and model performance were evaluated in a 5-fold cross-validation. Last, the predictors identified in the most parsimonious (the partial path) model were refit using maximum-likelihood estimation (Cirrhosis Mortality Model [CiMM]), and its predictive performance was compared with that of the widely used Model for End Stage Liver Disease with sodium (MELD-Na) score.Main outcomes and measuresAll-cause mortality.ResultsOf the 107 939 patients with cirrhosis (mean [SD] age, 62.7 [9.6] years; 96.6% male; 66.3% white, 18.4% African American), the annual mortality rate ranged from 8.8% to 15.3%. In total, 32.7% of patients died within 3 years, and 46.2% died within 5 years after the index date. Models predicting 1-year mortality had good discrimination for the gradient descent boosting (area under the receiver operating characteristics curve [AUC], 0.81; 95% CI, 0.80-0.82), logistic regression with LASSO regularization (AUC, 0.78; 95% CI, 0.77-0.79), and the partial path logistic model (AUC, 0.78; 95% CI, 0.76-0.78). All models showed good calibration. The final CiMM model with machine learning-derived clinical variables offered significantly better discrimination than the MELD-Na score, with AUCs of 0.78 (95% CI, 0.77-0.79) vs 0.67 (95% CI, 0.66-0.68) for 1-year mortality, respectively (DeLong z = 17.00; P < .001).Conclusions and relevanceIn this study, simple machine learning techniques performed as well as the more advanced ensemble gradient boosting. Using the clinical variables identified from simple machine learning in a cirrhosis mortality model produced a new score more transparent than machine learning and more predictive than the MELD-Na score.

Project description:BackgroundAtrial fibrillation (AF) is a progressive disease, and its clinical type is classified according to the AF duration: paroxysmal AF, persistent AF (PeAF; AF duration of less than 1 year), and long-standing persistent AF (AF duration of more than 1 year). When considering the indication for catheter ablation, having a long AF duration is considered a risk factor for recurrence, and therefore, the duration of AF is an important factor in determining the treatment strategy for PeAF.ObjectiveThis study aims to improve the accuracy of the cardiologists' diagnosis of the AF duration, and the steps to achieve this goal are to develop a predictive model of the AF duration and validate the support performance of the prediction model.MethodsThe study included 272 patients with PeAF (aged 20-90 years), with data obtained between January 1, 2015, and December 31, 2023. Of those, 189 (69.5%) were included in the study, excluding 83 (30.5%) who met the exclusion criteria. Of the 189 patients included, 145 (76.7%) were used as training data to build the machine learning (ML) model and 44 (23.3%) were used as test data for predictive ability of the ML model. Using a questionnaire, 10 cardiologists (group A) evaluated whether the test data (44 patients) included AF of more than a 1-year duration (phase 1). Next, the same questionnaire was performed again after providing the ML model's answer (phase 2). Subsequently, another 10 cardiologists (group B) were shown the test results of group A, were made aware of the limitations of their own diagnostic abilities, and were then administered the same 2-stage test as group A.ResultsThe prediction results with the ML model using the test data provided 81.8% accuracy (72% sensitivity and 89% specificity). The mean percentage of correct answers in group A was 63.9% (SD 9.6%) for phase 1 and improved to 71.6% (SD 9.3%) for phase 2 (P=.01). The mean percentage of correct answers in group B was 59.8% (SD 5.3%) for phase 1 and improved to 68.2% (SD 5.9%) for phase 2 (P=.007). The mean percentage of answers that differed from the ML model's prediction for phase 2 (percentage of answers where cardiologists did not trust the ML model and believed their own determination) was 17.3% (SD 10.3%) in group A and 20.9% (SD 5%) in group B and was not significantly different (P=.85).ConclusionsML models predicting AF duration improved the diagnostic ability of cardiologists. However, cardiologists did not entirely rely on the ML model's prediction, even if they were aware of their diagnostic capability limitations.

Project description:BackgroundThough European Respiratory Society and American Thoracic Society (ERS/ATS) guidelines for pulmonary function test (PFT) interpretation recommend the use of the forced vital capacity (FVC) lower limit of normal (LLN) to exclude restriction, recent data suggest that the negative predictive value (NPV) of the FVC LLN is lower than has been accepted, particularly among non-Hispanic Black patients. We sought to develop and externally validate a machine learning (ML) model to predict restriction from spirometry and determine whether its use may improve the accuracy and equity of PFT interpretation.MethodsWe included PFTs with both static and dynamic lung volume measurements for patients between 18 and 80 years of age who were tested at pulmonary diagnostic labs within two health systems. We used PFTs from one health system to train logistic regression, random forest, and boosted tree models to predict restriction using demographic, anthropometric, and spirometric data. We used PFTs from the second health system to externally validate these models. The primary measure of model performance was the NPV. Racial equity was assessed by comparing the NPV among non-Hispanic Black and non-Hispanic White patients.FindingsA total of 42 462 PFTs were used for model development and 24 524 for external validation. The prevalence of restriction was 29.8% in the development dataset and 39.6% in the validation dataset. All three ML models outperformed the FVC LLN by a wide margin, both overall and among all demographic subgroups. The overall NPV of the random forest model (88.3%, 95% confidence interval [CI] 87.8% to 88.9%) was significantly greater than that of the FVC LLN (72.7%, 95% CI 72.1% to 73.3%). The NPV of the random forest model was greater than that of the FVC LLN among both non-Hispanic Black (74.6% [95% CI 72.5% to 76.6%] versus 49.5% [95% CI 47.8% to 51.2%]) and non-Hispanic White (90.9% [95% CI 90.3% to 91.5%] versus 79.6% [95% CI 78.9% to 80.3%]) patients.InterpretationML models to exclude restriction from spirometry improve the accuracy and equity of PFT interpretation but do not fully eliminate racial differences.

Project description:BackgroundPostoperative sepsis is one of the main causes of mortality after liver transplantation (LT). Our study aimed to develop and validate a predictive model for postoperative sepsis within 7 d in LT recipients using machine learning (ML) technology.MethodsData of 786 patients received LT from January 2015 to January 2020 was retrospectively extracted from the big data platform of Third Affiliated Hospital of Sun Yat-sen University. Seven ML models were developed to predict postoperative sepsis. The area under the receiver-operating curve (AUC), sensitivity, specificity, accuracy, and f1-score were evaluated as the model performances. The model with the best performance was validated in an independent dataset involving 118 adult LT cases from February 2020 to April 2021. The postoperative sepsis-associated outcomes were also explored in the study.ResultsAfter excluding 109 patients according to the exclusion criteria, 677 patients underwent LT were finally included in the analysis. Among them, 216 (31.9%) were diagnosed with sepsis after LT, which were related to more perioperative complications, increased postoperative hospital stay and mortality after LT (all p < .05). Our results revealed that a larger volume of red blood cell infusion, ascitic removal, blood loss and gastric drainage, less volume of crystalloid infusion and urine, longer anesthesia time, higher level of preoperative TBIL were the top 8 important variables contributing to the prediction of post-LT sepsis. The Random Forest Classifier (RF) model showed the best overall performance to predict sepsis after LT among the seven ML models developed in the study, with an AUC of 0.731, an accuracy of 71.6%, the sensitivity of 62.1%, and specificity of 76.1% in the internal validation set, and a comparable AUC of 0.755 in the external validation set.ConclusionsOur study enrolled eight pre- and intra-operative variables to develop an RF-based predictive model of post-LT sepsis to assist clinical decision-making procedure.

Project description:Background Early prediction model of hemodynamic instability has the potential to improve the critical care, whereas limited external validation on the generalizability. We aimed to independently validate the Hemodynamic Stability Index (HSI), a multi-parameter machine learning model, in predicting hemodynamic instability in Asian patients. Method Hemodynamic instability was marked by using inotropic, vasopressor, significant fluid therapy, and/or blood transfusions. This retrospective study included among 15,967 ICU patients who aged 20 years or older (not included 20 years) and stayed in ICU for more than 6 h admitted to Taipei Veteran General Hospital (TPEVGH) between January 1, 2010, and March 31, 2020, of whom hemodynamic instability occurred in 3053 patients (prevalence = 19%). These patients in unstable group received at least one intervention during their ICU stays, and the HSI score of both stable and unstable group was calculated in every hour before intervention. The model performance was assessed using the area under the receiver operating characteristic curve (AUROC) and was compared to single indicators like systolic blood pressure (SBP) and shock index. The hemodynamic instability alarm was set by selecting optimal threshold with high sensitivity, acceptable specificity, and lead time before intervention was calculated to indicate when patients were firstly identified as high risk of hemodynamic instability. Results The AUROC of HSI was 0.76 (95% CI, 0.75–0.77), which performed significantly better than shock Index (0.7; 95% CI, 0.69–0.71) and SBP (0.69; 95% CI, 0.68–0.70). By selecting 0.7 as a threshold, HSI predicted 72% of all 3053 patients who received hemodynamic interventions with 67% in specificity. Time-varying results also showed that HSI score significantly outperformed single indicators even up to 24 h before intervention. And 95% unstable patients can be identified more than 5 h in advance. Conclusions The HSI has acceptable discrimination but underestimates the risk of stable patients in predicting the onset of hemodynamic instability in an external cohort. Supplementary Information The online version contains supplementary material available at 10.1186/s13054-022-04088-9.

Project description:ContextChildhood lead poisoning prevention in the United States was marked by a largely failed medical approach from 1971 to 1990; an emergent (but small) healthy housing primary prevention strategy from 1991 to 2015; and implementation of large-scale proven interventions since then.ProgramChildhood Lead Poisoning Prevention & Healthy Housing.MethodsHistoric and recent health and housing data from the National Health and Nutrition Examination Survey (NHANES) and the American Healthy Homes Survey (AHHS) were retrieved to analyze trends and associated policy gaps.EvaluationApproximately 590 000 US children aged 1 through 5 years had elevated blood lead levels of 3.5 μg/dL and greater in 2016, and 4.3 million children resided in homes with lead paint in 2019. Despite large improvements, racial and other disparities remain stubbornly and statistically significant. The NHANES and the AHHS require larger sample sizes. The Centers for Disease Control and Prevention has not published children's blood lead surveillance and NHANES data in several years; the Department of Housing and Urban Development (HUD) has no analogous housing surveillance system; and the Environmental Protection Agency (EPA) and the Occupational Safety and Health Administration (OSHA) have not updated training, Superfund, and occupational standards in decades.DiscussionThe nation has been without a plan and an associated budget for more than 2 decades. Congress has not reformed the nation's main lead poisoning prevention laws in more than 30 years. Such reforms include stopping US companies from producing new residential lead paint in other countries; enabling the disclosure law to identify all residential lead hazards; closing loopholes in federally assisted housing regulations and mortgage insurance standards; harnessing tax policy to help homeowners mitigate lead hazards; streamlining training requirements; increasing the size of health and housing surveys and surveillance systems; and updating housing codes, medical guidance, dust lead standards, training, Superfund, and worker exposure limits. Congress and the president should reauthorize a cabinet-level task force (dormant since 2010) to develop a new strategic plan with an interagency budget to implement it. These reforms will scale and optimize markets, subsidies, enforcement, and other proven interventions to end ineffective, costly, harmful, and irrational cost shifting that threatens children, workers, and affordable housing.

Project description:ObjectiveHospitalized cancer patients are at high risk of venous thromboembolism (VTE). However, no predictive model has been specifically developed for this population. Machine learning (ML) is advantageous for model development. This study was aimed at developing predictive models using three different ML algorithms and logistic regression for VTE risk among hospitalized cancer patients and comparing their predictive performance.MethodsA retrospective case-control study was conducted on hospitalized cancer patients at Hunan Cancer Hospital, China, between October 1, 2021, and February 30, 2022. Patients diagnosed with vein thrombosis before or after admission were excluded. Patient, tumor, treatment, and laboratory indicator information was obtained from the hospital information system. The data were randomly split into distributions of 80% for training and 20% for testing. Logistic regression and three ML algorithms-the support vector machine, random forest, and extreme gradient boosting (XGBoost)-were used to develop the models. Model performance was compared using F1, G-mean, area under the receiver operating characteristic curve (AUROC), accuracy, precision, recall rate, and specificity. Feature rankings were achieved based on the permutation scores of the selected features in the optimal model.ResultsA total of 1100 patients (mean [SD] age, 54.75 [11.08] years; 485 [44.09%] male) were included in this study. There were 340 patients (30.9%) in the VTE group. The XGBoost model achieved the best performance with the following evaluation metrics: F1 (0.750), G-mean (0.816), AUROC (0.818), accuracy (0.845), precision (0.750), recall rate (0.750), and specificity (0.888). D-dimer level, diabetes, hypertension, pleural metastasis, and hematological malignancies were identified as the five most significant features of the XGBoost model.ConclusionsFour predictive models were developed using ML algorithms. The XGBoost model was the optimal predictive model compared with the other three models. This study indicates that ML may play an important role in VTE risk estimation among hospitalized patients with cancer and provides a reference for thromboprophylaxis.

Project description:BackgroundEarly childhood caries (ECC) is a major oral health problem among preschool children that can significantly influence children's quality of life. Machine learning can accurately predict the treatment outcome but its use in ECC management is limited. The aim of this study is to explore the application of machine learning in predicting the treatment outcome of ECC.MethodsThis study was a secondary analysis of a recently published clinical trial that recruited 1,070 children aged 3- to 4-year-old with ECC. Machine learning algorithms including Naive Bayes, logistic regression, decision tree, random forest, support vector machine, and extreme gradient boosting were adopted to predict the caries-arresting outcome of ECC at 30-month follow-up after receiving fluoride and silver therapy. Candidate predictors included clinical parameters (caries experience and oral hygiene status), oral health-related behaviours (toothbrushing habits, feeding history and snacking preference) and socioeconomic backgrounds of the children. Model performance was evaluated using discrimination and calibration metrics including accuracy, recall, precision, F1 score, area under the receiver operating characteristic curve (AUROC) and Brier score. Shapley additive explanations were deployed to identify the important predictors.ResultsAll machine learning models showed good performance in predicting the treatment outcome of ECC. The accuracy, recall, precision, F1 score, AUROC, and Brier score of the six models ranged from 0.674 to 0.740, 0.731 to 0.809, 0.762 to 0.802, 0.741 to 0.804, 0.771 to 0.859, and 0.134 to 0.227, respectively. The important predictors of the caries-arresting outcome were the surface and tooth location of the carious lesions, newly developed caries during follow-ups, baseline caries experience, whether the children had assisted toothbrushing and oral hygiene status.ConclusionsMachine learning can provide promising predictions of the treatment outcome of ECC. The identified key predictors would be particularly informative for targeted management of ECC.

Project description:BackgroundVolume overload is a common complication encountered in hospitalized patients, and the mainstay of therapy is diuresis. Unfortunately, the diuretic response in some individuals is inadequate despite a typical dose of loop diuretics, a phenomenon called diuretic resistance. An accurate prediction model that predicts diuretic resistance using predosing variables could inform the right diuretic dose for a prospective patient.MethodsTwo large, deidentified, publicly available, and independent intensive care unit (ICU) databases from the United States were used-the Medical Information Mart for Intensive Care III (MIMIC) and the Philips eICU databases. Loop diuretic resistance was defined as <1400 ml of urine per 40 mg of diuretic dose in 24 hours. Using 24-hour windows throughout admission, commonly accessible variables were obtained and incorporated into the model. Data imputation was performed using a highly accurate machine learning method. Using XGBoost, several models were created using train and test datasets from the eICU database. These were then combined into an ensemble model optimized for increased specificity and then externally validated on the MIMIC database.ResultsThe final ensemble model was composed of four separate models, each using 21 commonly available variables. The ensemble model outperformed individual models during validation. Higher serum creatinine, lower systolic blood pressure, lower serum chloride, higher age, and female sex were the most important predictors of diuretic resistance (in that order). The specificity of the model on external validation was 92%, yielding a positive likelihood ratio of 3.46 while maintaining overall discrimination (C-statistic 0.69).ConclusionsA diuretic resistance prediction model was created using machine learning and was externally validated in ICU populations. The model is easy to use, would provide actionable information at the bedside, and would be ready for implementation in existing electronic medical records. This study also provides a framework for the development of future machine learning models.

Dataset Information

Validation of a Machine Learning Model to Predict Childhood Lead Poisoning.

Importance

Objective

Design, setting, and participants

Exposures

Main outcomes and measures

Results

Conclusions and relevance

Publications

Validation of a Machine Learning Model to Predict Childhood Lead Poisoning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets