Deep learning for classification of pediatric chest radiographs by WHO's standardized methodology.
Ontology highlight
ABSTRACT: Background
The World Health Organization (WHO)-defined radiological pneumonia is a preferred endpoint in pneumococcal vaccine efficacy and effectiveness studies in children. Automating the WHO methodology may support more widespread application of this endpoint.Methods
We trained a deep learning model to classify pneumonia CXRs in children using the World Health Organization (WHO)'s standardized methodology. The model was pretrained on CheXpert, a dataset containing 224,316 adult CXRs, and fine-tuned on PERCH, a pediatric dataset containing 4,172 CXRs. The model was then tested on two pediatric CXR datasets released by WHO. We also compared the model's performance to that of radiologists and pediatricians.Results
The average area under the receiver operating characteristic curve (AUC) for primary endpoint pneumonia (PEP) across 10-fold validation of PERCH images was 0.928; average AUC after testing on WHO images was 0.977. The model's classification performance was better on test images with high inter-observer agreement; however, the model still outperformed human assessments in AUC and precision-recall spaces on low agreement images.Conclusion
A deep learning model can classify pneumonia CXR images in children at a performance comparable to human readers. Our method lays a strong foundation for the potential inclusion of computer-aided readings of pediatric CXRs in vaccine trials and epidemiology studies.
SUBMITTER: Chen Y
PROVIDER: S-EPMC8216551 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:PurposeTo develop a machine learning model to classify the severity grades of pulmonary edema on chest radiographs.Materials and methodsIn this retrospective study, 369 071 chest radiographs and associated radiology reports from 64 581 patients (mean age, 51.71 years; 54.51% women) from the MIMIC-CXR chest radiograph dataset were included. This dataset was split into patients with and without congestive heart failure (CHF). Pulmonary edema severity labels from the associated radiology reports were extracted from patients with CHF as four different ordinal levels: 0, no edema; 1, vascular congestion; 2, interstitial edema; and 3, alveolar edema. Deep learning models were developed using two approaches: a semisupervised model using a variational autoencoder and a pretrained supervised learning model using a dense neural network. Receiver operating characteristic curve analysis was performed on both models.ResultsThe area under the receiver operating characteristic curve (AUC) for differentiating alveolar edema from no edema was 0.99 for the semisupervised model and 0.87 for the pretrained models. Performance of the algorithm was inversely related to the difficulty in categorizing milder states of pulmonary edema (shown as AUCs for semisupervised model and pretrained model, respectively): 2 versus 0, 0.88 and 0.81; 1 versus 0, 0.79 and 0.66; 3 versus 1, 0.93 and 0.82; 2 versus 1, 0.69 and 0.73; and 3 versus 2, 0.88 and 0.63.ConclusionDeep learning models were trained on a large chest radiograph dataset and could grade the severity of pulmonary edema on chest radiographs with high performance.Supplemental material is available for this article.See also the commentary by Auffermann in this issue.© RSNA, 2021.
| S-EPMC8043362 | biostudies-literature
Project description:We propose a deep learning system to automatically detect four explainable emphysema signs on frontal and lateral chest radiographs. Frontal and lateral chest radiographs from 3000 studies were retrospectively collected. Two radiologists annotated these with 4 radiological signs of pulmonary emphysema identified from the literature. A patient with ≥2 of these signs present is considered emphysema positive. Using separate deep learning systems for frontal and lateral images we predict the presence of each of the four visual signs and use these to determine emphysema positivity. The ROC and AUC results on a set of 422 held-out cases, labeled by both radiologists, are reported. Comparison with a black-box model which predicts emphysema without the use of explainable visual features is made on the annotations from both radiologists, as well as the subset that they agreed on. DeLong's test is used to compare with the black-box model ROC and McNemar's test to compare with radiologist performance. In 422 test cases, emphysema positivity was predicted with AUCs of 0.924 and 0.946 using the reference standard from each radiologist separately. Setting model sensitivity equivalent to that of the second radiologist, our model has a comparable specificity (p = 0.880 and p = 0.143 for each radiologist respectively). Our method is comparable with the black-box model with AUCs of 0.915 (p = 0.407) and 0.935 (p = 0.291), respectively. On the 370 cases where both radiologists agreed (53 positives), our model achieves an AUC of 0.981, again comparable to the black-box model AUC of 0.972 (p = 0.289). Our proposed method can predict emphysema positivity on chest radiographs as well as a radiologist or a comparable black-box method. It additionally produces labels for four visual signs to ensure the explainability of the result. The dataset is publicly available at https://doi.org/10.5281/zenodo.6373392.
| S-EPMC9333227 | biostudies-literature
Project description:BackgroundCoronavirus disease 2019 (COVID-19) is a pandemic disease. Fast and accurate diagnosis of COVID-19 from chest radiography may enable more efficient allocation of scarce medical resources and hence improved patient outcomes. Deep learning classification of chest radiographs may be a plausible step towards this. We hypothesize that bone suppression of chest radiographs may improve the performance of deep learning classification of COVID-19 phenomena in chest radiographs.MethodsTwo bone suppression methods (Gusarev et al. and Rajaraman et al.) were implemented. The Gusarev and Rajaraman methods were trained on 217 pairs of normal and bone-suppressed chest radiographs from the X-ray Bone Shadow Suppression dataset (https://www.kaggle.com/hmchuong/xray-bone-shadow-supression). Two classifier methods with different network architectures were implemented. Binary classifier models were trained on the public RICORD-1c and RSNA Pneumonia Challenge datasets. An external test dataset was created retrospectively from a set of 320 COVID-19 positive patients from Queen Elizabeth Hospital (Hong Kong, China) and a set of 518 non-COVID-19 patients from Pamela Youde Nethersole Eastern Hospital (Hong Kong, China), and used to evaluate the effect of bone suppression on classifier performance. Classification performance, quantified by sensitivity, specificity, negative predictive value (NPV), accuracy and area under the receiver operating curve (AUC), for non-suppressed radiographs was compared to that for bone suppressed radiographs. Some of the pre-trained models used in this study are published at (https://github.com/danielnflam).ResultsBone suppression of external test data was found to significantly (P<0.05) improve AUC for one classifier architecture [from 0.698 (non-suppressed) to 0.732 (Rajaraman-suppressed)]. For the other classifier architecture, suppression did not significantly (P>0.05) improve or worsen classifier performance.ConclusionsRajaraman suppression significantly improved classification performance in one classification architecture, and did not significantly worsen classifier performance in the other classifier architecture. This research could be extended to explore the impact of bone suppression on classification of different lung pathologies, and the effect of other image enhancement techniques on classifier performance.
| S-EPMC9246721 | biostudies-literature
Project description:As one of the most ubiquitous diagnostic imaging tests in medical practice, chest radiography requires timely reporting of potential findings and diagnosis of diseases in the images. Automated, fast, and reliable detection of diseases based on chest radiography is a critical step in radiology workflow. In this work, we developed and evaluated various deep convolutional neural networks (CNN) for differentiating between normal and abnormal frontal chest radiographs, in order to help alert radiologists and clinicians of potential abnormal findings as a means of work list triaging and reporting prioritization. A CNN-based model achieved an AUC of 0.9824 ± 0.0043 (with an accuracy of 94.64 ± 0.45%, a sensitivity of 96.50 ± 0.36% and a specificity of 92.86 ± 0.48%) for normal versus abnormal chest radiograph classification. The CNN model obtained an AUC of 0.9804 ± 0.0032 (with an accuracy of 94.71 ± 0.32%, a sensitivity of 92.20 ± 0.34% and a specificity of 96.34 ± 0.31%) for normal versus lung opacity classification. Classification performance on the external dataset showed that the CNN model is likely to be highly generalizable, with an AUC of 0.9444 ± 0.0029. The CNN model pre-trained on cohorts of adult patients and fine-tuned on pediatric patients achieved an AUC of 0.9851 ± 0.0046 for normal versus pneumonia classification. Pretraining with natural images demonstrates benefit for a moderate-sized training image set of about 8500 images. The remarkable performance in diagnostic accuracy observed in this study shows that deep CNNs can accurately and effectively differentiate normal and abnormal chest radiographs, thereby providing potential benefits to radiology workflow and patient care.
| S-EPMC7224391 | biostudies-literature
Project description:ImportanceChest radiography is the most common diagnostic imaging test in medicine and may also provide information about longevity and prognosis.ObjectiveTo develop and test a convolutional neural network (CNN) (named CXR-risk) to predict long-term mortality, including noncancer death, from chest radiographs.Design, setting, and participantsIn this prognostic study, CXR-risk CNN development (n = 41 856) and testing (n = 10 464) used data from the screening radiography arm of the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial (PLCO) (n = 52 320), a community cohort of asymptomatic nonsmokers and smokers (aged 55-74 years) enrolled at 10 US sites from November 8, 1993, through July 2, 2001. External testing used data from the screening radiography arm of the National Lung Screening Trial (NLST) (n = 5493), a community cohort of heavy smokers (aged 55-74 years) enrolled at 21 US sites from August 2002, through April 2004. Data analysis was performed from January 1, 2018, to May 23, 2019.ExposureDeep learning CXR-risk score (very low, low, moderate, high, and very high) based on CNN analysis of the enrollment radiograph.Main outcomes and measuresAll-cause mortality. Prognostic value was assessed in the context of radiologists' diagnostic findings (eg, lung nodule) and standard risk factors (eg, age, sex, and diabetes) and for cause-specific mortality.ResultsAmong 10 464 PLCO participants (mean [SD] age, 62.4 [5.4] years; 5405 men [51.6%]; median follow-up, 12.2 years [interquartile range, 10.5-12.9 years]) and 5493 NLST test participants (mean [SD] age, 61.7 [5.0] years; 3037 men [55.3%]; median follow-up, 6.3 years [interquartile range, 6.0-6.7 years]), there was a graded association between CXR-risk score and mortality. The very high-risk group had mortality of 53.0% (PLCO) and 33.9% (NLST), which was higher compared with the very low-risk group (PLCO: unadjusted hazard ratio [HR], 18.3 [95% CI, 14.5-23.2]; NLST: unadjusted HR, 15.2 [95% CI, 9.2-25.3]; both P < .001). This association was robust to adjustment for radiologists' findings and risk factors (PLCO: adjusted HR [aHR], 4.8 [95% CI, 3.6-6.4]; NLST: aHR, 7.0 [95% CI, 4.0-12.1]; both P < .001). Comparable results were seen for lung cancer death (PLCO: aHR, 11.1 [95% CI, 4.4-27.8]; NLST: aHR, 8.4 [95% CI, 2.5-28.0]; both P ≤ .001) and for noncancer cardiovascular death (PLCO: aHR, 3.6 [95% CI, 2.1-6.2]; NLST: aHR, 47.8 [95% CI, 6.1-374.9]; both P < .001) and respiratory death (PLCO: aHR, 27.5 [95% CI, 7.7-97.8]; NLST: aHR, 31.9 [95% CI, 3.9-263.5]; both P ≤ .001).Conclusions and relevanceIn this study, the deep learning CXR-risk score stratified the risk of long-term mortality based on a single chest radiograph. Individuals at high risk of mortality may benefit from prevention, screening, and lifestyle interventions.
| S-EPMC6646994 | biostudies-literature
Project description:Chest radiographs (CXRs) are a valuable diagnostic tool in epidemiologic studies of pneumonia. The World Health Organization (WHO) methodology for the interpretation of pediatric CXRs has not been evaluated beyond its intended application as an endpoint measure for bacterial vaccine trials.The Pneumonia Etiology Research for Child Health (PERCH) study enrolled children aged 1-59 months hospitalized with WHO-defined severe and very severe pneumonia from 7 low- and middle-income countries. An interpretation process categorized each CXR into 1 of 5 conclusions: consolidation, other infiltrate, both consolidation and other infiltrate, normal, or uninterpretable. Two members of a 14-person reading panel, who had undertaken training and standardization in CXR interpretation, interpreted each CXR. Two members of an arbitration panel provided additional independent reviews of CXRs with discordant interpretations at the primary reading, blinded to previous reports. Further discordance was resolved with consensus discussion.A total of 4172 CXRs were obtained from 4232 cases. Observed agreement for detecting consolidation (with or without other infiltrate) between primary readers was 78% (? = 0.50) and between arbitrators was 84% (? = 0.61); agreement for primary readers and arbitrators across 5 conclusion categories was 43.5% (? = 0.25) and 48.5% (? = 0.32), respectively. Disagreement was most frequent between conclusions of other infiltrate and normal for both the reading panel and the arbitration panel (32% and 30% of discordant CXRs, respectively).Agreement was similar to that of previous evaluations using the WHO methodology for detecting consolidation, but poor for other infiltrates despite attempts at a rigorous standardization process.
| S-EPMC5447844 | biostudies-other
Project description:In this study, we implemented a system to classify lung opacities from frontal chest x-ray radiographs. We also proposed a training method to address the class imbalance problem presented in the dataset. We participated in the Radiological Society of America (RSNA) 2018 Pneumonia Detection Challenge and used the datasets provided by the RSNA for further research. Using convolutional neural networks, we implemented a training procedure termed batch control to manipulate the data distribution of positive and negative cases in each training batch. The batch control method regulated and stabilized the performance of the deep-learning models, allowing the adaptive sensitivity of the network models to meet the specific application. The convolutional neural network is practical for classifying lung opacities on chest x-ray radiographs. The batch control method is advantageous for sensitivity regulation and optimization for class-unbalanced datasets.
| S-EPMC9584230 | biostudies-literature
Project description:Purpose: Manual interpretation of chest radiographs is a challenging task and is prone to errors. An automated system capable of categorizing chest radiographs based on the pathologies identified could aid in the timely and efficient diagnosis of chest pathologies. Method: For this retrospective study, 4476 chest radiographs were collected between January and April 2021 from two tertiary care hospitals. Three expert radiologists established the ground truth, and all radiographs were analyzed using a deep-learning AI model to detect suspicious ROIs in the lungs, pleura, and cardiac regions. Three test readers (different from the radiologists who established the ground truth) independently reviewed all radiographs in two sessions (unaided and AI-aided mode) with a washout period of one month. Results: The model demonstrated an aggregate AUROC of 91.2% and a sensitivity of 88.4% in detecting suspicious ROIs in the lungs, pleura, and cardiac regions. These results outperform unaided human readers, who achieved an aggregate AUROC of 84.2% and sensitivity of 74.5% for the same task. When using AI, the aided readers obtained an aggregate AUROC of 87.9% and a sensitivity of 85.1%. The average time taken by the test readers to read a chest radiograph decreased by 21% (p < 0.01) when using AI. Conclusion: The model outperformed all three human readers and demonstrated high AUROC and sensitivity across two independent datasets. When compared to unaided interpretations, AI-aided interpretations were associated with significant improvements in reader performance and chest radiograph interpretation time.
| S-EPMC9914339 | biostudies-literature
Project description:BackgroundAlthough chest radiographs have not been utilised well for classifying stroke subtypes, they could provide a plethora of information on cardioembolic stroke. This study aimed to develop a deep convolutional neural network that could diagnose cardioembolic stroke based on chest radiographs.MethodsOverall, 4,064 chest radiographs of consecutive patients with acute ischaemic stroke were collected from a prospectively maintained stroke registry. Chest radiographs were randomly partitioned into training/validation (n = 3,255) and internal test (n = 809) datasets in an 8:2 ratio. A densely connected convolutional network (ASTRO-X) was trained to diagnose cardioembolic stroke based on chest radiographs. The performance of ASTRO-X was evaluated using the area under the receiver operating characteristic curve. Gradient-weighted class activation mapping was used to evaluate the region of focus of ASTRO-X. External testing was performed with 750 chest radiographs of patients with acute ischaemic stroke from 7 hospitals.FindingsThe areas under the receiver operating characteristic curve of ASTRO-X were 0.86 (95% confidence interval [CI], 0.83-0.89) and 0.82 (95% CI, 0.79-0.85) during the internal and multicentre external testing, respectively. The gradient-weighted class activation map demonstrated that ASTRO-X was focused on the area where the left atrium was located. Compared with cases predicted as non-cardioembolism by ASTRO-X, cases predicted as cardioembolism by ASTRO-X had higher left atrial volume index and lower left ventricular ejection fraction in echocardiography.InterpretationASTRO-X, a deep neural network developed to diagnose cardioembolic stroke based on chest radiographs, demonstrated good classification performance and biological plausibility.FundingGrant No. 14-2020-046 and 08-2016-051 from the Seoul National University Bundang Research Fund and NRF-2020M3E5D9079768 from the National Research Foundation of Korea.
| S-EPMC8264106 | biostudies-literature
Project description:BackgroundThe Society of Thoracic Surgeons Predicted Risk of Mortality (STS-PROM) estimates mortality risk only for certain common procedures (eg, coronary artery bypass or valve surgery) and is cumbersome, requiring greater than 60 inputs. We hypothesized that deep learning can estimate postoperative mortality risk based on a preoperative chest radiograph for cardiac surgeries in which STS-PROM scores were available (STS index procedures) or unavailable (non-STS index procedures).MethodsWe developed a deep learning model (CXR-CTSurgery) to predict postoperative mortality based on preoperative chest radiographs in 9283 patients at Massachusetts General Hospital (MGH) having cardiac surgery before April 8, 2014. CXR-CTSurgery was tested on 3615 different MGH patients and externally tested on 2840 patients from Brigham and Women's Hospital (BWH) having surgery after April 8, 2014. Discrimination for mortality was compared with the STS-PROM using the C-statistic. Calibration was assessed using the observed-to-expected ratio (O/E ratio).ResultsFor STS index procedures, CXR-CTSurgery had a C-statistic similar to STS-PROM at MGH (CXR-CTSurgery: 0.83 vs STS-PROM: 0.88; P = .20) and BWH (0.74 vs 0.80; P = .14) testing cohorts. The CXR-CTSurgery C-statistic for non-STS index procedures was similar to STS index procedures in the MGH (0.87 vs 0.83) and BWH (0.73 vs 0.74) testing cohorts. For STS index procedures, CXR-CTSurgery had better calibration than the STS-PROM in the MGH (O/E ratio: 0.74 vs 0.52) and BWH (O/E ratio: 0.91 vs 0.73) testing cohorts.ConclusionsCXR-CTSurgery predicts postoperative mortality based on a preoperative CXR with similar discrimination and better calibration than the STS-PROM. This may be useful when the STS-PROM cannot be calculated or for non-STS index procedures.
| S-EPMC11373441 | biostudies-literature