Dataset Information

Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study.

ABSTRACT:

Objective

To determine variability in interpretation of chest radiographs among tuberculosis specialists, radiologists, and respiratory specialists.

Design

Observational study.

Setting

Tuberculosis and respiratory disease services, Samara region, Russian Federation.

Participants

101 clinicians involved in the diagnosis and management of pulmonary tuberculosis and respiratory diseases.

Main outcome measures

Interobserver and intraobserver agreement on the interpretation of 50 digital chest radiographs, using a scale of poor to very good agreement (kappa coefficient: < or = 0.20 poor, 0.21-0.40 fair, 0.41-0.60 moderate, 0.61-0.80 good, and 0.81-1.00 very good).

Results

Agreement on the presence or absence of an abnormality was fair only (kappa = 0.380, 95% confidence interval 0.376 to 0.384), moderate for localisation of the abnormality (0.448, 0.444 to 0.452), and fair for a diagnosis of tuberculosis (0.387, 0.382 to 0.391). The highest levels of agreement were among radiologists. Level of experience (years of work in the specialty) influenced agreement on presence of abnormalities and cavities. Levels of intraobserver agreement were fair.

Conclusions

Population screening for tuberculosis in Russia may be less than optimal owing to limited agreement on interpretation of chest radiographs, and may have implications for radiological screening programmes in other countries.

SUBMITTER: Balabanova Y

PROVIDER: S-EPMC1184248 | biostudies-literature | 2005 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study.

Balabanova Y Y Coker R R Fedorin I I Zakharova S S Plavinskij S S Krukov N N Atun R R Drobniewski F F

BMJ (Clinical research ed.) 20050801 7513

<h4>Objective</h4>To determine variability in interpretation of chest radiographs among tuberculosis specialists, radiologists, and respiratory specialists.<h4>Design</h4>Observational study.<h4>Setting</h4>Tuberculosis and respiratory disease services, Samara region, Russian Federation.<h4>Participants</h4>101 clinicians involved in the diagnosis and management of pulmonary tuberculosis and respiratory diseases.<h4>Main outcome measures</h4>Interobserver and intraobserver agreement on the inter ...[more]

PMID: 16096305

Similar Datasets

Project description:PurposeTo study the extent of pulmonary involvement in coronavirus 19 (COVID-19) with quantitative CT and to assess the impact of disease burden on opacity visibility on chest radiographs.Materials and methodsThis retrospective study included 20 pairs of CT scans and same-day chest radiographs from 17 patients with COVID-19, along with 20 chest radiographs of controls. All pulmonary opacities were semiautomatically segmented on CT images, producing an anteroposterior projection image to match the corresponding frontal chest radiograph. The quantitative CT lung opacification mass (QCTmass) was defined as (opacity attenuation value + 1000 HU)/1000 × 1.065 (g/mL) × combined volume (cm3) of the individual opacities. Eight thoracic radiologists reviewed the 40 radiographs, and a receiver operating characteristic curve analysis was performed for the detection of lung opacities. Logistic regression analysis was performed to identify factors affecting opacity visibility on chest radiographs.ResultsThe mean QCTmass per patient was 72.4 g ± 120.8 (range, 0.7-420.7 g), and opacities occupied 3.2% ± 5.8 (range, 0.1%-19.8%) and 13.9% ± 18.0 (range, 0.5%-57.8%) of the lung area on the CT images and projected images, respectively. The radiographs had a median sensitivity of 25% and specificity of 90% among radiologists. Nineteen of 186 opacities were visible on chest radiographs, and a median area of 55.8% of the projected images was identifiable on radiographs. Logistic regression analysis showed that QCTmass (P < .001) and combined opacity volume (P < .001) significantly affected opacity visibility on radiographs.ConclusionQCTmass varied among patients with COVID-19. Chest radiographs had high specificity for detecting lung opacities in COVID-19 but a low sensitivity. QCTmass and combined opacity volume were significant determinants of opacity visibility on radiographs.Earlier incorrect version appeared online. This article was corrected on April 6, 2020 and December 14, 2020.Supplemental material is available for this article.© RSNA, 2020.

Project description:Data-driven deep learning (DL) methods using convolutional neural networks (CNNs) demonstrate promising performance in natural image computer vision tasks. However, their use in medical computer vision tasks faces several limitations, viz., (i) adapting to visual characteristics that are unlike natural images; (ii) modeling random noise during training due to stochastic optimization and backpropagation-based learning strategy; (iii) challenges in explaining DL black-box behavior to support clinical decision-making; and (iv) inter-reader variability in the ground truth (GT) annotations affecting learning and evaluation. This study proposes a systematic approach to address these limitations through application to the pandemic-caused need for Coronavirus disease 2019 (COVID-19) detection using chest X-rays (CXRs). Specifically, our contribution highlights significant benefits obtained through (i) pretraining specific to CXRs in transferring and fine-tuning the learned knowledge toward improving COVID-19 detection performance; (ii) using ensembles of the fine-tuned models to further improve performance over individual constituent models; (iii) performing statistical analyses at various learning stages for validating results; (iv) interpreting learned individual and ensemble model behavior through class-selective relevance mapping (CRM)-based region of interest (ROI) localization; and, (v) analyzing inter-reader variability and ensemble localization performance using Simultaneous Truth and Performance Level Estimation (STAPLE) methods. We find that ensemble approaches markedly improved classification and localization performance, and that inter-reader variability and performance level assessment helps guide algorithm design and parameter optimization. To the best of our knowledge, this is the first study to construct ensembles, perform ensemble-based disease ROI localization, and analyze inter-reader variability and algorithm performance for COVID-19 detection in CXRs.

Project description:BackgroundAdult critical care and radiographical societies have recommended changing practice from routine screening radiographs to on-demand chest radiographs (CXRs) for stable mechanically ventilated adult patients. There are no similar recommendations for patients in the pediatric ICU. Reducing the frequency with which unneeded CXRs are obtained can decrease radiation exposure and reduce waste, a substantial contributor to rising health care costs. We aimed to reduce unneeded daily screening CXRs in a pediatric cardiovascular ICU (CICU) by 20% in 6 months.MethodsCriteria delineating which subjects in the CICU required daily screening CXRs were created and added to the daily rounding sheet for discussion for each subject. The primary goal of this study was to reduce CXRs in mechanically ventilated subjects as our previous practice had been to order daily CXRs. Respiratory therapists increased the frequency of evaluating and documenting endotracheal tube positioning prior to the initiation of this project. The outcome measure was the percentage subjects who received a daily screening CXR. The ratio of daily screening CXRs to the number of total CXRs ordered and unplanned extubations were followed as balancing measures.ResultsThe number of subjects who received a daily screening CXR decreased from a baseline of 67% to 44% over the course of this project. There was no change in the ratio of daily screening CXRs to the number of total CXRs ordered or an increase in unplanned extubations. With an estimated cost of $268 per CXR, a reduction of 33% in routine screening CXRs saves an estimated $250,000 annually.ConclusionsA decrease in daily screening CXRs can be sustained through the development of specific criteria to determine which patients need screening radiographs. This can be achieved without an increase in CXRs obtained at other times throughout the day or an increase in unplanned extubations. This eliminates unneeded health care expenditures, improves resource allocation for radiology technicians, and decreases disruptive interventions for patients.

Project description:ObjectiveThis study aimed to investigate the feasibility of using artificial intelligence (AI) to identify normal chest radiography (CXR) from the worklist of radiologists in a health-screening environment.Materials and methodsThis retrospective simulation study was conducted using the CXRs of 5887 adults (mean age ± standard deviation, 55.4 ± 11.8 years; male, 4329) from three health screening centers in South Korea using a commercial AI (Lunit INSIGHT CXR3, version 3.5.8.8). Three board-certified thoracic radiologists reviewed CXR images for referable thoracic abnormalities and grouped the images into those with visible referable abnormalities (identified as abnormal by at least one reader) and those with clearly visible referable abnormalities (identified as abnormal by at least two readers). With AI-based simulated exclusion of normal CXR images, the percentages of normal images sorted and abnormal images erroneously removed were analyzed. Additionally, in a random subsample of 480 patients, the ability to identify visible referable abnormalities was compared among AI-unassisted reading (i.e., all images read by human readers without AI), AI-assisted reading (i.e., all images read by human readers with AI assistance as concurrent readers), and reading with AI triage (i.e., human reading of only those rendered abnormal by AI).ResultsOf 5887 CXR images, 405 (6.9%) and 227 (3.9%) contained visible and clearly visible abnormalities, respectively. With AI-based triage, 42.9% (2354/5482) of normal CXR images were removed at the cost of erroneous removal of 3.5% (14/405) and 1.8% (4/227) of CXR images with visible and clearly visible abnormalities, respectively. In the diagnostic performance study, AI triage removed 41.6% (188/452) of normal images from the worklist without missing visible abnormalities and increased the specificity for some readers without decreasing sensitivity.ConclusionThis study suggests the feasibility of sorting and removing normal CXRs using AI with a tailored cut-off to increase efficiency and reduce the workload of radiologists.

Project description:The recent medical applications of deep-learning (DL) algorithms have demonstrated their clinical efficacy in improving speed and accuracy of image interpretation. If the DL algorithm achieves a performance equivalent to that achieved by physicians in chest radiography (CR) diagnoses with Coronavirus disease 2019 (COVID-19) pneumonia, the automatic interpretation of the CR with DL algorithms can significantly reduce the burden on clinicians and radiologists in sudden surges of suspected COVID-19 patients. The aim of this study was to evaluate the efficacy of the DL algorithm for detecting COVID-19 pneumonia on CR compared with formal radiology reports. This is a retrospective study of adult patients that were diagnosed as positive COVID-19 cases based on the reverse transcription polymerase chain reaction among all the patients who were admitted to five emergency departments and one community treatment center in Korea from February 18, 2020 to May 1, 2020. The CR images were evaluated with a publicly available DL algorithm. For reference, CR images without chest computed tomography (CT) scans classified as positive for COVID-19 pneumonia were used given that the radiologist identified ground-glass opacity, consolidation, or other infiltration in retrospectively reviewed CR images. Patients with evidence of pneumonia on chest CT scans were also classified as COVID-19 pneumonia positive outcomes. The overall sensitivity and specificity of the DL algorithm for detecting COVID-19 pneumonia on CR were 95.6%, and 88.7%, respectively. The area under the curve value of the DL algorithm for the detection of COVID-19 with pneumonia was 0.921. The DL algorithm demonstrated a satisfactory diagnostic performance comparable with that of formal radiology reports in the CR-based diagnosis of pneumonia in COVID-19 patients. The DL algorithm may offer fast and reliable examinations that can facilitate patient screening and isolation decisions, which can reduce the medical staff workload during COVID-19 pandemic situations.

Dataset Information

Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study.

Objective

Design

Setting

Participants

Main outcome measures

Results

Conclusions

Publications

Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets