Dataset Information

The effect of a tuberculosis chest X-ray image reference set on non-expert reader performance.

ABSTRACT: In low-resource settings, limitations in diagnostic accuracy of chest X-rays (CXR) for pulmonary tuberculosis (PTB) relate partly to non-expert interpretation. We piloted a TB CXR Image Reference Set (TIRS) to improve non-expert performance in an operational setting in Malawi.Nineteen doctors and clinical officers read 60 CXR of patients with suspected PTB, at baseline and using TIRS. Two officers also used the CXR Reading and Recording System (CRRS). Correct treatment decisions were assessed against a "gold standard" of mycobacterial culture and expert performance.TIRS significantly increased overall non-expert sensitivity from 67.6 (SD 14.9) to 75.5 (SD 11.1, P?=?0.013), approaching expert values of 84.2 (SD 5.2). Among doctors, correct decisions increased from 60.7 % (SD 7.9) to 67.1 % (SD 8.0, P?=?0.054). Clinical officers increased in sensitivity from 68.0 % (SD 15) to 77.4 % (SD 10.7, P?=?0.056), but decreased in specificity from 55.0 % (SD 23.9) to 40.8 % (SD 10.4, P?=?0.049). Two officers made correct treatment decisions with TIRS in 62.7 %. CRRS training increased this to 67.8 %.Use of a CXR image reference set increased correct decisions by doctors to treat PTB. This tool may provide a low-cost intervention improving non-expert performance, translating into improved clinical care. Further evaluation is warranted.• Tuberculosis treatment decisions are influenced by CXR findings, despite improved laboratory diagnostics. • In low-resource settings, CXR interpretation is performed largely by non-experts. • We piloted the effect of a simple reference training set of CXRs. • Use of the reference set increased the number of correct treatment decisions. This effect was more marked for doctors than clinical officers. • Further evaluation of this simple training tool is warranted.

SUBMITTER: Waitt CJ

PROVIDER: S-EPMC3738845 | biostudies-other | 2013 Sep

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

The effect of a tuberculosis chest X-ray image reference set on non-expert reader performance.

Waitt Catriona J CJ Joekes Elizabeth C EC Jesudason Natasha N Waitt Peter I PI Goodson Patrick P Likumbo Ganizani G Kampondeni Samuel S Faragher E Brian EB Squire S Bertel SB

European radiology 20130508 9

<h4>Objectives</h4>In low-resource settings, limitations in diagnostic accuracy of chest X-rays (CXR) for pulmonary tuberculosis (PTB) relate partly to non-expert interpretation. We piloted a TB CXR Image Reference Set (TIRS) to improve non-expert performance in an operational setting in Malawi.<h4>Methods</h4>Nineteen doctors and clinical officers read 60 CXR of patients with suspected PTB, at baseline and using TIRS. Two officers also used the CXR Reading and Recording System (CRRS). Correct t ...[more]

PMID: 23652843

Similar Datasets

Project description:ImportanceThe efficient and accurate interpretation of radiologic images is paramount.ObjectiveTo evaluate whether a deep learning-based artificial intelligence (AI) engine used concurrently can improve reader performance and efficiency in interpreting chest radiograph abnormalities.Design, setting, and participantsThis multicenter cohort study was conducted from April to November 2021 and involved radiologists, including attending radiologists, thoracic radiology fellows, and residents, who independently participated in 2 observer performance test sessions. The sessions included a reading session with AI and a session without AI, in a randomized crossover manner with a 4-week washout period in between. The AI produced a heat map and the image-level probability of the presence of the referrable lesion. The data used were collected at 2 quaternary academic hospitals in Boston, Massachusetts: Beth Israel Deaconess Medical Center (The Medical Information Mart for Intensive Care Chest X-Ray [MIMIC-CXR]) and Massachusetts General Hospital (MGH).Main outcomes and measuresThe ground truths for the labels were created via consensual reading by 2 thoracic radiologists. Each reader documented their findings in a customized report template, in which the 4 target chest radiograph findings and the reader confidence of the presence of each finding was recorded. The time taken for reporting each chest radiograph was also recorded. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated for each target finding.ResultsA total of 6 radiologists (2 attending radiologists, 2 thoracic radiology fellows, and 2 residents) participated in the study. The study involved a total of 497 frontal chest radiographs-247 from the MIMIC-CXR data set (demographic data for patients were not available) and 250 chest radiographs from MGH (mean [SD] age, 63 [16] years; 133 men [53.2%])-from adult patients with and without 4 target findings (pneumonia, nodule, pneumothorax, and pleural effusion). The target findings were found in 351 of 497 chest radiographs. The AI was associated with higher sensitivity for all findings compared with the readers (nodule, 0.816 [95% CI, 0.732-0.882] vs 0.567 [95% CI, 0.524-0.611]; pneumonia, 0.887 [95% CI, 0.834-0.928] vs 0.673 [95% CI, 0.632-0.714]; pleural effusion, 0.872 [95% CI, 0.808-0.921] vs 0.889 [95% CI, 0.862-0.917]; pneumothorax, 0.988 [95% CI, 0.932-1.000] vs 0.792 [95% CI, 0.756-0.827]). AI-aided interpretation was associated with significantly improved reader sensitivities for all target findings, without negative impacts on the specificity. Overall, the AUROCs of readers improved for all 4 target findings, with significant improvements in detection of pneumothorax and nodule. The reporting time with AI was 10% lower than without AI (40.8 vs 36.9 seconds; difference, 3.9 seconds; 95% CI, 2.9-5.2 seconds; P < .001).Conclusions and relevanceThese findings suggest that AI-aided interpretation was associated with improved reader performance and efficiency for identifying major thoracic findings on a chest radiograph.

Project description:ContextMedical image perception training generally focuses on abnormalities, whereas normal images are more prevalent in medical practice. Furthermore, instructional sequences that let students practice prior to expert instruction (inductive) may lead to improved performance compared with methods that give students expert instruction before practice (deductive). This study investigates the effects of the proportion of normal images and practice-instruction order on learning to interpret medical images. It is hypothesised that manipulation of the proportion of normal images will lead to a sensitivity-specificity trade-off and that students in practice-first (inductive) conditons need more time per practice case but will correctly identify more test cases.MethodsThird-year medical students (n = 103) learned radiograph interpretation by practising cases with, respectively, 30% or 70% normal radiographs prior to expert instruction (practice-first order) or after expert instruction (instruction-first order). After training, students performed a test (60% normal) and sensitivity (% of correctly identified abnormal radiographs), specificity (% of correctly identified normal radiographs), diagnostic performance (% of correct diagnoses) and case duration were measured.ResultsThe conditions with 30% of normal images scored higher on sensitivity but the conditions with 70% of normal images scored higher on specificity, indicating a sensitivity and specificity trade-off. Those who participated in inductive conditions took less time per practice case but more per test case. They had similar test sensitivity, but scored lower on test specificity.ConclusionsThe proportion of normal images impacted the sensitivity-specificity trade-off. This trade-off should be an important consideration for the alignment of training with future practice. Furthermore, the deductive conditions unexpectedly scored higher on specificity when participants took less time per case. An inductive approach did not lead to higher diagnostic performance, possibly because participants might already have relevant prior knowledge. Deductive approaches are therefore advised for the training of advanced learners.

Dataset Information

The effect of a tuberculosis chest X-ray image reference set on non-expert reader performance.

Publications

The effect of a tuberculosis chest X-ray image reference set on non-expert reader performance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets