Dataset Information

Multi-Reader-Multi-Split Annotation of Emphysema in Computed Tomography.

ABSTRACT: Emphysema is visible on computed tomography (CT) as low-density lesions representing the destruction of the pulmonary alveoli. To train a machine learning model on the emphysema extent in CT images, labeled image data is needed. The provision of these labels requires trained readers, who are a limited resource. The purpose of the study was to test the reading time, inter-observer reliability and validity of the multi-reader-multi-split method for acquiring CT image labels from radiologists. The approximately 500 slices of each stack of lung CT images were split into 1-cm chunks, with 17 thin axial slices per chunk. The chunks were randomly distributed to 26 readers, radiologists and radiology residents. Each chunk was given a quick score concerning emphysema type and severity in the left and right lung separately. A cohort of 102 subjects, with varying degrees of visible emphysema in the lung CT images, was selected from the SCAPIS pilot, performed in 2012 in Gothenburg, Sweden. In total, the readers created 9050 labels for 2881 chunks. Image labels were compared with regional annotations already provided at the SCAPIS pilot inclusion. The median reading time per chunk was 15 s. The inter-observer Krippendorff's alpha was 0.40 and 0.53 for emphysema type and score, respectively, and higher in the apical part than in the basal part of the lungs. The multi-split emphysema scores were generally consistent with regional annotations. In conclusion, the multi-reader-multi-split method provided reasonably valid image labels, with an estimation of the inter-observer reliability.

SUBMITTER: Liden M

PROVIDER: S-EPMC7572947 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multi-Reader-Multi-Split Annotation of Emphysema in Computed Tomography.

Lidén Mats M Hjelmgren Ola O Vikgren Jenny J Thunberg Per P

Journal of digital imaging 20200810 5

Emphysema is visible on computed tomography (CT) as low-density lesions representing the destruction of the pulmonary alveoli. To train a machine learning model on the emphysema extent in CT images, labeled image data is needed. The provision of these labels requires trained readers, who are a limited resource. The purpose of the study was to test the reading time, inter-observer reliability and validity of the multi-reader-multi-split method for acquiring CT image labels from radiologists. The ...[more]

PMID: 32779016

Similar Datasets

Project description:BACKGROUND:Pulmonary emphysema is divided into 3 major subtypes at autopsy: centrilobular, paraseptal, and panlobular emphysema. These subtypes can be defined by visual assessment on computed tomography (CT); however, clinical characteristics of emphysema subtypes on CT are not well defined. We developed a reliable approach to visual assessment of emphysema subtypes on CT and examined if emphysema subtypes have distinct characteristics. METHODS:The Multi-Ethnic Study of Atherosclerosis COPD Study recruited smokers with chronic obstructive pulmonary disease (COPD) and controls ages 50-79 years with ? 10 pack-years. Participants underwent CT following a standardized protocol. Definitions of centrilobular, paraseptal, and panlobular emphysema were obtained by literature review. Six-minute walk distance and pulmonary function were performed following guidelines. RESULTS:Twenty-seven percent of 318 smokers had emphysema on CT. Interrater reliability of emphysema subtype was substantial (K: 0.70). Compared with participants without emphysema, individuals with centrilobular or panlobular emphysema had greater dyspnea, reduced walk distance, greater hyperinflation, and lower diffusing capacity. In contrast, individuals with paraseptal emphysema were similar to controls, except for male predominance. Centrilobular, but not panlobular or paraseptal, emphysema was associated with greater smoking history (+21 pack-years P <.001). Panlobular, but not other types of emphysema, was associated with reduced body mass index (-5 kg/m(2); P = .01). Other than for dyspnea, these findings were independent of the forced expiratory volume in 1 second. Seventeen percent of smokers without COPD on spirometry had emphysema, which was independently associated with reduced walk distance. CONCLUSIONS:Emphysema subtypes on CT are common in smokers with and without COPD. Centrilobular and panlobular emphysema, but not paraseptal emphysema, have considerable symptomatic and physiological consequences.

Project description:BackgroundThere is progress to be made in building artificially intelligent systems to detect abnormalities that are not only accurate but can handle the true breadth of findings that radiologists encounter in body (chest, abdomen, and pelvis) computed tomography (CT). Currently, the major bottleneck for developing multi-disease classifiers is a lack of manually annotated data. The purpose of this work was to develop high throughput multi-label annotators for body CT reports that can be applied across a variety of abnormalities, organs, and disease states thereby mitigating the need for human annotation.MethodsWe used a dictionary approach to develop rule-based algorithms (RBA) for extraction of disease labels from radiology text reports. We targeted three organ systems (lungs/pleura, liver/gallbladder, kidneys/ureters) with four diseases per system based on their prevalence in our dataset. To expand the algorithms beyond pre-defined keywords, attention-guided recurrent neural networks (RNN) were trained using the RBA-extracted labels to classify reports as being positive for one or more diseases or normal for each organ system. Alternative effects on disease classification performance were evaluated using random initialization or pre-trained embedding as well as different sizes of training datasets. The RBA was tested on a subset of 2158 manually labeled reports and performance was reported as accuracy and F-score. The RNN was tested against a test set of 48,758 reports labeled by RBA and performance was reported as area under the receiver operating characteristic curve (AUC), with 95% CIs calculated using the DeLong method.ResultsManual validation of the RBA confirmed 91-99% accuracy across the 15 different labels. Our models extracted disease labels from 261,229 radiology reports of 112,501 unique subjects. Pre-trained models outperformed random initialization across all diseases. As the training dataset size was reduced, performance was robust except for a few diseases with a relatively small number of cases. Pre-trained classification AUCs reached > 0.95 for all four disease outcomes and normality across all three organ systems.ConclusionsOur label-extracting pipeline was able to encompass a variety of cases and diseases in body CT reports by generalizing beyond strict rules with exceptional accuracy. The method described can be easily adapted to enable automated labeling of hospital-scale medical data sets for training image-based disease classifiers.

Project description:BackgroundIn vivo high-resolution micro-computed tomography allows for longitudinal image-based measurements in animal models of lung disease. The combination of repetitive high resolution imaging with fully automated quantitative image analysis in mouse models of lung fibrosis lung benefits preclinical research. This study aimed to develop and validate such an automated micro-computed tomography analysis algorithm for quantification of aerated lung volume in mice; an indicator of pulmonary fibrosis and emphysema severity.MethodologyMice received an intratracheal instillation of bleomycin (n = 8), elastase (0.25 U elastase n = 9, 0.5 U elastase n = 8) or saline control (n = 6 for fibrosis, n = 5 for emphysema). A subset of mice was scanned without intervention, to evaluate potential radiation-induced toxicity (n = 4). Some bleomycin-instilled mice were treated with imatinib for proof of concept (n = 8). Mice were scanned weekly, until four weeks after induction, when they underwent pulmonary function testing, lung histology and collagen quantification. Aerated lung volumes were calculated with our automated algorithm.Principal findingsOur automated image-based aerated lung volume quantification method is reproducible with low intra-subject variability. Bleomycin-treated mice had significantly lower scan-derived aerated lung volumes, compared to controls. Aerated lung volume correlated with the histopathological fibrosis score and total lung collagen content. Inversely, a dose-dependent increase in lung volume was observed in elastase-treated mice. Serial scanning of individual mice is feasible and visualized dynamic disease progression. No radiation-induced toxicity was observed. Three-dimensional images provided critical topographical information.ConclusionsWe report on a high resolution in vivo micro-computed tomography image analysis algorithm that runs fully automated and allows quantification of aerated lung volume in mice. This method is reproducible with low inherent measurement variability. We show that it is a reliable quantitative tool to investigate experimental lung fibrosis and emphysema in mice. Its non-invasive nature has the unique benefit to allow dynamic 4D evaluation of disease processes and therapeutic interventions.

Dataset Information

Multi-Reader-Multi-Split Annotation of Emphysema in Computed Tomography.

Publications

Multi-Reader-Multi-Split Annotation of Emphysema in Computed Tomography.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets