Dataset Information

Classification of hyper-scale multimodal imaging datasets.

ABSTRACT: Algorithms that classify hyper-scale multi-modal datasets, comprising of millions of images, into constituent modality types can help researchers quickly retrieve and classify diagnostic imaging data, accelerating clinical outcomes. This research aims to demonstrate that a deep neural network that is trained on a hyper-scale dataset (4.5 million images) composed of heterogeneous multi-modal data can be used to obtain significant modality classification accuracy (96%). By combining 102 medical imaging datasets, a dataset of 4.5 million images was created. A ResNet-50, ResNet-18, and VGG16 were trained to classify these images by the imaging modality used to capture them (Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and X-ray) across many body locations. The classification accuracy of the models was then tested on unseen data. The best performing model achieved classification accuracy of 96% on unseen data, which is on-par, or exceeds the accuracy of more complex implementations using EfficientNets or Vision Transformers (ViTs). The model achieved a balanced accuracy of 86%. This research shows it is possible to train Deep Learning (DL) Convolutional Neural Networks (CNNs) with hyper-scale multimodal datasets, composed of millions of images. Such models can find use in real-world applications with volumes of image data in the hyper-scale range, such as medical imaging repositories, or national healthcare institutions. Further research can expand this classification capability to include 3D-scans.

SUBMITTER: Macfadyen C

PROVIDER: S-EPMC10718410 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Classification of hyper-scale multimodal imaging datasets.

Macfadyen Craig C Duraiswamy Ajay A Harris-Birtill David D

PLOS digital health 20231213 12

Algorithms that classify hyper-scale multi-modal datasets, comprising of millions of images, into constituent modality types can help researchers quickly retrieve and classify diagnostic imaging data, accelerating clinical outcomes. This research aims to demonstrate that a deep neural network that is trained on a hyper-scale dataset (4.5 million images) composed of heterogeneous multi-modal data can be used to obtain significant modality classification accuracy (96%). By combining 102 medical im ...[more]

PMID: 38091333

Similar Datasets

Project description:PurposeTo develop a severity classification for macular telangiectasia type 2 (MacTel) disease using multimodal imaging.DesignAn algorithm was used on data from a prospective natural history study of MacTel for classification development.SubjectsA total of 1733 participants enrolled in an international natural history study of MacTel.MethodsThe Classification and Regression Trees (CART), a predictive nonparametric algorithm used in machine learning, analyzed the features of the multimodal imaging important for the development of a classification, including reading center gradings of the following digital images: stereoscopic color and red-free fundus photographs, fluorescein angiographic images, fundus autofluorescence images, and spectral-domain (SD)-OCT images. Regression models that used least square method created a decision tree using features of the ocular images into different categories of disease severity.Main outcome measuresThe primary target of interest for the algorithm development by CART was the change in best-corrected visual acuity (BCVA) at baseline for the right and left eyes. These analyses using the algorithm were repeated for the BCVA obtained at the last study visit of the natural history study for the right and left eyes.ResultsThe CART analyses demonstrated 3 important features from the multimodal imaging for the classification: OCT hyper-reflectivity, pigment, and ellipsoid zone loss. By combining these 3 features (as absent, present, noncentral involvement, and central involvement of the macula), a 7-step scale was created, ranging from excellent to poor visual acuity. At grade 0, 3 features are not present. At the most severe grade, pigment and exudative neovascularization are present. To further validate the classification, using the Generalized Estimating Equation regression models, analyses for the annual relative risk of progression over a period of 5 years for vision loss and for progression along the scale were performed.ConclusionsThis analysis using the data from current imaging modalities in participants followed in the MacTel natural history study informed a classification for MacTel disease severity featuring variables from SD-OCT. This classification is designed to provide better communications to other clinicians, researchers, and patients.Financial disclosuresProprietary or commercial disclosure may be found after the references.

Dataset Information

Classification of hyper-scale multimodal imaging datasets.

Publications

Classification of hyper-scale multimodal imaging datasets.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets