Dataset Information

Captioning Ultrasound Images Automatically.

ABSTRACT: We describe an automatic natural language processing (NLP)-based image captioning method to describe fetal ultrasound video content by modelling the vocabulary commonly used by sonographers and sonologists. The generated captions are similar to the words spoken by a sonographer when describing the scan experience in terms of visual content and performed scanning actions. Using full-length second-trimester fetal ultrasound videos and text derived from accompanying expert voice-over audio recordings, we train deep learning models consisting of convolutional neural networks and recurrent neural networks in merged configurations to generate captions for ultrasound video frames. We evaluate different model architectures using established general metrics (BLEU, ROUGE-L) and application-specific metrics. Results show that the proposed models can learn joint representations of image and text to generate relevant and descriptive captions for anatomies, such as the spine, the abdomen, the heart, and the head, in clinical fetal ultrasound scans.

SUBMITTER: Alsharid M

PROVIDER: S-EPMC6978141 | biostudies-literature | 2019 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Captioning Ultrasound Images Automatically.

Alsharid Mohammad M Sharma Harshita H Drukker Lior L Chatelain Pierre P Papageorghiou Aris T AT Noble J Alison JA

Medical image computing and computer-assisted intervention : MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention 20191010

We describe an automatic natural language processing (NLP)-based image captioning method to describe fetal ultrasound video content by modelling the vocabulary commonly used by sonographers and sonologists. The generated captions are similar to the words spoken by a sonographer when describing the scan experience in terms of visual content and performed scanning actions. Using full-length second-trimester fetal ultrasound videos and text derived from accompanying expert voice-over audio recordin ...[more]

PMID: 31976493

Similar Datasets

Project description:BackgroundThis study aimed to develop an automated method to measure the gray-white matter ratio (GWR) from brain computed tomography (CT) scans of patients with out-of-hospital cardiac arrest (OHCA) and assess its significance in predicting early-stage neurological outcomes.MethodsPatients with OHCA who underwent brain CT imaging within 12 h of return of spontaneous circulation were enrolled in this retrospective study. The primary outcome endpoint measure was a favorable neurological outcome, defined as cerebral performance category 1 or 2 at hospital discharge. We proposed an automated method comprising image registration, K-means segmentation, segmentation refinement, and GWR calculation to measure the GWR for each CT scan. The K-means segmentation and segmentation refinement was employed to refine the segmentations within regions of interest (ROIs), consequently enhancing GWR calculation accuracy through more precise segmentations.ResultsOverall, 443 patients were divided into derivation N=265, 60% and validation N=178, 40% sets, based on age and sex. The ROI Hounsfield unit values derived from the automated method showed a strong correlation with those obtained from the manual method. Regarding outcome prediction, the automated method significantly outperformed the manual method in GWR calculation (AUC 0.79 vs. 0.70) across the entire dataset. The automated method also demonstrated superior performance across sensitivity, specificity, and positive and negative predictive values using the cutoff value determined from the derivation set. Moreover, GWR was an independent predictor of outcomes in logistic regression analysis. Incorporating the GWR with other clinical and resuscitation variables significantly enhanced the performance of prediction models compared to those without the GWR.ConclusionsAutomated measurement of the GWR from non-contrast brain CT images offers valuable insights for predicting neurological outcomes during the early post-cardiac arrest period.

Dataset Information

Captioning Ultrasound Images Automatically.

Publications

Captioning Ultrasound Images Automatically.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets