An evaluation of performance measures for arterial brain vessel segmentation.
Ontology highlight
ABSTRACT: Background
Arterial brain vessel segmentation allows utilising clinically relevant information contained within the cerebral vascular tree. Currently, however, no standardised performance measure is available to evaluate the quality of cerebral vessel segmentations. Thus, we developed a performance measure selection framework based on manual visual scoring of simulated segmentation variations to find the most suitable measure for cerebral vessel segmentation.Methods
To simulate segmentation variations, we manually created non-overlapping segmentation errors common in magnetic resonance angiography cerebral vessel segmentation. In 10 patients, we generated a set of approximately 300 simulated segmentation variations for each ground truth image. Each segmentation was visually scored based on a predefined scoring system and segmentations were ranked based on 22 performance measures common in the literature. The correlation of visual scores with performance measure rankings was calculated using the Spearman correlation coefficient.Results
The distance-based performance measures balanced average Hausdorff distance (rank = 1) and average Hausdorff distance (rank = 2) provided the segmentation rankings with the highest average correlation with manual rankings. They were followed by overlap-based measures such as Dice coefficient (rank = 7), a standard performance measure in medical image segmentation.Conclusions
Average Hausdorff distance-based measures should be used as a standard performance measure in evaluating cerebral vessel segmentation quality. They can identify more relevant segmentation errors, especially in high-quality segmentations. Our findings have the potential to accelerate the validation and development of novel vessel segmentation approaches.
SUBMITTER: Aydin OU
PROVIDER: S-EPMC8283850 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Introduction: Arterial brain vessel assessment is crucial for the diagnostic process in patients with cerebrovascular disease. Non-invasive neuroimaging techniques, such as time-of-flight (TOF) magnetic resonance angiography (MRA) imaging are applied in the clinical routine to depict arteries. They are, however, only visually assessed. Fully automated vessel segmentation integrated into the clinical routine could facilitate the time-critical diagnosis of vessel abnormalities and might facilitate the identification of valuable biomarkers for cerebrovascular events. In the present work, we developed and validated a new deep learning model for vessel segmentation, coined BRAVE-NET, on a large aggregated dataset of patients with cerebrovascular diseases. Methods: BRAVE-NET is a multiscale 3-D convolutional neural network (CNN) model developed on a dataset of 264 patients from three different studies enrolling patients with cerebrovascular diseases. A context path, dually capturing high- and low-resolution volumes, and deep supervision were implemented. The BRAVE-NET model was compared to a baseline Unet model and variants with only context paths and deep supervision, respectively. The models were developed and validated using high-quality manual labels as ground truth. Next to precision and recall, the performance was assessed quantitatively by Dice coefficient (DSC); average Hausdorff distance (AVD); 95-percentile Hausdorff distance (95HD); and via visual qualitative rating. Results: The BRAVE-NET performance surpassed the other models for arterial brain vessel segmentation with a DSC = 0.931, AVD = 0.165, and 95HD = 29.153. The BRAVE-NET model was also the most resistant toward false labelings as revealed by the visual analysis. The performance improvement is primarily attributed to the integration of the multiscaling context path into the 3-D Unet and to a lesser extent to the deep supervision architectural component. Discussion: We present a new state-of-the-art of arterial brain vessel segmentation tailored to cerebrovascular pathology. We provide an extensive experimental validation of the model using a large aggregated dataset encompassing a large variability of cerebrovascular disease and an external set of healthy volunteers. The framework provides the technological foundation for improving the clinical workflow and can serve as a biomarker extraction tool in cerebrovascular diseases.
| S-EPMC7861225 | biostudies-literature
Project description:Fingerprint recognition plays an important role in many commercial applications and is used by millions of people every day, e.g. for unlocking mobile phones. Fingerprint image segmentation is typically the first processing step of most fingerprint algorithms and it divides an image into foreground, the region of interest, and background. Two types of error can occur during this step which both have a negative impact on the recognition performance: 'true' foreground can be labeled as background and features like minutiae can be lost, or conversely 'true' background can be misclassified as foreground and spurious features can be introduced. The contribution of this paper is threefold: firstly, we propose a novel factorized directional bandpass (FDB) segmentation method for texture extraction based on the directional Hilbert transform of a Butterworth bandpass (DHBB) filter interwoven with soft-thresholding. Secondly, we provide a manually marked ground truth segmentation for 10560 images as an evaluation benchmark. Thirdly, we conduct a systematic performance comparison between the FDB method and four of the most often cited fingerprint segmentation algorithms showing that the FDB segmentation method clearly outperforms these four widely used methods. The benchmark and the implementation of the FDB method are made publicly available.
| S-EPMC4865205 | biostudies-literature
Project description:Image segmentation of retinal blood vessels is a process that can help to predict and diagnose cardiovascular related diseases, such as hypertension and diabetes, which are known to affect the retinal blood vessels' appearance. This work proposes an unsupervised method for the segmentation of retinal vessels images using a combined matched filter, Frangi's filter and Gabor Wavelet filter to enhance the images. The combination of these three filters in order to improve the segmentation is the main motivation of this work. We investigate two approaches to perform the filter combination: weighted mean and median ranking. Segmentation methods are tested after the vessel enhancement. Enhanced images with median ranking are segmented using a simple threshold criterion. Two segmentation procedures are applied when considering enhanced retinal images using the weighted mean approach. The first method is based on deformable models and the second uses fuzzy C-means for the image segmentation. The procedure is evaluated using two public image databases, Drive and Stare. The experimental results demonstrate that the proposed methods perform well for vessel segmentation in comparison with state-of-the-art methods.
| S-EPMC4769136 | biostudies-literature
Project description:We compare three widely used brain volumetry methods available in the software packages FSL, SPM5, and FreeSurfer and evaluate their performance using simulated and real MR brain data sets. We analyze the accuracy of gray and white matter volume measurements and their robustness against changes of image quality using the BrainWeb MRI database. These images are based on "gold-standard" reference brain templates. This allows us to assess between- (same data set, different method) and also within-segmenter (same method, variation of image quality) comparability, for both of which we find pronounced variations in segmentation results for gray and white matter volumes. The calculated volumes deviate up to >10% from the reference values for gray and white matter depending on method and image quality. Sensitivity is best for SPM5, volumetric accuracy for gray and white matter was similar in SPM5 and FSL and better than in FreeSurfer. FSL showed the highest stability for white (<5%), FreeSurfer (6.2%) for gray matter for constant image quality BrainWeb data. Between-segmenter comparisons show discrepancies of up to >20% for the simulated data and 24% on average for the real data sets, whereas within-method performance analysis uncovered volume differences of up to >15%. Since the discrepancies between results reach the same order of magnitude as volume changes observed in disease, these effects limit the usability of the segmentation methods for following volume changes in individual patients over time and should be taken into account during the planning and analysis of brain volume studies.
| S-EPMC6870639 | biostudies-literature
Project description:Background and Purpose: Intracranial arterial calcification (IAC) has been the focus of much attention by clinicians and researchers as an indicator of intracranial atherosclerosis, but correlations of IAC patterns (intimal or medial) with the presence of atherosclerotic plaques and plaque stability are still a matter of debate. Our study aimed to assess the associations of IAC patterns identified on computed tomography (CT) with the presence of plaque detected on vessel wall magnetic resonance imaging and plaque stability. Materials and Methods: Patients with stroke or transient ischemic attack and intracranial artery stenosis were recruited. IAC was detected and localized (intima or media) on non-contrast CT images. Intracranial atherosclerotic plaques were identified using vessel wall magnetic resonance imaging and matched to corresponding CT images. Associations between IAC patterns and culprit atherosclerotic plaques were assessed by using multivariate regression. Results: Seventy-five patients (mean age, 63.4 ± 11.6 years; males, 46) were included. Two hundred and twenty-one segments with IAC were identified on CT in 66 patients, including 86 (38.9%) predominantly intimal calcifications and 135 (61.1%) predominantly medial calcifications. A total of 72.0% of intimal calcifications coexisted with atherosclerotic plaques, whereas only 10.2% of medial calcifications coexisted with plaques. Intimal calcification was more commonly shown in non-culprit plaques than culprit plaques (25.9 vs. 9.4%, P = 0.008). The multivariate mixed logistic regression adjusted for the degree of stenosis showed that intimal calcification was significantly associated with non-culprit plaques (OR, 2.971; 95% CI, 1.036-8.517; P = 0.043). Conclusion: Our findings suggest that intimal calcification may indicate the existence of a stable form of atherosclerotic plaque, but plaques can exist in the absence of intimal calcification especially in the middle cerebral artery.
| S-EPMC8319500 | biostudies-literature
Project description:Over the past two decades a significant number of OCT segmentation approaches have been proposed in the literature. Each methodology has been conceived for and/or evaluated using specific datasets that do not reflect the complexities of the majority of widely available retinal features observed in clinical settings. In addition, there does not exist an appropriate OCT dataset with ground truth that reflects the realities of everyday retinal features observed in clinical settings. While the need for unbiased performance evaluation of automated segmentation algorithms is obvious, the validation process of segmentation algorithms have been usually performed by comparing with manual labelings from each study and there has been a lack of common ground truth. Therefore, a performance comparison of different algorithms using the same ground truth has never been performed. This paper reviews research-oriented tools for automated segmentation of the retinal tissue on OCT images. It also evaluates and compares the performance of these software tools with a common ground truth.
| S-EPMC5025289 | biostudies-literature
Project description:ObjectiveTo develop an automated vessel wall segmentation method using convolutional neural networks to facilitate the quantification on magnetic resonance (MR) vessel wall images of patients with intracranial atherosclerotic disease (ICAD).MethodsVessel wall images of 56 subjects were acquired with our recently developed whole-brain three-dimensional (3-D) MR vessel wall imaging (VWI) technique. An intracranial vessel analysis (IVA) framework was presented to extract, straighten, and resample the interested vessel segment into 2-D slices. A U-net-like fully convolutional networks (FCN) method was proposed for automated vessel wall segmentation by hierarchical extraction of low- and high-order convolutional features.ResultsThe network was trained and validated on 1160 slices and tested on 545 slices. The proposed segmentation method demonstrated satisfactory agreement with manual segmentations with Dice coefficient of 0.89 for the lumen and 0.77 for the vessel wall. The method was further applied to a clinical study of additional 12 symptomatic and 12 asymptomatic patients with >50% ICAD stenosis at the middle cerebral artery (MCA). Normalized wall index at the focal MCA ICAD lesions was found significantly larger in symptomatic patients compared to asymptomatic patients.ConclusionWe have presented an automated vessel wall segmentation method based on FCN as well as the IVA framework for 3-D intracranial MR VWI.SignificanceThis approach would make large-scale quantitative plaque analysis more realistic and promote the adoption of MR VWI in ICAD management.
| S-EPMC6788976 | biostudies-literature
Project description:Background and purposeIn radiotherapy, automatic organ-at-risk segmentation algorithms allow faster delineation times, but clinically relevant contour evaluation remains challenging. Commonly used measures to assess automatic contours, such as volumetric Dice Similarity Coefficient (DSC) or Hausdorff distance, have shown to be good measures for geometric similarity, but do not always correlate with clinical applicability of the contours, or time needed to adjust them. This study aimed to evaluate the correlation of new and commonly used evaluation measures with time-saving during contouring.Materials and methodsTwenty lung cancer patients were used to compare user-adjustments after atlas-based and deep-learning contouring with manual contouring. The absolute time needed (s) of adjusting the auto-contour compared to manual contouring was recorded, from this relative time-saving (%) was calculated. New evaluation measures (surface DSC and added path length, APL) and conventional evaluation measures (volumetric DSC and Hausdorff distance) were correlated with time-recordings and time-savings, quantified with the Pearson correlation coefficient, R.ResultsThe highest correlation (R = 0.87) was found between APL and absolute adaption time. Lower correlations were found for APL with relative time-saving (R = -0.38), for surface DSC with absolute adaption time (R = -0.69) and relative time-saving (R = 0.57). Volumetric DSC and Hausdorff distance also showed lower correlation coefficients for absolute adaptation time (R = -0.32 and 0.64, respectively) and relative time-saving (R = 0.44 and -0.64, respectively).ConclusionSurface DSC and APL are better indicators for contour adaptation time and time-saving when using auto-segmentation and provide more clinically relevant and better quantitative measures for automatically-generated contour quality, compared to commonly-used geometry-based measures.
| S-EPMC7807544 | biostudies-literature
Project description:PurposeRecent advances in magnetic resonance (MR) scanner quality and the rapidly improving nature of facial recognition software have necessitated the introduction of MR defacing algorithms to protect patient privacy. As a result, there are a number of MR defacing algorithms available to the neuroimaging community, with several appearing in just the last five years. While some qualities of these defacing algorithms, such as patient identifiability, have been explored in previous works, the potential impact of defacing on neuroimage processing has yet to be explored.ApproachWe qualitatively evaluate eight MR defacing algorithms on 179 subjects from the OASIS-3 cohort and the 21 subjects from the Kirby-21 dataset. We also evaluate the effects of defacing on two neuroimaging pipelines-SLANT and FreeSurfer-by comparing the segmentation consistency between the original and defaced images.ResultsDefacing can alter brain segmentation and even lead to catastrophic failures, which are more frequent with some algorithms such as Quickshear, MRI_Deface, and FSL_deface. Compared to FreeSurfer, SLANT is less affected by defacing. On outputs that pass the quality check, the effects of defacing are less pronounced than those of rescanning, as measured by the Dice similarity coefficient.ConclusionsThe effects of defacing are noticeable and should not be disregarded. Extra attention, in particular, should be paid to the possibility of catastrophic failures. It is crucial to adopt a robust defacing algorithm and perform a thorough quality check before releasing defaced datasets. To improve the reliability of analysis in scenarios involving defaced MRIs, it's encouraged to include multiple brain segmentation pipelines.
| S-EPMC10246049 | biostudies-literature
Project description:Background and Objective: Studies on relations between arterial stiffness and full spectrum of radiological features of cerebral small vessel disease (CSVD) are scarce. We aim to investigate the association of arterial stiffness with lacunes, white matter hyperintensities (WMH), microbleeds (CMBs), dilated perivascular spaces (PVS), and brain atrophy in a community-based sample. Methods: A total of 953 participants (55.7 ± 9.4 years) who underwent brachial-ankle pulse wave velocity (baPWV) and brain magnetic resonance imaging were included. Lacunes, CMBs, and PVS were visually rated. Brain structure and WMH were automatically segmented. Brain parenchyma fraction (BPF), a surrogate index of brain atrophy, was calculated as a ratio of brain parenchyma volume to total intracranial volume. Multivariable logistic and linear regressions were used to investigate the associations between baPWV and CSVD. Subsequently, we explored these associations in strata of age. Results: Increased baPWV was associated with severe PVS in white matter (OR, 1.09; 95%CI, 1.01-1.17; p = 0.022), larger WMH volume (β, 0.08; 95%CI, 0.04-0.12; p < 0.001), lower BPF (β, -0.09; 95%CI, -0.15- -0.03; p = 0.007), and marginally associated with strictly lobar CMBs (OR, 1.11; 95%CI, 1.00-1.23; p = 0.055), but not with lacunes. WMH volume mediated the relation between baPWV and BPF. In age subgroup analysis, the association of baPWV with PVS in white matter was stronger among those aged <55 years, whereas the association with brain atrophy was more prominent among those aged ≥55 years. Increased baPWV was associated with larger WMH volume in both younger and older individuals. Conclusions: Increased arterial stiffness was associated with most of imaging markers of CSVD, including PVS in white matter, larger WMH volume, strictly lobar CMBs, and brain atrophy, but not lacunes. The mechanisms underlying these associations and their potential clinical significances warrant further investigations.
| S-EPMC6121106 | biostudies-literature