Dataset Information

Algorithm Variability in Quantification of Epithelial Defect Size in Microbial Keratitis Images.

ABSTRACT:

Purpose

To investigate the sources of measurement variability when quantifying the morphology of microbial keratitis (MK) from slit-lamp photography (SLP) images using a semiautomated, image-analysis algorithm.

Methods

Prospectively enrolled patients with MK underwent SLP to obtain images of their epithelial defects (ED). Eyes were stained with fluorescein and imaged multiple times under blue light, at low and high magnifications. A masked research assistant chose the 3 best images and annotated each 3 times to provide seed regions corresponding to ED and healthy cornea. The algorithm returned the ED area for each seeded image. Eyes without EDs and algorithm failures were excluded. Variance components were estimated with a random effects model and intraclass correlation coefficients estimated with intragrader reliability.

Results

A total of 42 eyes from 42 MK participants were photographed. After excluding poor quality images, eyes with no EDs, and algorithm failures, 34 patients with 92 images and 274 seeds were analyzed. No significant differences in the average ED area were found between seedings or high- versus low-SLP magnifications (all P > 0.5, paired t tests). Minimal measurement variability was because of image (0.9%), magnification (0.2%), or seed (0.1%). Most variability was attributable to differences in ED sizes between patients (85.2%). 13.7% of variability was unexplained. Multiple iterations of the algorithm on the same image showed good consistency (intraclass correlation coefficient = 0.98, 95% confidence interval, 0.97-0.99).

Conclusions

Image-analysis algorithms showed good reliability for measuring the ED area from SLP images. Most measurement variability was because of between-patient differences, not imaging settings or application of the algorithm by the user.

SUBMITTER: Kriegel MF

PROVIDER: S-EPMC7732187 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Acute myocardial infarct (AMI) size depicted by late gadolinium enhancement cardiovascular magnetic resonance (CMR) is increasingly used as an efficacy endpoint in randomized trials comparing AMI therapies. Infarct size is quantified using manual planimetry (MANUAL), visual scoring (VISUAL), or automated techniques using signal-intensity thresholding (AUTO). Although AUTO is considered the most reproducible, prior studies did not account for the subjective determination of endocardial/epicardial borders, which all methods require. For MANUAL and VISUAL, prior studies did not address how to treat intermediate signal intensities due to partial volume.To assess sources of variability, AMI size was measured in 30 patients and 12 controls by 3 core-laboratories using 8 methods, each separated by more than 2 months time (n = 720 evaluations). The methods were: (1,2) AUTOSegment, AUTOFWHM (using Segment software or the full-width-at-half-maximum algorithm, respectively); (3,4) AUTO-UCSegment, AUTO-UCFWHM (user correction for endocardial border pixels, no-reflow, etc.); (5) MANUAL; (6) MANUAL-ISI (adjustment for intermediate signal-intensities); (7) VISUAL; (8) VISUAL-ISI.Mean infarct size varied between 16.8% and 27.2% of LV mass depending on method. Even automated techniques with no user interaction for infarct borders resulted in significant within-patient variability given the need to subjectively trace endocardial/epicardial contours. The coefficient-of-variation (CV) was 10.6% and 14.6% for AUTOSegment and AUTOFWHM, respectively. For manual and visual categories, reproducibility was improved when intermediate signal-intensities were considered (MANUAL-ISI vs MANUAL: CV = 8.3% vs 14.4%; p = 0.03; VISUAL-ISI vs VISUAL: CV = 8.4% vs 10.9%; p = 0.01). For AUTO-UCSegment, MANUAL-ISI, and VISUAL-ISI (best technique in each category) within-patient variability due to the quantification method was less than 10% of total variability, and the required sample sizes for detecting a 5% absolute difference in infarct size were 62, 63, and 62 patients, respectively.Among CMR core-laboratories, an important source of variability in infarct size quantification is the subjective delineation of endocardial/epicardial borders. When intermediate signal intensities are considered in manual planimetry and visual scoring, reproducibility and impact on sample size are similar to automated techniques.