Dataset Information

Accuracy of computer-assisted vertical cup-to-disk ratio grading for glaucoma screening.

ABSTRACT:

Purpose

Glaucoma screening can be performed by assessing the vertical-cup-to-disk ratio (VCDR) of the optic nerve head from fundus photography, but VCDR grading is inherently subjective. This study investigated whether computer software could improve the accuracy and repeatability of VCDR assessment.

Methods

In this cross-sectional diagnostic accuracy study, 5 ophthalmologists independently assessed the VCDR from a set of 200 optic disk images, with the median grade used as the reference standard for subsequent analyses. Eight non-ophthalmologists graded each image by two different methods: by visual inspection and with assistance from a custom-made publicly available software program. Agreement with the reference standard grade was assessed for each method by calculating the intraclass correlation coefficient (ICC), and the sensitivity and specificity determined relative to a median ophthalmologist grade of ?0.7.

Results

VCDR grades ranged from 0.1 to 0.9 for visual assessment and from 0.1 to 1.0 for software-assisted grading, with a median grade of 0.4 for each. Agreement between each of the 8 graders and the reference standard was higher for visual inspection (median ICC 0.65, interquartile range 0.57 to 0.82) than for software-assisted grading (median ICC 0.59, IQR 0.44 to 0.71); P = 0.02, Wilcoxon signed-rank test). Visual inspection and software assistance had similar sensitivity and specificity for detecting glaucomatous cupping.

Conclusion

The computer software used in this study did not improve the reproducibility or validity of VCDR grading from fundus photographs compared with simple visual inspection. More clinical experience was correlated with higher agreement with the ophthalmologist VCDR reference standard.

SUBMITTER: Snyder BM

PROVIDER: S-EPMC6687168 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accuracy of computer-assisted vertical cup-to-disk ratio grading for glaucoma screening.

Snyder Blake M BM Nam Sang Min SM Khunsongkiet Preeyanuch P Ausayakhun Sakarin S Leeungurasatien Thidarat T Leiter Maxwell R MR Sevastopolsky Artem A Joye Ashlin S AS Berlinberg Elyse J EJ Liu Yingna Y Ramirez David A DA Moe Caitlin A CA Ausayakhun Somsanguan S Stamper Robert L RL Keenan Jeremy D JD

PloS one 20190808 8

<h4>Purpose</h4>Glaucoma screening can be performed by assessing the vertical-cup-to-disk ratio (VCDR) of the optic nerve head from fundus photography, but VCDR grading is inherently subjective. This study investigated whether computer software could improve the accuracy and repeatability of VCDR assessment.<h4>Methods</h4>In this cross-sectional diagnostic accuracy study, 5 ophthalmologists independently assessed the VCDR from a set of 200 optic disk images, with the median grade used as the re ...[more]

PMID: 31393904

Similar Datasets

Project description:After the US Food and Drug Administration (FDA) approved computer-aided detection (CAD) for mammography in 1998, and the Centers for Medicare and Medicaid Services (CMS) provided increased payment in 2002, CAD technology disseminated rapidly. Despite sparse evidence that CAD improves accuracy of mammographic interpretations and costs over $400 million a year, CAD is currently used for most screening mammograms in the United States.To measure performance of digital screening mammography with and without CAD in US community practice.We compared the accuracy of digital screening mammography interpreted with (n = 495 818) vs without (n = 129 807) CAD from 2003 through 2009 in 323 973 women. Mammograms were interpreted by 271 radiologists from 66 facilities in the Breast Cancer Surveillance Consortium. Linkage with tumor registries identified 3159 breast cancers in 323 973 women within 1 year of the screening.Mammography performance (sensitivity, specificity, and screen-detected and interval cancers per 1000 women) was modeled using logistic regression with radiologist-specific random effects to account for correlation among examinations interpreted by the same radiologist, adjusting for patient age, race/ethnicity, time since prior mammogram, examination year, and registry. Conditional logistic regression was used to compare performance among 107 radiologists who interpreted mammograms both with and without CAD.Screening performance was not improved with CAD on any metric assessed. Mammography sensitivity was 85.3% (95% CI, 83.6%-86.9%) with and 87.3% (95% CI, 84.5%-89.7%) without CAD. Specificity was 91.6% (95% CI, 91.0%-92.2%) with and 91.4% (95% CI, 90.6%-92.0%) without CAD. There was no difference in cancer detection rate (4.1 in 1000 women screened with and without CAD). Computer-aided detection did not improve intraradiologist performance. Sensitivity was significantly decreased for mammograms interpreted with vs without CAD in the subset of radiologists who interpreted both with and without CAD (odds ratio, 0.53; 95% CI, 0.29-0.97).Computer-aided detection does not improve diagnostic accuracy of mammography. These results suggest that insurers pay more for CAD with no established benefit to women.

Project description:ObjectivesSuccessful lung cancer screening delivery requires sensitive, timely reporting of low-dose computed tomography (LDCT) scans, placing a demand on radiology resources. Trained non-radiologist readers and computer-assisted detection (CADe) software may offer strategies to optimise the use of radiology resources without loss of sensitivity. This report examines the accuracy of trained reporting radiographers using CADe support to report LDCT scans performed as part of the Lung Screen Uptake Trial (LSUT).MethodsIn this observational cohort study, two radiographers independently read all LDCT performed within LSUT and reported on the presence of clinically significant nodules and common incidental findings (IFs), including recommendations for management. Reports were compared against a 'reference standard' (RS) derived from nodules identified by study radiologists without CADe, plus consensus radiologist review of any additional nodules identified by the radiographers.ResultsA total of 716 scans were included, 158 of which had one or more clinically significant pulmonary nodules as per our RS. Radiographer sensitivity against the RS was 68-73.7%, with specificity of 92.1-92.7%. Sensitivity for detection of proven cancers diagnosed from the baseline scan was 83.3-100%. The spectrum of IFs exceeded what could reasonably be covered in radiographer training.ConclusionOur findings highlight the complexity of LDCT reporting requirements, including the limitations of CADe and the breadth of IFs. We are unable to recommend CADe-supported radiographers as a sole reader of LDCT scans, but propose potential avenues for further research including initial triage of abnormal LDCT or reporting of follow-up surveillance scans.Key points• Successful roll-out of mass screening programmes for lung cancer depends on timely, accurate CT scan reporting, placing a demand on existing radiology resources. • This observational cohort study examines the accuracy of trained radiographers using computer-assisted detection (CADe) software to report lung cancer screening CT scans, as a potential means of supporting reporting workflows in LCS programmes. • CADe-supported radiographers were less sensitive than radiologists at identifying clinically significant pulmonary nodules, but had a low false-positive rate and good sensitivity for detection of confirmed cancers.

Project description:Our objective was to compare the diagnostic accuracies of and to determine the correlations between the disc damage likelihood scale (DDLS) and anatomical and functional tests used for glaucoma detection. A total of 54 healthy subjects (54 eyes) and 47 primary open-angle glaucoma patients (47 eyes) were included in this cross-sectional observational study. DDLS scores and cup-to-disc (C/D) ratios were evaluated. Subjects underwent standard automated perimetry (SAP), optic disc and retinal nerve fiber layer (RNFL) imaging with time and spectral-domain optical coherence tomography (TD and SD-OCT), Heidelberg Retina Tomograph (HRT II), and scanning laser polarimetry (GDx-VCC). Areas under the receiver operating characteristic curves (AROCs) for DDLS and diagnostic tests parameters were calculated. DDLS correlations (Spearman's rank) among these parameters were analyzed. Fifty-four eyes were healthy and 47 had glaucoma, including 16 preperimetric glaucoma. DDLS, vertical and horizontal C/D ratios had the largest AROCs (0.92, 0.94 and 0.91, respectively). DDLS diagnostic accuracy was better than the accuracies of HRT II parameters, TD and SD-OCT RNFL thicknesses, and SAP mean deviation (MD) index. There were no significant differences between the accuracies of the DDLS and the C/D ratios, TD-OCT vertical (0.89) and horizontal (0.86) C/D ratios, TD-OCT C/D area ratio (0.89), and GDx-VCC NFI (0.81). DDLS showed significant strong correlations with vertical (r = 0.79) and horizontal (0.74) C/D ratios, and with the parameters vertical C/D ratio and C/D area ratio from HRT II (both 0.77) and TD-OCT (0.75 and 0.72, respectively). DDLS had significant moderate correlations with most of the other structural measurements and SAP MD. The optic disc clinical evaluation with DDLS system and C/D ratio demonstrated excellent accuracy in distinguishing glaucomatous from healthy eyes. DDLS had moderate to strong correlations with most structural and functional parameters. These findings stress the importance of optic disc clinical examination to detect glaucoma in a clinical scenario.

Project description:BACKGROUND:Web applications that employ natural language processing technologies to support systematic reviewers during abstract screening have become more common. The goal of our project was to conduct a case study to explore a screening approach that temporarily replaces a human screener with a semi-automated screening tool. METHODS:We evaluated the accuracy of the approach using DistillerAI as a semi-automated screening tool. A published comparative effectiveness review served as the reference standard. Five teams of professional systematic reviewers screened the same 2472 abstracts in parallel. Each team trained DistillerAI with 300 randomly selected abstracts that the team screened dually. For all remaining abstracts, DistillerAI replaced one human screener and provided predictions about the relevance of records. A single reviewer also screened all remaining abstracts. A second human screener resolved conflicts between the single reviewer and DistillerAI. We compared the decisions of the machine-assisted approach, single-reviewer screening, and screening with DistillerAI alone against the reference standard. RESULTS:The combined sensitivity of the machine-assisted screening approach across the five screening teams was 78% (95% confidence interval [CI], 66 to 90%), and the combined specificity was 95% (95% CI, 92 to 97%). By comparison, the sensitivity of single-reviewer screening was similar (78%; 95% CI, 66 to 89%); however, the sensitivity of DistillerAI alone was substantially worse (14%; 95% CI, 0 to 31%) than that of the machine-assisted screening approach. Specificities for single-reviewer screening and DistillerAI were 94% (95% CI, 91 to 97%) and 98% (95% CI, 97 to 100%), respectively. Machine-assisted screening and single-reviewer screening had similar areas under the curve (0.87 and 0.86, respectively); by contrast, the area under the curve for DistillerAI alone was just slightly better than chance (0.56). The interrater agreement between human screeners and DistillerAI with a prevalence-adjusted kappa was 0.85 (95% CI, 0.84 to 0.86%). CONCLUSIONS:The accuracy of DistillerAI is not yet adequate to replace a human screener temporarily during abstract screening for systematic reviews. Rapid reviews, which do not require detecting the totality of the relevant evidence, may find semi-automation tools to have greater utility than traditional systematic reviews.

Dataset Information

Accuracy of computer-assisted vertical cup-to-disk ratio grading for glaucoma screening.

Purpose

Methods

Results

Conclusion

Publications

Accuracy of computer-assisted vertical cup-to-disk ratio grading for glaucoma screening.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets