Dataset Information

Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

ABSTRACT: Measurable competence derived from comprehensive and advanced training in grading digital images is critical in studies using a reading center to evaluate retinal fundus images from infants at risk for retinopathy of prematurity (ROP). Details of certification for nonphysician trained readers (TRs) have not yet been described.To describe a centralized system for grading ROP digital images by TRs in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.Multicenter observational cohort study conducted from July 1, 2010, to June 30, 2014. The TRs were trained by experienced ROP specialists and certified to detect ROP morphology in digital retinal images under supervision of an ophthalmologist reading center director. An ROP reading center was developed with standard hardware, secure Internet access, and customized image viewing software with an electronic grading form. A detailed protocol for grading was developed. Based on results of TR gradings, a computerized algorithm determined whether referral-warranted ROP (RW-ROP; defined as presence of plus disease, zone I ROP, and stage 3 or worse ROP) was present in digital images from infants with birth weight less than 1251 g enrolled from May 25, 2011, through October 31, 2013. Independent double grading was done by the TRs with adjudication of discrepant fields performed by the reading center director.Digital retinal images.Intragrader and intergrader variability and monitoring for temporal drift.Four TRs underwent rigorous training and certification. A total of 5520 image sets were double graded, with 24.5% requiring adjudication for at least 1 component of RW-ROP. For individual RW-ROP components, the adjudication rate was 3.9% for plus disease, 12.4% for zone I ROP, and 16.9% for stage 3 or worse ROP. The weighted ? for intergrader agreement (n?=?80 image sets) was 0.72 (95% CI, 0.52-0.93) for RW-ROP, 0.57 (95% CI, 0.37-0.77) for plus disease, 0.43 (95% CI, 0.24-0.63) for zone I ROP, and 0.67 (95% CI, 0.47-0.88) for stage 3 or worse ROP. The weighted ? for grade-regrade agreement was 0.77 (95% CI, 0.57-0.97) for RW-ROP, 0.87 (95% CI, 0.67-1.00) for plus disease, 0.70 (95% CI, 0.51-0.90) for zone I ROP, and 0.77 (95% CI, 0.57-0.97) for stage 3 or worse ROP.These data suggest that the e-ROP system for training and certifying nonphysicians to grade ROP images under the supervision of a reading center director reliably detects potentially serious ROP with good intragrader and intergrader consistency and minimal temporal drift.

SUBMITTER: Daniel E

PROVIDER: S-EPMC4910817 | biostudies-literature | 2015 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

Daniel Ebenezer E Quinn Graham E GE Hildebrand P Lloyd PL Ells Anna A Hubbard G Baker GB Capone Antonio A Martin E Revell ER Ostroff Candace P CP Smith Eli E Pistilli Maxwell M Ying Gui-Shuang GS

JAMA ophthalmology 20150601 6

<h4>Importance</h4>Measurable competence derived from comprehensive and advanced training in grading digital images is critical in studies using a reading center to evaluate retinal fundus images from infants at risk for retinopathy of prematurity (ROP). Details of certification for nonphysician trained readers (TRs) have not yet been described.<h4>Objective</h4>To describe a centralized system for grading ROP digital images by TRs in the Telemedicine Approaches to Evaluating Acute-Phase Retinop ...[more]

PMID: 25811772

Similar Datasets

Project description:PurposeTo describe the clinical characteristics of intraocular hemorrhages (IOHs) in infants in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study and to evaluate their potential use for prediction of disease severity.DesignSecondary data analysis from a prospective study.ParticipantsPreterm infants with birth weight (BW) ≤1250 g.MethodsInfants underwent serial digital retinal imaging in both eyes starting at 32 weeks' postmenstrual age. Nonphysician trained readers (TRs) evaluated all image sets from eyes that ever had IOHs documented on image evaluation or eye examination for the presence, location, type, area, and relation of the IOH to the junction between vascularized and avascular retina. Associations of IOH with demographic and neonatal factors, and with the presence and severity of retinopathy of prematurity (ROP) were investigated by univariate and multivariate analyses. Sensitivity and specificity of the telemedicine system for detecting referral-warranted ROP (RW-ROP) were calculated with and without incorporating hemorrhage into the standardized grading protocol.Main outcome measuresRetinal and vitreous hemorrhage.ResultsAmong 1239 infants (mean [standard deviation] BW = 864 [212] g; gestational age [GA] = 27 [2.2] weeks) who underwent an average of 3.2 imaging sessions, 22% had an IOH in an eye on at least 1 of the e-ROP visits. Classification of IOH was preretinal (57%), blot (57%), dot (38%), flame-shaped (16%), and vitreous (8%); most IOHs were unilateral (70%). The IOH resolved in 35% of eyes by the next imaging session and in the majority (76%) of cases by 8 weeks after initial detection. Presence of IOH was inversely associated with BW and GA and significantly associated (P < 0.0001) with the presence and severity of ROP (BW and GA adjusted odds ratios [ORs] of 2.46 for any ROP, 2.88 for stage 3, and 3.19 for RW-ROP). Incorporating IOH into the grading protocol minimally altered the sensitivity of the system (94% vs. 95%).ConclusionsApproximately 1 in 5 preterm infants examined had IOHs, generally unilateral. The presence of hemorrhage was directly correlated with both presence and severity of ROP and inversely correlated with BW and GA, although including hemorrhage in the grading algorithm only minimally improved the sensitivity of the telemedicine system to detect RW-ROP.

Project description:PurposeTo develop a risk score for predicting treatment-requiring retinopathy of prematurity (TR-ROP) in the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) study.DesignSecond analyses of an observational cohort study.ParticipantsInfants with birth weight (BW) <1251 g who had ≥1 imaging session by 34 weeks of postmenstrual age (PMA) and ≥1 subsequent retinopathy of prematurity (ROP) examination for determining TR-ROP by study-certified ophthalmologists.MethodsNonphysician trained readers evaluated wide-field retinal image sets for characteristics of ROP, pre-plus/plus disease, and retinal hemorrhage. Risk score points for predicting TR-ROP were derived from the regression coefficients of significant predictors in a multivariate logistic regression model.Main outcome measuresTR-ROP.ResultsEighty-five of 771 infants (11.0%) developed TR-ROP. In a multivariate model, significant predictors for TR-ROP were gestational age (GA) (odds ratio [OR], 5.7; 95% confidence interval [CI], 1.7-18.9 for ≤25 vs. ≥28 weeks), need for respiratory support (OR, 7.0; 95% CI, 1.3-37.1 for high-frequency oscillatory ventilation vs. no respiratory support), slow weight gain (OR, 2.4; 95% CI, 1.2-4.6 for weight gain ≤12 g/day vs. >15 g/day), and image findings at the first image session including number of quadrants with pre-plus (OR, 3.8; 95% CI, 1.5-9.7 for 4 pre-plus quadrants vs. no pre-plus), stage and zone of ROP (OR, 4.7; 95% CI, 2.1-11.8 for stage 1-2 zone I, OR, 5.9; 95% CI, 2.1-16.6 for stage 3 zone I vs. no ROP), and presence of blot hemorrhage (OR, 3.1; 95% CI, 1.4-6.7). Image findings predicted TR-ROP better than GA (area under receiver operating characteristic curve [AUC] = 0.82 vs. 0.75, P = 0.03). The risk of TR-ROP steadily increased with higher risk score and predicted TR-ROP well (AUC = 0.88; 95% CI, 0.85-0.92). Risk score ≥3 points for predicting TR-ROP had a sensitivity of 98.8%, specificity of 40.1%, and positive and negative predictive values of 17.0% and 99.6%, respectively.ConclusionsImage characteristics at 34 PMA weeks or earlier independently predict TR-ROP. If externally validated in other infants, risk score, calculated from image findings, GA, weight gain, and respiratory support, enables early identification of infants in need of increased surveillance for TR-ROP.

Project description:IMPORTANCE:Detection of treatment-requiring retinopathy of prematurity (ROP) involves serial eye examinations. An ROP prediction model using predictive factors could identify high-risk infants and reduce required eye examinations. OBJECTIVE:To determine predictive factors for the development of referral-warranted (RW) ROP. DESIGN, SETTING, AND PARTICIPANTS:This multicenter observational cohort study included secondary analysis of data from the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity Study. Infants included in the study had a birth weight (BW) of less than 1251 g. EXPOSURES:Serial ROP examinations of premature infants who had 2 or more ROP examinations. MAIN OUTCOMES AND MEASURES:Incidence of RW-ROP (defined as the presence of plus disease, zone I ROP, or ROP stage 3 or greater in either eye) and associations with predictive factors. RESULTS:Among 979 infants without RW-ROP at first study-related eye examination (median postmenstrual age, 33 weeks; range, 29-40 weeks) who underwent at least 2 eye examinations, 149 (15.2%) developed RW-ROP. In a multivariate model, significant predictors for RW-ROP were male sex (odds ratio [OR], 1.80; 95% CI, 1.13-2.86 vs female), nonblack race (OR, 2.76; 95% CI, 1.50-5.08 for white vs black race and OR, 4.81; 95% CI, 2.19-10.6 for other vs black race), low BW (OR, 5.16; 95% CI, 1.12-7.20 for ?500 g vs >1100 g), younger gestational age (OR, 9.79; 95% CI, 3.49-27.5 for ?24 weeks vs ?28 weeks), number of quadrants with preplus disease (OR, 7.12; 95% CI, 2.53-20.1 for 1-2 quadrants and OR, 18.4; 95% CI, 4.28-79.4 for 3-4 quadrants vs no preplus disease), stage 2 ROP (OR, 4.13; 95% CI, 2.13-8.00 vs no ROP), the presence of retinal hemorrhage (OR, 4.36; 95% CI, 1.57-12.1 vs absence), the need for respiratory support (OR, 4.99; 95% CI, 1.89-13.2 for the need for controlled mechanical ventilator; OR, 11.0; 95% CI, 2.26-53.8 for the need for high-frequency oscillatory ventilation vs no respiratory support), and slow weight gain (OR, 2.44; 95% CI, 1.22-4.89 for weight gain ?12 g/d vs >18 g/d). These characteristics predicted the development of RW-ROP significantly better than BW and gestational age (area under receiver operating characteristic curve, 0.88 vs 0.78; P?<?.001). CONCLUSIONS AND RELEVANCE:When controlling for very low BW and prematurity, the presence of preplus disease, stage 2 ROP, retinal hemorrhage, and the need for ventilation at time of first study-related eye examination were strong independent predictors for RW-ROP. These predictors may help identify infants in need of timely eye examinations.

Project description:To determine the symmetry on retinal image grading of fellow eyes for retinopathy of prematurity (ROP) features (stage, zone and plus disease) and severity, to provide the basis for the within subject comparison for ROP trials.Secondary analyses of data from the Telemedicine Approaches to Evaluating of Acute-Phase ROP (e-ROP) Study.Infants with birth weight less than 1251g.Infants underwent serial retinal imaging sessions in both eyes by certified imagers starting at 32-weeks postmenstrual age (PMA). Two trained non-physician readers graded each eye independently for ROP features in a 5 retinal-image set from each session. Discrepancies were adjudicated by a reading center supervisor. Readers were masked to all eye examination results, previous gradings of both eyes, current grading of the fellow eye, and demographic data.The inter-eye agreement assessed using percent exact agreement and weighted kappa (Kw ) for stage, zone, plus, referral-warranted ROP (RW-ROP, defined as presence of stage 3 or above, plus disease, or zone I ROP), and severity of ROP.Among 3918 image sessions in 1235 infants, the percent agreement (Kw ) between paired eyes was 75.3% (0.65) for stage of ROP, 82.3% (0.68) for zone of ROP, 78.7% (0.51) for plus disease, 84.7% (0.56) for RW-ROP, and 72.7% (0.63) for severity of ROP. Similar inter-eye agreements were found when considering ROP features at the first image session, at the last image session, at any image session. Based on image evaluations from all sessions, 412 (33.4%) infants had ROP stage 3 or above, 148 (12.0%) had zone I ROP, 70 (5.7%) had plus disease, and 419 (33.9%) had RW-ROP in one or both eyes; symmetrical findings were present in 71.4% for ROP stage 3 or above, 56.8% for zone I ROP, 50.0% for plus disease, and 73.7% for RW-ROP.Masked image evaluations by trained readers showed good inter-eye agreement in ROP characteristics, consistent with the high inter-eye agreement in ROP from clinical examinations by ophthalmologists in other studies. These data confirm that acute ROP is typically symmetrical and supports within subject comparison in ROP trials.

Project description:As effective treatments for potentially blinding retinopathy of prematurity (ROP) have been introduced, the importance of consistency in findings has increased, especially with the shift toward retinal imaging in infants at risk of ROP.To characterize discrepancies in findings of ROP between digital retinal image grading and examination results from the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity study, conducted from May 2011 to October 2013.A poststudy consensus review of images was conducted by 4 experts, who examined discrepancies in findings between image grades by trained nonphysician readers and physician examination results in infants with referral-warranted ROP (RW-ROP). Images were obtained from 13 North American neonatal intensive care units from eyes of infants with birth weights less than 1251 g. For discrepancy categories with more than 100 cases, 40 were randomly selected; in total, 188 image sets were reviewed.Consensus evaluation of discrepant image and examination findings for RW-ROP components.Among 5350 image set pairs, there were 161 instances in which image grading did not detect RW-ROP noted on clinical examination (G-/E+) and 854 instances in which grading noted RW-ROP when the examination did not (G+/E-). Among the sample of G-/E+ cases, 18 of 32 reviews (56.3%) agreed with clinical examination findings that ROP was present in zone I and 18 of 40 (45.0%) agreed stage 3 ROP was present, but only 1 of 20 (5.0%) agreed plus disease was present. Among the sample of G+/E- cases, 36 of 40 reviews (90.0%) agreed with readers that zone I ROP was present, 23 of 40 (57.5%) agreed with readers that stage 3 ROP was present, and 4 of 16 (25.0%) agreed that plus disease was present. Based on the consensus review results of the sampled cases, we estimated that review would agree with clinical examination findings in 46.5% of the 161 G-/E+ cases (95% CI, 41.6-51.6) and agree with trained reader grading in 70.0% of the 854 G+/E- cases (95% CI, 67.3-72.8) for the presence of RW-ROP.This report highlights limitations and strengths of both the remote evaluation of fundus images and bedside clinical examination of infants at risk for ROP. These findings highlight the need for standardized approaches as ROP telemedicine becomes more widespread.

Project description:To report on a 6-year experience with wide-field digital imaging based telemedicine (WFDI telemedicine) to reduce the risk for blindness from retinopathy of prematurity (ROP).Wide-angle digital fundus cameras (RetCam 120, Massie Lab, Pleasanton, CA, USA) were installed in five neonatal intensive care units (NICUs) in Germany. All prematures at risk were screened with WFDI, and the local ophthalmologists were asked to continue binocular indirect ophthalmoscopy (BIO) according to the German guidelines. Image data were coded and transferred to the Reading Centre in Regensburg. Image evaluation and additional BIO of infants with suspected treatment-requiring ROP (STR-ROP i.e. threshold ROP zone II, prethreshold ROP zone I (type-1 ROP according to ETROP), and ROP possibly requiring treatment but not reliably classifiable from the images) were performed by paediatric ophthalmologists at the Reading Centre. ROP was classified following ICROP, ETROP, and revised ICROP criteria. Outcome measures were incidence of clinically relevant ROP (CR-ROP, i.e. any ROP up to mid-peripheral zone III, <or= stage 3+), sensitivity to detect STR-ROP, and positive predictive value to detect treatment-requiring ROP (TR-ROP).In total, 1,222 prematures at risk were screened (mean BW 1395 g, SD +/-507 g; mean GA 30 wks, SD +/-3 wks). The overall incidence of CR-ROP was 27.6% (71.8% mild = stage 1 to 3 without plus disease, 15.7% prethreshold = type-1 ROP according to ETROP, 12.5% threshold according to ICROP). Zone I disease was present in 3.3%, zone II disease in 76.5%, and zone III disease in 20.2%. According to ETROP, 95 infants were type-1 or type-2 ROP; 67.4% type-1 ROP, and 32.6% type-2 ROP. Of all 1,222 infants, 3.5% received treatment. Following ETROP (not applied in the study), 5.3% would have been treated. The sensitivity for detecting STR-ROP was 100%, and the positive predictive value for TR-ROP 82.4% (28/34) at the time of the first referral (28 infants, <or= stage 3+ in zone I or II).All TR-ROP was detected in time, showing the potential of our telemedical screening program. The overall incidence of CR-ROP was comparable to ROP incidences reported in other West European countries.

Project description:The present strategy to identify infants needing treatment for retinopathy of prematurity (ROP) requires repeated examinations of at-risk infants by physicians. However, less than 10% ultimately require treatment. Retinal imaging by nonphysicians with remote image interpretation by nonphysicians may provide a more efficient strategy.To evaluate the validity of a telemedicine system to identify infants who have sufficiently severe ROP to require evaluation by an ophthalmologist.An observational study of premature infants starting at 32 weeks' postmenstrual age was conducted. This study involved 1257 infants with birth weight less than 1251 g in neonatal intensive care units in 13 North American centers enrolled from May 25, 2011, through October 31, 2013.Infants underwent regularly scheduled diagnostic examinations by an ophthalmologist and digital imaging by nonphysician staff using a wide-field digital camera. Ophthalmologists documented findings consistent with referral-warranted (RW) ROP (ie, zone I ROP, stage 3 ROP or worse, or plus disease). A standard 6-image set per eye was sent to a central server and graded by 2 trained, masked, nonphysician readers. A reading supervisor adjudicated disagreements.The validity of grading retinal image sets was based on the sensitivity and specificity for detecting RW-ROP compared with the criterion standard diagnostic examination.A total of 1257 infants (mean birth weight, 864 g; mean gestational age, 27 weeks) underwent a median of 3 sessions of examinations and imaging. Diagnostic examination identified characteristics of RW-ROP in 18.2% of eyes (19.4% of infants). Remote grading of images of an eye at a single session had sensitivity of 81.9% (95% CI, 77.4-85.6) and specificity of 90.1% (95% CI, 87.9-91.8). When both eyes were considered for the presence of RW-ROP, as would routinely be done in a screening, the sensitivity was 90.0% (95% CI, 85.4-93.5), with specificity of 87.0% (95% CI, 84.0-89.5), negative predictive value of 97.3%, and positive predictive value of 62.5% at the observed RW-ROP rate of 19.4%.When compared with the criterion standard diagnostic examination, these results provide strong support for the validity of remote evaluation by trained nonphysician readers of digital retinal images taken by trained nonphysician imagers from infants at risk for RW-ROP.clinicaltrials.gov Identifier:NCT01264276.

Dataset Information

Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

Publications

Validated System for Centralized Grading of Retinopathy of Prematurity: Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity (e-ROP) Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets