Dataset Information

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms.

ABSTRACT: To evaluate the interobserver agreement (IOA) between the initial radiology resident and the final staff radiologist reports of combined computed tomographic pulmonary angiograms (CTPA) and computed tomographic venograms (CTV) performed during on-call hours.Approval by the institutional review board was obtained. Six-hundred and ninety-six consecutive studies (CTPA or CTPA with CTV) performed during on-call hours and interpreted by 30 residents were identified. Radiology residents' reports were compared to the final staff reports. Three tests outcomes were considered (positive, P; negative, N; indeterminate, I). Discordant cases were reviews by a chest radiologist.CTPAs were reported by staff radiologists as positive for pulmonary embolism (PE) in 18% (126/694), with a kappa of 0.81 (95% CI 0.77-0.86) with 3 outcomes (P, N, I), and a kappa of 0.89 (95% CI 0.85-0.94) with 2 outcomes (P, N). Regarding PE location, good concordance was observed for positive studies, with a kappa of 0.86 (95% CI 0.78 - 0.95). CTVs were reported as positive by staff radiologists in 8.5% (33/388), with a kappa of 0.66 (95% CI 0.55-0.77) with 3 outcomes (P, N, I), and a kappa of 0.89 (95% CI 0.8-1.0) with 2 outcomes (P, N). The IOA between residents and staff radiologists increased with increasing residency year level for CTPAs, but did not for CTVs.Very good and good IOA were observed between resident and staff radiologist interpretations for CTPA and CTV, respectively, with tendency towards improved IOA as residency level of training increased for CTPA, but not for CTV.

SUBMITTER: Tamjeedi B

PROVIDER: S-EPMC4418836 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms.

Tamjeedi Bahar B Correa José J Semionov Alexandre A Mesurolle Benoît B

PloS one 20150504 5

<h4>Objectives</h4>To evaluate the interobserver agreement (IOA) between the initial radiology resident and the final staff radiologist reports of combined computed tomographic pulmonary angiograms (CTPA) and computed tomographic venograms (CTV) performed during on-call hours.<h4>Materials and methods</h4>Approval by the institutional review board was obtained. Six-hundred and ninety-six consecutive studies (CTPA or CTPA with CTV) performed during on-call hours and interpreted by 30 residents we ...[more]

PMID: 25938666

Similar Datasets

Project description:Background Classification of lung cancer screening CT scans depends on measurement of lung nodule size. Information about interobserver agreement is limited. Purpose To assess interobserver agreement in the measurements and American College of Radiology Lung CT Screening Reporting and Data System (Lung-RADS) classifications of solid lung nodules detected at lung cancer screening using manual measurements of average diameter and computer-aided semiautomated measurements of average diameter and volume (CT volumetry). Materials and Methods Two radiologists and one radiology resident retrospectively measured lung nodules from screening CT scans obtained between September 2016 and June 2018 with a Lung-RADS (version 1.0) classification of 2, 3, 4A, or 4B in the clinical setting. Average manual diameter and semiautomated computer-aided diameter and volume measurements were converted to the corresponding Lung-RADS categories. Interobserver agreement in raw measurements was assessed using intraclass correlation and Bland-Altman indexes, and interobserver agreement in Lung-RADS classification was assessed using bi-rater κ. Results One hundred twenty patients (mean age, 63 years ± 6 [standard deviation]; 67 women) were evaluated. All manual, semiautomated diameter, and semiautomated volume measurements were obtained by all three readers in 120 of 147 nodules (82%). Intraclass correlation coefficients were greater than or equal to 0.95 for all reader pairs using all measurement methods and were highest using volumetry. Bias and 95% limits of agreement for average diameter were smaller with semiautomated measurements than with manual measurements. κ values across all Lung-RADS classifications were greater than or equal to 0.81, with the lowest being for manual measurements and the highest being for volumetric measurements. Forty-three of 120 (36%) of the nodules were classified into a lower Lung-RADS category on the basis of volumetry compared with using manual diameter measurements by at least one reader, whereas the reverse occurred for four of 120 (3%) of the nodules. Conclusion Interobserver agreement was high with manual diameter measurements and increased with semiautomated CT volumetric measurements. Semiautomated CT volumetry enabled classification of more nodules into lower Lung CT Screening Reporting and Data System categories than manual or semiautomated diameter measurements. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Nishino in this issue.

Project description:The American Stroke Association guidelines emphasized the need for further high-quality studies that assess agreement by radiologists and nonradiologists engaged in emergency telestroke assessments and decision-making. Therefore, the objective of this study was to determine the level of agreement of baseline brain CT scan interpretations of patients with acute stroke presenting to telestroke spoke hospitals between central reading committee neuroradiologists and each of 2 groups, spoke hospital radiologists and hub hospital vascular neurologists (telestrokologists).The Stroke Team Remote Evaluation Using a Digital Observation Camera Arizona trial was a prospective, urban single-hub, rural 2-spoke, randomized, blinded, controlled trial of a 2-way, site-independent, audiovisual telemedicine and teleradiology system designed for remote evaluation of adult patients with acute stroke versus telephone consultation to assess eligibility for treatment with intravenous thrombolysis. In the telemedicine arm, the subjects' CT scans were interpreted by the hub telestrokologist and in the telephone arm by the spoke radiologist. All subjects' CT scans were subsequently interpreted centrally, independently, and blindly by 2 hub neuroradiologists. The primary CT outcome was determination of a CT-based contraindication to thrombolytic treatment. Kappa statistics and exact agreement rates were used to analyze interobserver agreement.Fifty-four subjects underwent random assignment. The overall agreement for the presence of radiological contraindications to thrombolysis was excellent (0.91) and did not differ substantially between the hub telestrokologist to neuroradiologist and spoke radiologist to neuroradiologist (0.92 and 0.89, respectively).In the context of a telestroke network designed to assess patients with acute stroke syndromes, agreement over the presence or absence of radiological contraindications to thrombolysis was excellent whether the comparisons were between a telestrokologist and neuroradiologist or between spoke radiologist and neuroradiologist.URL: http://www.clinicaltrials.gov. Unique identifier: NCT00623350.

Project description:Recently, a standardized framework system for interpreting somatostatin receptor (SSTR)-targeted PET/CT, termed the SSTR reporting and data system (RADS) 1.0, was introduced, providing reliable standards and criteria for SSTR-targeted imaging. We determined the interobserver reliability of SSTR-RADS for interpretation of 68Ga-DOTATOC PET/CT scans in a multicentric, randomized setting. Methods: A set of 51 randomized 68Ga-DOTATOC PET/CT scans was independently assessed by 4 masked readers with different levels of experience (2 experienced readers and 2 inexperienced readers) trained on the SSTR-RADS 1.0 criteria (based on a 5-point scale from 1 [definitively benign] to 5 [high certainty that neuroendocrine neoplasia is present]). For each scan, SSTR-RADS scores were assigned to a maximum of 5 target lesions (TLs). An overall scan impression based on SSTR-RADS was indicated, and interobserver agreement rates on a TL-based, on an organ-based, and on an overall SSTR-RADS score-based level were computed. The readers were also asked to decide whether peptide receptor radionuclide therapy (PRRT) should be considered on the basis of the assigned RADS scores. Results: Among the selected TLs, 153 were chosen by at least 2 readers (all 4 readers selected the same TLs in 58 of 153 [37.9%] instances). The interobserver agreement for SSTR-RADS scoring among identical TLs was good (intraclass correlation coefficient [ICC] ≥ 0.73 for 4, 3, and 2 identical TLs). For lymph node and liver lesions, excellent interobserver agreement rates were derived (ICC, 0.91 and 0.77, respectively). Moreover, the interobserver agreement for an overall scan impression based on SSTR-RADS was excellent (ICC, 0.88). The SSTR-RADS-based decision to use PRRT also demonstrated excellent agreement, with an ICC of 0.80. No significant differences between experienced and inexperienced readers for an overall scan impression and TL-based SSTR-RADS scoring were observed (P ≥ 0.18), thereby suggesting that SSTR-RADS seems to be readily applicable even for less experienced readers. Conclusion: SSTR-RADS-guided assessment demonstrated a high concordance rate, even among readers with different levels of experience, supporting the adoption of SSTR-RADS for trials, clinical routine, or outcome studies.

Project description:We evaluated observer agreement for 68Ga-DOTATATE PET/CT interpretations in patients with neuroendocrine tumor (NET). METHODS:68Ga-DOTATATE PET/CT was performed on 50 patients with known or suspected NET of the small bowel (n = 19), pancreas (n = 14), lung (n = 4), or other location (n = 13). The images were reviewed by 7 observers, who used a standardized interpretation approach. The observers were classified as having a low level of experience (<500 scans or <5 y experience with 68Ga-DOTATATE PET/CT; n = 4) or a high level of experience (?500 scans or ?5 y experience with 68Ga-DOTATATE PET/CT; n = 3). Interpretation by the primary nuclear medicine physician, who had access to all clinical and imaging data, served as the reference standard. Interobserver agreement was determined by the Cohen ? statistic and intraclass correlation coefficient (ICC) with corresponding 95% confidence interval (95%CI). RESULTS:Interobserver agreement was substantial, and the median number of false findings was low for the overall scan result: that is, positive versus negative scan result (? = 0.80; 95%CI, 0.74-0.86; false findings, 3), organ involvement (? = 0.70; 95%CI, 0.64-0.76; false findings, 5), and lymph node involvement (? = 0.71; 95%CI, 0.65-0.78; false findings, 6). Interobserver agreement was substantial to almost perfect, and the average absolute difference (?) from the reference observer was low for number of organ and lymph node metastases (organ: ICC, 0.84; 95%CI, 0.77-0.89; ? = 0.45; lymph node: ICC, 0.77; 95%CI, 0.69-0.84; ? = 0.45), tumor SUVmax (ICC, 0.99; 95%CI, 0.97-0.99; ? = 0.44), and reference SUV (spleen: ICC, 0.81; ? = 1.10; liver: ICC, 0.79; ? = 0.62). Interpretations of appropriateness for peptide-receptor radionuclide therapy varied more significantly among observers (? = 0.64; 95%CI, 0.57-0.70), and a higher frequency of false-positive recommendations for peptide-receptor radionuclide therapy occurred in observers with low experience than in those with high experience (range, 7-12 vs. 4-8). CONCLUSION:The interpretation of 68Ga-DOTATATE PET/CT images for NET staging is consistent among observers with low and high levels of experience. However, image-based recommendations for or against peptide-receptor radionuclide therapy require experience and training.

Project description:Recently, the standardized reporting and data system for prostate-specific membrane antigen (PSMA)-targeted PET imaging studies, termed PSMA-RADS version 1.0, was introduced. We aimed to determine the interobserver agreement for applying PSMA-RADS to imaging interpretation of 18F-DCFPyL (2-(3-{1-carboxy-5-[(6-18F-fluoro-pyridine-3-carbonyl)-amino]-pentyl}-ureido)-pentanedioic acid) PET examinations in a prospective setting mimicking the typical clinical workflow at a prostate cancer referral center. Methods: Four readers (2 experienced readers (ERs, >3 y of PSMA-targeted PET interpretation experience) and 2 inexperienced readers (IRs, <1 y of experience)), who had all read the initial publication on PSMA-RADS 1.0, assessed 50 18F-DCFPyL PET/CT studies independently. Per scan, a maximum of 5 target lesions was selected by the observers, and a PSMA-RADS score for every target lesion was recorded. No specific preexisting conditions were placed on the selection of the target lesions, although PSMA-RADS 1.0 suggests that readers focus on the most avid or largest lesions. An overall scan impression based on PSMA-RADS was indicated, and interobserver agreement rates on a target lesion-based, on an organ-based, and on an overall PSMA-RADS score-based level were computed. Results: The number of target lesions identified by each observer was as follows: ER 1, 123; ER 2, 134; IR 1, 123; and IR 2, 120. Among those selected target lesions, 125 were chosen by at least 2 individual observers (all 4 readers selected the same target lesion in 58 of 125 [46.4%] instances, 3 readers in 40 of 125 [32%], and 2 observers in 27 of 125 [21.6%]). The interobserver agreement for PSMA-RADS scoring among identical target lesions was good (intraclass correlation coefficient [ICC] for 4, 3, and 2 identical target lesions, ≥0.60, respectively). For lymph nodes, an excellent interobserver agreement was derived (ICC, 0.79). The interobserver agreement for an overall scan impression based on PSMA-RADS was also excellent (ICC, 0.84), with a significant difference for ER (ICC, 0.97) vs. IR (ICC, 0.74) (P = 0.005). Conclusion: PSMA-RADS demonstrated a high concordance rate in this study, even among readers with different levels of experience. This finding suggests that PSMA-RADS can be effectively used for communication with clinicians and can be implemented in the collection of data for large prospective trials.

Dataset Information

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms.

Publications

Interobserver Agreement between On-Call Radiology Resident and General Radiologist Interpretations of CT Pulmonary Angiograms and CT Venograms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets