Dataset Information

Interobserver agreement of various thyroid imaging reporting and data systems.

ABSTRACT: Ultrasonography is the best available tool for the initial work-up of thyroid nodules. Substantial interobserver variability has been documented in the recognition and reporting of some of the lesion characteristics. A number of classification systems have been developed to estimate the likelihood of malignancy: several of them have been endorsed by scientific societies, but their reproducibility is yet to be assessed. We evaluated the interobserver variability of the AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS classification systems and the interobserver concordance in the indication to FNA biopsy. Two raters independently evaluated 1055 ultrasound images of thyroid nodules identified in 265 patients at multiple time points, in two separate sets (501 and 554 images). After the first set of nodules, a joint reading was performed to reach a consensus in the feature definitions. The interobserver agreement (Krippendorff alpha) in the first set of nodules was 0.47, 0.49, 0.49, 0.61 and 0.53, for AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS systems, respectively. The agreement for the indication to biopsy was substantial to near-perfect, being 0.73, 0.61, 0.75, 0.68 and 0.82, respectively (Cohen's kappa). For all systems, agreement on the nodules of the second set increased. Despite the wide variability in the description of single ultrasonographic features, the classification systems may improve the interobserver agreement that further ameliorates after a specific training. When selecting nodules to be submitted to FNA biopsy, that is main purpose of these classifications, the interobserver agreement is substantial to almost perfect.

SUBMITTER: Grani G

PROVIDER: S-EPMC5744624 | biostudies-literature | 2018 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interobserver agreement of various thyroid imaging reporting and data systems.

Grani Giorgio G Lamartina Livia L Cantisani Vito V Maranghi Marianna M Lucia Piernatale P Durante Cosimo C

Endocrine connections 20171201 1

Ultrasonography is the best available tool for the initial work-up of thyroid nodules. Substantial interobserver variability has been documented in the recognition and reporting of some of the lesion characteristics. A number of classification systems have been developed to estimate the likelihood of malignancy: several of them have been endorsed by scientific societies, but their reproducibility is yet to be assessed. We evaluated the interobserver variability of the AACE/ACE/AME, ACR, ATA, EU- ...[more]

PMID: 29196301

Similar Datasets

Project description:PurposeTo evaluate accuracy and interobserver variability with the use of the Prostate Imaging Reporting and Data System (PI-RADS) version 2.0 for detection of prostate cancer at multiparametric magnetic resonance (MR) imaging in a biopsy-naïve patient population.Materials and methodsThis retrospective HIPAA-compliant study was approved by the local ethics committee, and written informed consent was obtained from all patients for use of their imaging and histopathologic data in future research studies. In 101 biopsy-naïve patients with elevated prostate-specific antigen levels who underwent multiparametric MR imaging of the prostate and subsequent transrectal ultrasonography (US)-MR imaging fusion-guided biopsy, suspicious lesions detected at multiparametric MR imaging were scored by five readers who were blinded to pathologic results by using to the newly revised PI-RADS and the scoring system developed in-house. Interobserver agreement was evaluated by using κ statistics, and the correlation of pathologic results with each of the two scoring systems was evaluated by using the Kendall τ correlation coefficient.ResultsSpecimens of 162 lesions in 94 patients were sampled by means of transrectal US-MR imaging fusion biopsy. Results for 87 (54%) lesions were positive for prostate cancer. Kendall τ values with the PI-RADS and the in-house-developed scoring system, respectively, at T2-weighted MR imaging in the peripheral zone were 0.51 and 0.17 and in the transitional zone, 0.45 and -0.11; at diffusion-weighted MR imaging, 0.42 and 0.28; at dynamic contrast material-enhanced MR imaging, 0.23 and 0.24, and overall suspicion scores were 0.42 and 0.49. Median κ scores among all possible pairs of readers for PI-RADS and the in-house-developed scoring system, respectively, for T2-weighted MR images in the peripheral zone were 0.47 and 0.15; transitional zone, 0.37 and 0.07; diffusion-weighted MR imaging, 0.41 and 0.57; dynamic contrast-enhanced MR imaging, 0.48 and 0.41; and overall suspicion scores, 0.46 and 0.55.ConclusionUse of the revised PI-RADS provides moderately reproducible MR imaging scores for detection of clinically relevant disease.

Project description:The aim was to compare the usefulness of selected thyroid sonographic risk-stratification systems in the diagnostics of nodules with indeterminate/suspicious cytology or unequivocal cytology in a population with a history of iodine deficiency. The diagnostic efficacy of ACR-TIRADS (the American College of Radiology Thyroid Imaging Reporting and Data Systems), EU-TIRADS (European Thyroid Association TIRADS), Korean-TIRADS, Kwak-TIRADS, AACE/ACE-AME-guidelines (American Association of Clinical Endocrinologists/ American College of Endocrinology-Associazione Medici Endocrinologi guidelines) and ATA-guidelines (American Thyroid Association guidelines) was evaluated in 1000 nodules with determined histopathological diagnosis: 329 FLUS/AUS (10.6% cancers), 167 SFN/SHT (11.6% cancers), 44 SM (77.3% cancers), 298 BL (benign lesions), 162 MN (malignant neoplasms). The percentage of PTC (papillary thyroid carcinoma) among cancers was higher in Bethesda MN (86.4%) and SM (suspicion of malignancy) nodules (91.2%) than in FLUS/AUS (57.1%, p < 0.005) and SFN/SHT (suspicion of follicular neoplasm/ suspicion of Hürthle cell tumor) nodules (36.8%, p < 0.001). TIRADS efficacy was higher for MN (AUC: 0.827-0.874) and SM nodules (AUC: 0.775-0.851) than for FLUS/AUS (AUC: 0.655-0.701) or SFN/SHT nodules (AUC: 0.593-0.621). FLUS/AUS (follicular lesion of undetermined significance/ atypia of undetermined significance) nodules assigned to a high risk TIRADS category had malignancy risk of 25%. In the SFN/SHT subgroup none TIRADS category changed nodule's malignancy risk. EU-TIRADS and AACE/ACE-AME-guidelines would allow diagnosing the highest number of PTC, FTC (follicular thyroid carcinoma), HTC (Hürthle cell carcinoma), MTC (medullary thyroid carcinoma). The highest OR value was for Kwak-TIRADS (12.6) and Korean-TIRADS (12.0). Conclusions: TIRADS efficacy depends on the incidence of PTC among cancers. All evaluated TIRADS facilitate the selection of FLUS/AUS nodules for the surgical treatment but these systems are not efficient in the management of SFN/SHT nodules.

Project description:Recently, the standardized reporting and data system for prostate-specific membrane antigen (PSMA)-targeted PET imaging studies, termed PSMA-RADS version 1.0, was introduced. We aimed to determine the interobserver agreement for applying PSMA-RADS to imaging interpretation of 18F-DCFPyL (2-(3-{1-carboxy-5-[(6-18F-fluoro-pyridine-3-carbonyl)-amino]-pentyl}-ureido)-pentanedioic acid) PET examinations in a prospective setting mimicking the typical clinical workflow at a prostate cancer referral center. Methods: Four readers (2 experienced readers (ERs, >3 y of PSMA-targeted PET interpretation experience) and 2 inexperienced readers (IRs, <1 y of experience)), who had all read the initial publication on PSMA-RADS 1.0, assessed 50 18F-DCFPyL PET/CT studies independently. Per scan, a maximum of 5 target lesions was selected by the observers, and a PSMA-RADS score for every target lesion was recorded. No specific preexisting conditions were placed on the selection of the target lesions, although PSMA-RADS 1.0 suggests that readers focus on the most avid or largest lesions. An overall scan impression based on PSMA-RADS was indicated, and interobserver agreement rates on a target lesion-based, on an organ-based, and on an overall PSMA-RADS score-based level were computed. Results: The number of target lesions identified by each observer was as follows: ER 1, 123; ER 2, 134; IR 1, 123; and IR 2, 120. Among those selected target lesions, 125 were chosen by at least 2 individual observers (all 4 readers selected the same target lesion in 58 of 125 [46.4%] instances, 3 readers in 40 of 125 [32%], and 2 observers in 27 of 125 [21.6%]). The interobserver agreement for PSMA-RADS scoring among identical target lesions was good (intraclass correlation coefficient [ICC] for 4, 3, and 2 identical target lesions, ≥0.60, respectively). For lymph nodes, an excellent interobserver agreement was derived (ICC, 0.79). The interobserver agreement for an overall scan impression based on PSMA-RADS was also excellent (ICC, 0.84), with a significant difference for ER (ICC, 0.97) vs. IR (ICC, 0.74) (P = 0.005). Conclusion: PSMA-RADS demonstrated a high concordance rate in this study, even among readers with different levels of experience. This finding suggests that PSMA-RADS can be effectively used for communication with clinicians and can be implemented in the collection of data for large prospective trials.

Project description:Recently, a standardized framework system for interpreting somatostatin receptor (SSTR)-targeted PET/CT, termed the SSTR reporting and data system (RADS) 1.0, was introduced, providing reliable standards and criteria for SSTR-targeted imaging. We determined the interobserver reliability of SSTR-RADS for interpretation of 68Ga-DOTATOC PET/CT scans in a multicentric, randomized setting. Methods: A set of 51 randomized 68Ga-DOTATOC PET/CT scans was independently assessed by 4 masked readers with different levels of experience (2 experienced readers and 2 inexperienced readers) trained on the SSTR-RADS 1.0 criteria (based on a 5-point scale from 1 [definitively benign] to 5 [high certainty that neuroendocrine neoplasia is present]). For each scan, SSTR-RADS scores were assigned to a maximum of 5 target lesions (TLs). An overall scan impression based on SSTR-RADS was indicated, and interobserver agreement rates on a TL-based, on an organ-based, and on an overall SSTR-RADS score-based level were computed. The readers were also asked to decide whether peptide receptor radionuclide therapy (PRRT) should be considered on the basis of the assigned RADS scores. Results: Among the selected TLs, 153 were chosen by at least 2 readers (all 4 readers selected the same TLs in 58 of 153 [37.9%] instances). The interobserver agreement for SSTR-RADS scoring among identical TLs was good (intraclass correlation coefficient [ICC] ≥ 0.73 for 4, 3, and 2 identical TLs). For lymph node and liver lesions, excellent interobserver agreement rates were derived (ICC, 0.91 and 0.77, respectively). Moreover, the interobserver agreement for an overall scan impression based on SSTR-RADS was excellent (ICC, 0.88). The SSTR-RADS-based decision to use PRRT also demonstrated excellent agreement, with an ICC of 0.80. No significant differences between experienced and inexperienced readers for an overall scan impression and TL-based SSTR-RADS scoring were observed (P ≥ 0.18), thereby suggesting that SSTR-RADS seems to be readily applicable even for less experienced readers. Conclusion: SSTR-RADS-guided assessment demonstrated a high concordance rate, even among readers with different levels of experience, supporting the adoption of SSTR-RADS for trials, clinical routine, or outcome studies.

Dataset Information

Interobserver agreement of various thyroid imaging reporting and data systems.

Publications

Interobserver agreement of various thyroid imaging reporting and data systems.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets