Reproducibility and Interobserver Agreement of Different Thyroid Imaging and Reporting Data Systems (TIRADS).
Ontology highlight
ABSTRACT: Introduction
A number of classification systems (TIRADS) have been developed to estimate the likelihood of malignancy in thyroid nodules, but their reproducibility is yet to be assessed. We evaluated the interobserver variability and diagnostic performance of the TIRADS in Kwak's modification (Kw-TIRADS) and European TIRADS (EU-TIRADS).Methods
Two independent specialists, blinded concerning the morphology of the nodules, evaluated ultrasound images of 153 thyroid nodules identified in 149 patients at multiple time points.Results
The interobserver agreement (Cohen's κ) was 0.52 and 0.67 for Kw-TIRADS and EU-TIRADS, respectively, and rated as substantial. There were strong correlations between Kw-TIRADS and EU-TIRADS for the two observers with Spearman's coefficients of 0.731 (p = 0.00025) and 0.661 (p = 0.0012), respectively. Sensitivity of Kw-TIRADS for the diagnosis of thyroid cancer was 95-92.31% and that of EU-TIRADS was 92.31-89.74%, with specificity of about 60% for both TIRADS.Conclusion
Despite the wide variability in the description of single ultrasonographic features, both Kw-TIRADS and EU-TIRADS may be a useful diagnostic tool in clinical practice.
SUBMITTER: Sych YP
PROVIDER: S-EPMC8077651 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Ultrasonography is the best available tool for the initial work-up of thyroid nodules. Substantial interobserver variability has been documented in the recognition and reporting of some of the lesion characteristics. A number of classification systems have been developed to estimate the likelihood of malignancy: several of them have been endorsed by scientific societies, but their reproducibility is yet to be assessed. We evaluated the interobserver variability of the AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS classification systems and the interobserver concordance in the indication to FNA biopsy. Two raters independently evaluated 1055 ultrasound images of thyroid nodules identified in 265 patients at multiple time points, in two separate sets (501 and 554 images). After the first set of nodules, a joint reading was performed to reach a consensus in the feature definitions. The interobserver agreement (Krippendorff alpha) in the first set of nodules was 0.47, 0.49, 0.49, 0.61 and 0.53, for AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS systems, respectively. The agreement for the indication to biopsy was substantial to near-perfect, being 0.73, 0.61, 0.75, 0.68 and 0.82, respectively (Cohen's kappa). For all systems, agreement on the nodules of the second set increased. Despite the wide variability in the description of single ultrasonographic features, the classification systems may improve the interobserver agreement that further ameliorates after a specific training. When selecting nodules to be submitted to FNA biopsy, that is main purpose of these classifications, the interobserver agreement is substantial to almost perfect.
| S-EPMC5744624 | biostudies-literature
Project description:The aim was to compare the usefulness of selected thyroid sonographic risk-stratification systems in the diagnostics of nodules with indeterminate/suspicious cytology or unequivocal cytology in a population with a history of iodine deficiency. The diagnostic efficacy of ACR-TIRADS (the American College of Radiology Thyroid Imaging Reporting and Data Systems), EU-TIRADS (European Thyroid Association TIRADS), Korean-TIRADS, Kwak-TIRADS, AACE/ACE-AME-guidelines (American Association of Clinical Endocrinologists/ American College of Endocrinology-Associazione Medici Endocrinologi guidelines) and ATA-guidelines (American Thyroid Association guidelines) was evaluated in 1000 nodules with determined histopathological diagnosis: 329 FLUS/AUS (10.6% cancers), 167 SFN/SHT (11.6% cancers), 44 SM (77.3% cancers), 298 BL (benign lesions), 162 MN (malignant neoplasms). The percentage of PTC (papillary thyroid carcinoma) among cancers was higher in Bethesda MN (86.4%) and SM (suspicion of malignancy) nodules (91.2%) than in FLUS/AUS (57.1%, p < 0.005) and SFN/SHT (suspicion of follicular neoplasm/ suspicion of Hürthle cell tumor) nodules (36.8%, p < 0.001). TIRADS efficacy was higher for MN (AUC: 0.827-0.874) and SM nodules (AUC: 0.775-0.851) than for FLUS/AUS (AUC: 0.655-0.701) or SFN/SHT nodules (AUC: 0.593-0.621). FLUS/AUS (follicular lesion of undetermined significance/ atypia of undetermined significance) nodules assigned to a high risk TIRADS category had malignancy risk of 25%. In the SFN/SHT subgroup none TIRADS category changed nodule's malignancy risk. EU-TIRADS and AACE/ACE-AME-guidelines would allow diagnosing the highest number of PTC, FTC (follicular thyroid carcinoma), HTC (Hürthle cell carcinoma), MTC (medullary thyroid carcinoma). The highest OR value was for Kwak-TIRADS (12.6) and Korean-TIRADS (12.0). Conclusions: TIRADS efficacy depends on the incidence of PTC among cancers. All evaluated TIRADS facilitate the selection of FLUS/AUS nodules for the surgical treatment but these systems are not efficient in the management of SFN/SHT nodules.
| S-EPMC7408998 | biostudies-literature
Project description:Due to the widespread use of ultrasound, small thyroid nodules (TNs) ? 10 mm are common findings. Standardized approaches for the risk stratification of TNs with Thyroid Imaging Reporting and Data Systems (TIRADS) were evaluated for the clinical routine. With TIRADS, the risk of malignancy in TNs is calculated by scoring the number or combination of suspicious ultrasound features, leading to recommendations for further diagnostic steps. However, there are only scarce data on the performance of TIRADS for small TNs. The aim was to compare three different TIRADS for risk stratification of small TNs in routine clinical practice. We conducted a retrospective cohort analysis of TNs ? 10 mm and their available histology. Nodules were classified according to three different TIRADS. In the study, 140 patients (n = 113 female) with 145 thyroid nodules (n = 76 malignant) were included. Most of the malignant nodules were papillary carcinoma (97%), and the remaining 3% were medullary carcinoma. For all tested TIRADS, the prevalence of malignancy rose with increasing category levels. The highest negative predictive value was found for ACR TI-RADS and the highest positive predictive value for Kwak-TIRADS. All tested variants of TIRADS showed comparable diagnostic performance for the risk stratification of small TNs. TIRADS seems to be a promising tool to reliably assess the risk of malignancy of small TNs.
| S-EPMC7019412 | biostudies-literature
Project description:PurposeTo explore the characteristics of C-TIRADS by comparing it with ACR-TIRADS, Kwak-TIRADS, KSThR-TIRADS and EU-TIRADS.MethodsA total of 1096 nodules were collected from 884 patients undergoing thyroidectomy in our center between May 2018 and December 2020. Divided the nodules into two groups: ">10mm" and "≤10mm". Ultrasound characteristics of each nodule were observed and recorded by 2 doctors, then classified based on ACR-TIRADS, Kwak-TIRADS, KSThR-TIRADS, EU-TIRADS, and C-TIRADS.ResultsA total of 682 benign nodules cases (62.23%) and 414 malignant nodules cases (37.77%) were identified. The ICC value of each guideline was:0.937(ACR-TIRADS), 0.858(EU-IRADS), 0.811(Kwak-TIRADS), 0.835(KTA/KSThR-TIRADS) and 0.854(C-TIRADS). The nodule malignancy rates in the groups(Kwak-TIRADS 4B, C-TIRADS 4B、4C) of two sizes were significantly different (all p<0.05). There was no statistical difference in the other grades of two sizes (all p>0.05). Unnecessary biopsy rates were the lowest in C-TIRADS (49.02% p<0.001). Furthermore, Kwak-TIRADS had the highest sensitivity and NPV (89.9%, 91.0%, all p<0.05), while C-TIRADS had the highest specificity and PPV (82.3%, 69.2%, all p<0.05). C-TIRADS and Kwak-TIRADS had the highest accuracy (76.0%, 72.5%, P=0.071). The AUCs of the 5 guidelines were C-TIRADS(0.816, P<0.05), Kwak-TIRADS(0.789, P<0.05) KTA/KSThR-TIRADS and ACR-TIRADS(0.773, 0.763, P=0.305), EU-TIRADS(0.734, P<0.05). The AUCs of the five guidelines were not statistically different between "nodules>10mm" and "nodules ≤ 10mm" (all P>0.05).ConclusionsAll five guides showed excellent interobserver agreement. C-TIRADS was slightly efficient than Kwak-IRADS, KTA/KSThR-TIRADS and ACR-TIRADS, and had greater advantages than EU-TIRADS. The diagnostic abilities of the five guidelines for "nodules ≤ 10mm" were not inferior to that of "nodules> 10mm". C-TIRADS is simple and easy to implement and can provide effective thyroid tumor risk stratification for thyroid nodule diagnosis, especially in China.
| S-EPMC8578891 | biostudies-literature
Project description:Background The incidence of thyroid lumps is more and more high in population, and most biopsies of thyroid nodules are benign. To develop a practical risk stratification system based on five ultrasound features to stratify the malignancy risk of thyroid neoplasms. Methods This retrospective investigation enrolled 999 consecutive patients with 1,236 thyroid nodules who underwent ultrasound screening. Fine-needle aspiration and/or surgery was performed, and pathology results were obtained at the Seventh Affiliated Hospital of Sun Yat-sen University in Shenzhen, China, which is a tertiary referral center, from May 2018 to February 2022. Each thyroid nodule’s score was calculated based on five ultrasound features: composition, echogenicity, shape, margin, and echogenic foci. Additionally, each nodule’s malignancy rate was calculated. The chi-square test was used to test whether the malignancy rate was different among the three subcategories (scores of 4–6, 7–8, and 9 or more) of thyroid nodules. We proposed the revised Thyroid Imaging Reporting and Data System (R-TIRADS), and its sensitivity and specificity were compared to the two existing systems [the American College of Radiology TIRADS (ACR TIRADS) and the Korean Society of Thyroid Radiology TIRADS (K-TIRADS)]. Results The final dataset consisted of 425 nodules from 370 patients. The malignancy rates of three subcategories [malignancy rate: 28.8% (scores from 4–6), 64.7% (scores from 7–8), and 84.2% (scores of 9 or more)] were significantly different (P<0.01). The unnecessary biopsy rates of the three systems (ACR TIRADS, R-TIRADS, and K-TIRADS) were 28.7%, 25.2%, and 14.8%, respectively. The R-TIRADS presented better diagnostic performance than the ACR TIRADS or K-TIRADS [area under the curve: 0.79 (95% CI: 0.74–0.83) vs. 0.69 (95% CI: 0.64–0.75), P=0.046; 0.79 (95% CI: 0.74–0.83) vs. 0.66 (95% CI: 0.60–0.71), P=0.041, respectively]. The R-TIRADS had the highest sensitivity [0.746 (95% CI: 0.689–0.803)], followed by the K-TIRADS [0.399 (95% CI: 0.335–0.463), P=0.000] and ACR TIRADS [0.377 (95% CI: 0.314–0.441), P=0.000]. Conclusions The R-TIRADS enables radiologists to diagnose thyroid nodules efficiently, and the number of unnecessary fine-needle aspirations can be considerably reduced.
| S-EPMC10240023 | biostudies-literature
Project description:PurposeTo evaluate accuracy and interobserver variability with the use of the Prostate Imaging Reporting and Data System (PI-RADS) version 2.0 for detection of prostate cancer at multiparametric magnetic resonance (MR) imaging in a biopsy-naïve patient population.Materials and methodsThis retrospective HIPAA-compliant study was approved by the local ethics committee, and written informed consent was obtained from all patients for use of their imaging and histopathologic data in future research studies. In 101 biopsy-naïve patients with elevated prostate-specific antigen levels who underwent multiparametric MR imaging of the prostate and subsequent transrectal ultrasonography (US)-MR imaging fusion-guided biopsy, suspicious lesions detected at multiparametric MR imaging were scored by five readers who were blinded to pathologic results by using to the newly revised PI-RADS and the scoring system developed in-house. Interobserver agreement was evaluated by using κ statistics, and the correlation of pathologic results with each of the two scoring systems was evaluated by using the Kendall τ correlation coefficient.ResultsSpecimens of 162 lesions in 94 patients were sampled by means of transrectal US-MR imaging fusion biopsy. Results for 87 (54%) lesions were positive for prostate cancer. Kendall τ values with the PI-RADS and the in-house-developed scoring system, respectively, at T2-weighted MR imaging in the peripheral zone were 0.51 and 0.17 and in the transitional zone, 0.45 and -0.11; at diffusion-weighted MR imaging, 0.42 and 0.28; at dynamic contrast material-enhanced MR imaging, 0.23 and 0.24, and overall suspicion scores were 0.42 and 0.49. Median κ scores among all possible pairs of readers for PI-RADS and the in-house-developed scoring system, respectively, for T2-weighted MR images in the peripheral zone were 0.47 and 0.15; transitional zone, 0.37 and 0.07; diffusion-weighted MR imaging, 0.41 and 0.57; dynamic contrast-enhanced MR imaging, 0.48 and 0.41; and overall suspicion scores, 0.46 and 0.55.ConclusionUse of the revised PI-RADS provides moderately reproducible MR imaging scores for detection of clinically relevant disease.
| S-EPMC4666087 | biostudies-literature
Project description:The Paris System for Reporting Urinary Cytology represents a significant improvement in classification of urinary specimens. The system acknowledges the difficulty in cytologically diagnosing low-grade urothelial carcinomas and has developed categories to deal with this issue. The system uses six categories: unsatisfactory, negative for high-grade urothelial carcinoma (NHGUC), atypical urothelial cells, suspicious for high-grade urothelial carcinoma, high-grade urothelial carcinoma, other malignancies and a seventh subcategory (low-grade urothelial neoplasm).Three hundred and fifty-seven urine specimens were independently reviewed by four cytopathologists unaware of the previous diagnoses. Each cytopathologist rendered a diagnosis according to the Paris System categories. Agreement was assessed using absolute agreement and weighted chance-corrected agreement (kappa). Disagreements were classified as low impact and high impact based on the potential impact of a misclassification on clinical management.The average absolute agreement was 65% with an average expected agreement of 44%. The average chance-corrected agreement (kappa) was 0.32. Nine hundred and ninety-nine of 1902 comparisons between rater pairs were in agreement, but 12% of comparisons differed by two or more categories for the category NHGUC. Approximately 15% of the disagreements were classified as high clinical impact.Our findings indicated that the scheme recommended by the Paris System shows adequate precision for the category NHGUC, but the other categories demonstrated unacceptable interobserver variability. This low level of diagnostic precision may negatively impact the applicability of the Paris System for widespread clinical application.
| S-EPMC5545779 | biostudies-other
Project description:Recently, the standardized reporting and data system for prostate-specific membrane antigen (PSMA)-targeted PET imaging studies, termed PSMA-RADS version 1.0, was introduced. We aimed to determine the interobserver agreement for applying PSMA-RADS to imaging interpretation of 18F-DCFPyL (2-(3-{1-carboxy-5-[(6-18F-fluoro-pyridine-3-carbonyl)-amino]-pentyl}-ureido)-pentanedioic acid) PET examinations in a prospective setting mimicking the typical clinical workflow at a prostate cancer referral center. Methods: Four readers (2 experienced readers (ERs, >3 y of PSMA-targeted PET interpretation experience) and 2 inexperienced readers (IRs, <1 y of experience)), who had all read the initial publication on PSMA-RADS 1.0, assessed 50 18F-DCFPyL PET/CT studies independently. Per scan, a maximum of 5 target lesions was selected by the observers, and a PSMA-RADS score for every target lesion was recorded. No specific preexisting conditions were placed on the selection of the target lesions, although PSMA-RADS 1.0 suggests that readers focus on the most avid or largest lesions. An overall scan impression based on PSMA-RADS was indicated, and interobserver agreement rates on a target lesion-based, on an organ-based, and on an overall PSMA-RADS score-based level were computed. Results: The number of target lesions identified by each observer was as follows: ER 1, 123; ER 2, 134; IR 1, 123; and IR 2, 120. Among those selected target lesions, 125 were chosen by at least 2 individual observers (all 4 readers selected the same target lesion in 58 of 125 [46.4%] instances, 3 readers in 40 of 125 [32%], and 2 observers in 27 of 125 [21.6%]). The interobserver agreement for PSMA-RADS scoring among identical target lesions was good (intraclass correlation coefficient [ICC] for 4, 3, and 2 identical target lesions, ≥0.60, respectively). For lymph nodes, an excellent interobserver agreement was derived (ICC, 0.79). The interobserver agreement for an overall scan impression based on PSMA-RADS was also excellent (ICC, 0.84), with a significant difference for ER (ICC, 0.97) vs. IR (ICC, 0.74) (P = 0.005). Conclusion: PSMA-RADS demonstrated a high concordance rate in this study, even among readers with different levels of experience. This finding suggests that PSMA-RADS can be effectively used for communication with clinicians and can be implemented in the collection of data for large prospective trials.
| S-EPMC6278904 | biostudies-literature
Project description:AimTo assess the role of imaging in the early management of encephalitis and the agreement on findings in a well-defined cohort of suspected encephalitis cases enrolled in the Prospective Aetiological Study of Encephalitis conducted by the Health Protection Agency (now incorporated into Public Health England).Materials and methodsEighty-five CT examinations from 68 patients and 101 MRI examinations from 80 patients with suspected encephalitis were independently rated by three neuroradiologists blinded to patient and clinical details. The level of agreement on the interpretation of images was measured using the kappa statistic. The sensitivity, specificity, and negative and positive predictive values of CT and MRI for herpes simplex virus (HSV) encephalitis and acute disseminated encephalomyelitis (ADEM) were estimated.ResultsThe kappa value for interobserver agreement on rating the scans as normal or abnormal was good (0.65) for CT and moderate (0.59) for MRI. Agreement for HSV encephalitis was very good for CT (0.87) and MRI (0.82), but only fair for ADEM (0.32 CT; 0.31 MRI). Similarly, the overall sensitivity of imaging for HSV encephalitis was ∼80% for both CT and MRI, whereas for ADEM it was 0% for CT and 20% for MRI. MRI specificity for HSV encephalitis between 3-10 days after symptom onset was 100%.ConclusionThere is a subjective component to scan interpretation that can have important implications for the clinical management of encephalitis cases. Neuroradiologists were good at diagnosing HSV encephalitis; however, agreement was worse for ADEM and other alternative aetiologies. Findings highlight the importance of a comprehensive and multidisciplinary approach to diagnosing the cause of encephalitis that takes into account individual clinical, microbiological, and radiological features of each patient.
| S-EPMC5021199 | biostudies-literature
Project description:IntroductionPlacental pathology is key for investigating adverse pregnancy outcomes, however, lack of standardization in reporting has limited clinical utility. We evaluated a novel placental pathology synoptic report, comparing its robustness to narrative reports, and assessed interobserver agreement.Methods100 singleton placentas were included. Histology slides were examined by 2 senior perinatal pathologists and 2 pathology residents using a synoptic report (32 lesions). Historical narrative reports were compared to synoptic reports. Kappa scores were calculated for interobserver agreement between senior, resident, and senior vs resident pathologists.ResultsSynoptic reporting detected 169 (51.4%) lesion instances initially not included in historical reports. Amongst senior pathologists, 64% of all lesions examined demonstrated fair-to-excellent agreement (Kappa ≥0.41), with only 26% of Kappas ≥0.41 amongst those examined by resident pathologists. Well-characterized lesions (e.g., chorioamnionitis) demonstrated higher agreement, with lower agreement for uncommon lesions and those previously shown to have poor consensus.DiscussionSynoptic reporting is one proposed method to address issues in placenta pathology reporting. The synoptic report generally identifies more lesions compared to the narrative report, however clinical significance remains unclear. Interobserver agreement is likely related to differential in experience. Further efforts to improve overall standardization of placenta pathology reporting are needed.
| S-EPMC10559645 | biostudies-literature