Interobserver reproducibility of The Paris System for Reporting Urinary Cytology.
Ontology highlight
ABSTRACT:
SUBMITTER: Long T
PROVIDER: S-EPMC5545779 | biostudies-other | 2017
REPOSITORIES: biostudies-other
Similar Datasets
Project description:IntroductionA number of classification systems (TIRADS) have been developed to estimate the likelihood of malignancy in thyroid nodules, but their reproducibility is yet to be assessed. We evaluated the interobserver variability and diagnostic performance of the TIRADS in Kwak's modification (Kw-TIRADS) and European TIRADS (EU-TIRADS).MethodsTwo independent specialists, blinded concerning the morphology of the nodules, evaluated ultrasound images of 153 thyroid nodules identified in 149 patients at multiple time points.ResultsThe interobserver agreement (Cohen's κ) was 0.52 and 0.67 for Kw-TIRADS and EU-TIRADS, respectively, and rated as substantial. There were strong correlations between Kw-TIRADS and EU-TIRADS for the two observers with Spearman's coefficients of 0.731 (p = 0.00025) and 0.661 (p = 0.0012), respectively. Sensitivity of Kw-TIRADS for the diagnosis of thyroid cancer was 95-92.31% and that of EU-TIRADS was 92.31-89.74%, with specificity of about 60% for both TIRADS.ConclusionDespite the wide variability in the description of single ultrasonographic features, both Kw-TIRADS and EU-TIRADS may be a useful diagnostic tool in clinical practice.
| S-EPMC8077651 | biostudies-literature
Project description:Recently, the standardized reporting and data system for prostate-specific membrane antigen (PSMA)-targeted PET imaging studies, termed PSMA-RADS version 1.0, was introduced. We aimed to determine the interobserver agreement for applying PSMA-RADS to imaging interpretation of 18F-DCFPyL (2-(3-{1-carboxy-5-[(6-18F-fluoro-pyridine-3-carbonyl)-amino]-pentyl}-ureido)-pentanedioic acid) PET examinations in a prospective setting mimicking the typical clinical workflow at a prostate cancer referral center. Methods: Four readers (2 experienced readers (ERs, >3 y of PSMA-targeted PET interpretation experience) and 2 inexperienced readers (IRs, <1 y of experience)), who had all read the initial publication on PSMA-RADS 1.0, assessed 50 18F-DCFPyL PET/CT studies independently. Per scan, a maximum of 5 target lesions was selected by the observers, and a PSMA-RADS score for every target lesion was recorded. No specific preexisting conditions were placed on the selection of the target lesions, although PSMA-RADS 1.0 suggests that readers focus on the most avid or largest lesions. An overall scan impression based on PSMA-RADS was indicated, and interobserver agreement rates on a target lesion-based, on an organ-based, and on an overall PSMA-RADS score-based level were computed. Results: The number of target lesions identified by each observer was as follows: ER 1, 123; ER 2, 134; IR 1, 123; and IR 2, 120. Among those selected target lesions, 125 were chosen by at least 2 individual observers (all 4 readers selected the same target lesion in 58 of 125 [46.4%] instances, 3 readers in 40 of 125 [32%], and 2 observers in 27 of 125 [21.6%]). The interobserver agreement for PSMA-RADS scoring among identical target lesions was good (intraclass correlation coefficient [ICC] for 4, 3, and 2 identical target lesions, ?0.60, respectively). For lymph nodes, an excellent interobserver agreement was derived (ICC, 0.79). The interobserver agreement for an overall scan impression based on PSMA-RADS was also excellent (ICC, 0.84), with a significant difference for ER (ICC, 0.97) vs. IR (ICC, 0.74) (P = 0.005). Conclusion: PSMA-RADS demonstrated a high concordance rate in this study, even among readers with different levels of experience. This finding suggests that PSMA-RADS can be effectively used for communication with clinicians and can be implemented in the collection of data for large prospective trials.
| S-EPMC6278904 | biostudies-literature
Project description:Recently, a standardized framework system for interpreting somatostatin receptor (SSTR)-targeted PET/CT, termed the SSTR reporting and data system (RADS) 1.0, was introduced, providing reliable standards and criteria for SSTR-targeted imaging. We determined the interobserver reliability of SSTR-RADS for interpretation of 68Ga-DOTATOC PET/CT scans in a multicentric, randomized setting. Methods: A set of 51 randomized 68Ga-DOTATOC PET/CT scans was independently assessed by 4 masked readers with different levels of experience (2 experienced readers and 2 inexperienced readers) trained on the SSTR-RADS 1.0 criteria (based on a 5-point scale from 1 [definitively benign] to 5 [high certainty that neuroendocrine neoplasia is present]). For each scan, SSTR-RADS scores were assigned to a maximum of 5 target lesions (TLs). An overall scan impression based on SSTR-RADS was indicated, and interobserver agreement rates on a TL-based, on an organ-based, and on an overall SSTR-RADS score-based level were computed. The readers were also asked to decide whether peptide receptor radionuclide therapy (PRRT) should be considered on the basis of the assigned RADS scores. Results: Among the selected TLs, 153 were chosen by at least 2 readers (all 4 readers selected the same TLs in 58 of 153 [37.9%] instances). The interobserver agreement for SSTR-RADS scoring among identical TLs was good (intraclass correlation coefficient [ICC] ≥ 0.73 for 4, 3, and 2 identical TLs). For lymph node and liver lesions, excellent interobserver agreement rates were derived (ICC, 0.91 and 0.77, respectively). Moreover, the interobserver agreement for an overall scan impression based on SSTR-RADS was excellent (ICC, 0.88). The SSTR-RADS-based decision to use PRRT also demonstrated excellent agreement, with an ICC of 0.80. No significant differences between experienced and inexperienced readers for an overall scan impression and TL-based SSTR-RADS scoring were observed (P ≥ 0.18), thereby suggesting that SSTR-RADS seems to be readily applicable even for less experienced readers. Conclusion: SSTR-RADS-guided assessment demonstrated a high concordance rate, even among readers with different levels of experience, supporting the adoption of SSTR-RADS for trials, clinical routine, or outcome studies.
| S-EPMC8049367 | biostudies-literature
Project description:BackgroundKi67 labeling index (LI) is used as a predictive marker and is associated with prognosis in breast cancer. However, standardised methodologies for measurement are lacking which has limited its application in clinical practice. In this study, we evaluated the interobserver concordance of visual assessment of Ki67 LI in breast cancer.MethodsKi67-immunostained slides of 160 cases of primary invasive breast cancer were visual assessed by five breast pathologists with two different methods to choose the scoring fields: (1) hot-spot score, (2) average score. Proportions of positive invasive tumor cells at 10 % intervals were scored. The intra-class correlation coefficient (ICC) was used to assess the interobserver reproducibility.Results(1) A perfect concordance of Ki67 LI was demonstrated according to both score methods (P<0.0001). Average score method (ICC, 0.904) demonstrated a better correlation than hot-spot score method (ICC, 0.894). (2) By respective means according to two score methods, all cases were classified into three groups (?10%, 11%-30% and >30% Ki-67 LI). The concordance was relatively low in intermediate Ki67 LI group compared with low and high Ki67 LI groups. (3) All cases were classified into three groups by paired-difference (d) between means of hot-spot score and average score (d<5, 5?d<10, d?10). The consistency was observed to decrease with increasing paired-difference according to both methods.ConclusionsVisual assessment of Ki67 LI at 10 % intervals is a candidate for a standard method in breast cancer clinical practice. Average score and hot-spot score of visual assessment both demonstrated a perfect concordance, and an overall average assessment across the whole section including hot spots may be a better method. Interobserver concordance of intermediate Ki67 LI in which most cutoffs are located for making clinical decisions was relatively low.
| S-EPMC4416820 | biostudies-literature
Project description:Clinical significance of the pT4 category in colon cancer is increasing with several therapeutic implications. The aim of this study was to evaluate variability in diagnosing pT4a colon cancer. Twelve pathologists classified 66 preselected scanned Hematoxylin/Eosin-stained slides with tumor cells at a distance of 25-1500 ?m (n?=?22), 0-25 ?m (n?=?22), or on (n?=?22) the peritoneal surface. Inter- and intraobserver variability were calculated using Kappa statistics. For interlaboratory variability, pathology reports of pT3 and pT4a colon cancer were extracted from the Dutch Pathology Registry between 2012 and 2015. The proportion of pT4a (pT4a/(pT3+pT4a)) was compared between 33 laboratories. Potential risk of understaging was assessed by determining the average number of blocks taken from pT3 and pT4a N0-2M0 tumors with metachronous peritoneal metastasis. Interobserver variability among 12 pathologists was 0.50 (95%CI 0.41-0.60; moderate agreement). Intraobserver variability (8 pathologists) was 0.71 (substantial agreement). A total of 7745 reports with pT3 or pT4aN0-2M0 colon cancer from 33 laboratories were included for interlaboratory analysis. Median percentage of pT4a was 15.5% (range 3.2-24.6%). After adjustment for case mix, 8 labs diagnosed pT4a significantly less or more frequently than the median lab. Metachronous peritoneal metastases were histologically verified in 170 of 6629 pT3 and in 129 of 1116 pT4a tumors, with a mean number of blocks of 4.03(SD 1.51) and 4.78 (SD 1.76) taken from the primary tumors, respectively (p?<?0.001). A substantial variability in diagnosing pT4a colon cancer exists, both at pathologist and laboratory level. Diagnosis of pT4a stage appears to be challenging and there is a need for standardizing assessment of this pathological entity.
| S-EPMC7028812 | biostudies-literature
Project description:Ultrasonography is the best available tool for the initial work-up of thyroid nodules. Substantial interobserver variability has been documented in the recognition and reporting of some of the lesion characteristics. A number of classification systems have been developed to estimate the likelihood of malignancy: several of them have been endorsed by scientific societies, but their reproducibility is yet to be assessed. We evaluated the interobserver variability of the AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS classification systems and the interobserver concordance in the indication to FNA biopsy. Two raters independently evaluated 1055 ultrasound images of thyroid nodules identified in 265 patients at multiple time points, in two separate sets (501 and 554 images). After the first set of nodules, a joint reading was performed to reach a consensus in the feature definitions. The interobserver agreement (Krippendorff alpha) in the first set of nodules was 0.47, 0.49, 0.49, 0.61 and 0.53, for AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS systems, respectively. The agreement for the indication to biopsy was substantial to near-perfect, being 0.73, 0.61, 0.75, 0.68 and 0.82, respectively (Cohen's kappa). For all systems, agreement on the nodules of the second set increased. Despite the wide variability in the description of single ultrasonographic features, the classification systems may improve the interobserver agreement that further ameliorates after a specific training. When selecting nodules to be submitted to FNA biopsy, that is main purpose of these classifications, the interobserver agreement is substantial to almost perfect.
| S-EPMC5744624 | biostudies-literature
Project description:PurposeTo examine repeatability and reproducibility of ellipsoid zone (EZ) width measurements in patients with retinitis pigmentosa (RP) using a longitudinal reflectivity profile (LRP) analysis.MethodsWe examined Bioptigen optical coherence tomography (OCT) scans from 48 subjects with RP or Usher syndrome. Nominal scan lengths were 6, 7, or 10 mm, and the lateral scale of each scan was calculated using axial length measurements. LRPs were generated from OCT line scans, and the peak corresponding to EZ was manually identified using ImageJ. The locations at which the EZ peak disappeared were used to calculate EZ width. Each scan was analyzed twice by each of two observers, who were masked to their previous measurements and those of the other observer.ResultsOn average, horizontal width (HW) was significantly greater than vertical width (VW), and there was high interocular symmetry for both HW and VW. We observed excellent intraobserver repeatability with intraclass correlation coefficients (ICCs) ranging from 0.996 to 0.998 for HW and VW measurements. Interobserver reproducibility was also excellent for both HW (ICC = 0.989; 95% confidence interval [CI] = 0.983-0.995) and VW (ICC = 0.991; 95% CI = 0.985-0.996), with no significant bias observed between observers.ConclusionsEZ width can be measured using LRPs with excellent repeatability and reproducibility. Our observation of greater HW than VW is consistent with previous observations in RP, though the reason for this anisotropy remains unclear.Translational relevanceWe describe repeatability and reproducibility of a method for measuring EZ width in patients with RP or Usher syndrome. This approach could facilitate measurement of retinal band thickness and/or intensity.
| S-EPMC5989764 | biostudies-literature
Project description:PurposeTo evaluate accuracy and interobserver variability with the use of the Prostate Imaging Reporting and Data System (PI-RADS) version 2.0 for detection of prostate cancer at multiparametric magnetic resonance (MR) imaging in a biopsy-naïve patient population.Materials and methodsThis retrospective HIPAA-compliant study was approved by the local ethics committee, and written informed consent was obtained from all patients for use of their imaging and histopathologic data in future research studies. In 101 biopsy-naïve patients with elevated prostate-specific antigen levels who underwent multiparametric MR imaging of the prostate and subsequent transrectal ultrasonography (US)-MR imaging fusion-guided biopsy, suspicious lesions detected at multiparametric MR imaging were scored by five readers who were blinded to pathologic results by using to the newly revised PI-RADS and the scoring system developed in-house. Interobserver agreement was evaluated by using κ statistics, and the correlation of pathologic results with each of the two scoring systems was evaluated by using the Kendall τ correlation coefficient.ResultsSpecimens of 162 lesions in 94 patients were sampled by means of transrectal US-MR imaging fusion biopsy. Results for 87 (54%) lesions were positive for prostate cancer. Kendall τ values with the PI-RADS and the in-house-developed scoring system, respectively, at T2-weighted MR imaging in the peripheral zone were 0.51 and 0.17 and in the transitional zone, 0.45 and -0.11; at diffusion-weighted MR imaging, 0.42 and 0.28; at dynamic contrast material-enhanced MR imaging, 0.23 and 0.24, and overall suspicion scores were 0.42 and 0.49. Median κ scores among all possible pairs of readers for PI-RADS and the in-house-developed scoring system, respectively, for T2-weighted MR images in the peripheral zone were 0.47 and 0.15; transitional zone, 0.37 and 0.07; diffusion-weighted MR imaging, 0.41 and 0.57; dynamic contrast-enhanced MR imaging, 0.48 and 0.41; and overall suspicion scores, 0.46 and 0.55.ConclusionUse of the revised PI-RADS provides moderately reproducible MR imaging scores for detection of clinically relevant disease.
| S-EPMC4666087 | biostudies-literature
Project description:PURPOSE:In-vivo quantification of tumor uptake of 89-zirconium (89Zr)-labelled monoclonal antibodies (mAbs) with PET provides a potential tool in strategies to optimize tumor targeting and therapeutic efficacy. A specific challenge for 89Zr-immuno-PET is low tumor contrast. This is expected to result in interobserver variation in tumor delineation. Therefore, the aim of this study was to determine interobserver reproducibility of tumor uptake measures by tumor delineation on 89Zr-immuno-PET scans. METHODS:Data were obtained from previously published clinical studies performed with 89Zr-rituximab, 89Zr-cetuximab and 89Zr-trastuzumab. Tumor lesions on 89Zr-immuno-PET were identified as focal uptake exceeding local background by a nuclear medicine physician. Three observers independently manually delineated volumes of interest (VOI). Maximum, peak and mean standardized uptake values (SUVmax, SUVpeak and SUVmean) were used to quantify tumor uptake. Interobserver variability was expressed as the coefficient of variation (CoV). The performance of semi-automatic VOI delineation using 50% of background-corrected ACpeak was described. RESULTS:In total, 103 VOI were delineated (3-6 days post injection (D3-D6)). Tumor uptake (median, interquartile range) was 9.2 (5.2-12.6), 6.9 (4.0-9.6) and 5.5 (3.3-7.8) for SUVmax, SUVpeak and SUVmean. Interobserver variability was 0% (0-12), 0% (0-2) and 7% (5-14), respectively (n?=?103). The success rate of the semi-automatic method was 45%. Inclusion of background was the main reason for failure of semi-automatic VOI. CONCLUSIONS:This study shows that interobserver reproducibility of tumor uptake quantification on 89Zr-immuno-PET was excellent for SUVmax and SUVpeak using a standardized manual procedure for tumor segmentation. Semi-automatic delineation was not robust due to limited tumor contrast.
| S-EPMC6647131 | biostudies-literature
Project description:Ribonucleic acid sequencing (RNA-seq) identifies and quantifies RNA molecules from a biological sample. Transformation from raw sequencing data to meaningful gene or isoform counts requires an in silico bioinformatics pipeline. Such pipelines are modular in nature, built using selected software and biological references. Software is usually chosen and parameterized according to the sequencing protocol and biological question. However, while biological and technical noise is alleviated through replicates, biases due to the pipeline and choice of biological references are often overlooked. Here, we show that the current standard practice prevents reproducibility in RNA-seq studies by failing to specify required methodological information. Peer-reviewed articles are intended to apply currently accepted scientific and methodological standards. Inasmuch as the bias-less and optimal RNA-seq pipeline is not perfectly defined, methodological information holds a meaningful role in defining the results. This work illustrates the need for a standardized and explicit display of methodological information in RNA-seq experiments.
| S-EPMC7820846 | biostudies-literature