STUDY ON INTEROBSERVER AGREEMENT IN A GOITRE PREVALENCE SURVEY.
Ontology highlight
ABSTRACT:
SUBMITTER: Pillai PB
PROVIDER: S-EPMC5529923 | biostudies-other | 1995 Apr
REPOSITORIES: biostudies-other
Similar Datasets
Project description:PurposeTo evaluate the interobserver reproducibility of two-dimensional shear wave elastography (2D-SWE) in measuring liver stiffness (LS) and to investigate factors related to liver 2D-SWE.Materials and methodsA prospective study was conducted between August 2011 and August 2012 in rheumatoid arthritis patients who had been treated with methotrexate. Interobserver reproducibility of 2D-SWE was evaluated, and the relationship between interobserver difference in LS and related factors was analyzed using linear regression analyses. We considered age, sex, alanine transaminase, cholesterol, body mass index (BMI), and waist circumference as clinical factors, and the mean value of standard deviation (SDM), its difference between two examiners, mean diameter of the regions of interest (ROIM), and its difference in the elasticity map as investigation factors. The cut-off value for significant factors to predict interobserver discrepancies in LS-based fibrosis stage was also inspected.ResultsIn total, 176 patients were enrolled. The intraclass correlation coefficient between the two examiners was 0.784. In the univariate analysis, SDM and ROIM were independently associated with interobserver differences in LS as well as BMI, waist circumference, and the difference of ROI, but SDM and ROIM were the only ones significantly related in multivariate analysis (p<0.001 and p = 0.021, respectively). The best cut-off value for SDM in predicting interobserver discrepancy in LS-based fibrosis stage was 1.4.ConclusionsInterobserver reproducibility of 2D-SWE for measuring LS was good and SDM was the most significantly associated factor with interobserver differences in LS and interobserver discordance in LS-based fibrosis stage.
| S-EPMC5393601 | biostudies-literature
Project description:BackgroundLung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID-19. Previously described LUS manifestations for COVID-19 include B-lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID-19 is unknown.MethodsThis study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans (n = 180 independent observations) collected from patients with COVID-19, diagnosed via RT-PCR. These studies were randomly selected from an image database consisting of COVID-19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values (κ) were used to calculate IRR.ResultsThere was substantial IRR on the following items: normal LUS scan (κ = 0.79 [95% CI: 0.72-0.87]), presence of B-lines (κ = 0.79 [95% CI: 0.72-0.87]), ≥3 B-lines observed (κ = 0.72 [95% CI: 0.64-0.79]). Moderate IRR was observed for the presence of any consolidation (κ = 0.57 [95% CI: 0.50-0.64]), subpleural consolidation (κ = 0.49 [95% CI: 0.42-0.56]), and presence of effusion (κ = 0.49 [95% CI: 0.41-0.56]). Fair IRR was observed for pleural thickening (κ = 0.23 [95% CI: 0.15-0.30]).DiscussionMany LUS manifestations for COVID-19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID-19 may include the presence/count of B-lines or determining if a scan is normal. Clinical protocols for LUS with COVID-19 may require additional observers for the confirmation of less reliable findings such as consolidations.
| S-EPMC8013417 | biostudies-literature
Project description:To determine the interobserver agreement (IOA) of neonatal seizure detection using the gold standard of conventional, multichannel EEG.A cohort of full-term neonates at risk of acute encephalopathy was included in this prospective study. The EEG recordings of these neonates were independently reviewed for seizures by three international experts. The IOA was estimated using statistical measures including Fleiss' kappa and percentage agreement assessed over seizure events (event basis) and seizure duration (temporal basis).A total of 4066 h of EEG recordings from 70 neonates were reviewed with an average of 2555 seizures detected. The IOA was high with temporal assessment resulting in a kappa of 0.827 (95% CI: 0.769-0.865; n = 70). The median agreement was 83.0% (interquartile range [IQR]: 76.6-89.5%; n = 33) for seizure and 99.7% (IQR: 98.9-99.8%; n = 70) for nonseizure EEG. Analysis of events showed a median agreement of 83.0% (IQR: 72.9-86.6%; n = 33) for seizures with 0.018 disagreements per hour (IQR: 0.000-0.090 per hour; n = 70). Observers were more likely to disagree when a seizure was less than 30 sec. Overall, 33 neonates were diagnosed with seizures and 28 neonates were not, by all three observers. Of the remaining nine neonates with contradictory EEG detections, seven presented with low total seizure burden.The IOA is high among experts for the detection of neonatal seizures using conventional, multichannel EEG. Agreement is reduced when seizures are rare or have short duration. These findings support EEG-based decision making in the neonatal intensive care unit, inform EEG interpretation guidelines, and provide benchmarks for seizure detection algorithms.
| S-EPMC4693620 | biostudies-other
Project description:AimTo assess the role of imaging in the early management of encephalitis and the agreement on findings in a well-defined cohort of suspected encephalitis cases enrolled in the Prospective Aetiological Study of Encephalitis conducted by the Health Protection Agency (now incorporated into Public Health England).Materials and methodsEighty-five CT examinations from 68 patients and 101 MRI examinations from 80 patients with suspected encephalitis were independently rated by three neuroradiologists blinded to patient and clinical details. The level of agreement on the interpretation of images was measured using the kappa statistic. The sensitivity, specificity, and negative and positive predictive values of CT and MRI for herpes simplex virus (HSV) encephalitis and acute disseminated encephalomyelitis (ADEM) were estimated.ResultsThe kappa value for interobserver agreement on rating the scans as normal or abnormal was good (0.65) for CT and moderate (0.59) for MRI. Agreement for HSV encephalitis was very good for CT (0.87) and MRI (0.82), but only fair for ADEM (0.32 CT; 0.31 MRI). Similarly, the overall sensitivity of imaging for HSV encephalitis was ?80% for both CT and MRI, whereas for ADEM it was 0% for CT and 20% for MRI. MRI specificity for HSV encephalitis between 3-10 days after symptom onset was 100%.ConclusionThere is a subjective component to scan interpretation that can have important implications for the clinical management of encephalitis cases. Neuroradiologists were good at diagnosing HSV encephalitis; however, agreement was worse for ADEM and other alternative aetiologies. Findings highlight the importance of a comprehensive and multidisciplinary approach to diagnosing the cause of encephalitis that takes into account individual clinical, microbiological, and radiological features of each patient.
| S-EPMC5021199 | biostudies-literature
Project description:Ultrasonography is the best available tool for the initial work-up of thyroid nodules. Substantial interobserver variability has been documented in the recognition and reporting of some of the lesion characteristics. A number of classification systems have been developed to estimate the likelihood of malignancy: several of them have been endorsed by scientific societies, but their reproducibility is yet to be assessed. We evaluated the interobserver variability of the AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS classification systems and the interobserver concordance in the indication to FNA biopsy. Two raters independently evaluated 1055 ultrasound images of thyroid nodules identified in 265 patients at multiple time points, in two separate sets (501 and 554 images). After the first set of nodules, a joint reading was performed to reach a consensus in the feature definitions. The interobserver agreement (Krippendorff alpha) in the first set of nodules was 0.47, 0.49, 0.49, 0.61 and 0.53, for AACE/ACE/AME, ACR, ATA, EU-TIRADS and K-TIRADS systems, respectively. The agreement for the indication to biopsy was substantial to near-perfect, being 0.73, 0.61, 0.75, 0.68 and 0.82, respectively (Cohen's kappa). For all systems, agreement on the nodules of the second set increased. Despite the wide variability in the description of single ultrasonographic features, the classification systems may improve the interobserver agreement that further ameliorates after a specific training. When selecting nodules to be submitted to FNA biopsy, that is main purpose of these classifications, the interobserver agreement is substantial to almost perfect.
| S-EPMC5744624 | biostudies-literature
Project description:BACKGROUND:Compared to 1.5 T, 3 T magnetic resonance imaging (MRI) increases signal-to-noise ratio leading to improved image quality. However, its clinical relevance in clinically isolated syndrome suggestive of multiple sclerosis remains uncertain. OBJECTIVES:The purpose of this study was to investigate how 3 T MRI affects the agreement between raters on lesion detection and diagnosis. METHODS:We selected 30 patients and 10 healthy controls from our ongoing prospective multicentre cohort. All subjects received baseline 1.5 and 3 T brain and spinal cord MRI. Patients also received follow-up brain MRI at 3-6 months. Four experienced neuroradiologists and four less-experienced raters scored the number of lesions per anatomical region and determined dissemination in space and time (McDonald 2010). RESULTS:In controls, the mean number of lesions per rater was 0.16 at 1.5 T and 0.38 at 3 T ( p = 0.005). For patients, this was 4.18 and 4.40, respectively ( p = 0.657). Inter-rater agreement on involvement per anatomical region and dissemination in space and time was moderate to good for both field strengths. 3 T slightly improved agreement between experienced raters, but slightly decreased agreement between less-experienced raters. CONCLUSION:Overall, the interobserver agreement was moderate to good. 3 T appears to improve the reading for experienced readers, underlining the benefit of additional training.
| S-EPMC6393953 | biostudies-literature
Project description:PurposeThis study investigated interobserver agreement in lung ultrasonography (LUS) in pregnant women performed by obstetricians with different levels of expertise, with confirmation by an expert radiologist.MethodsThis prospective study was conducted at a tertiary "Coronavirus Pandemic Hospital" in April 2020. Pregnant women suspected to have coronavirus disease 2019 (COVID-19) were included. Two blinded experienced obstetricians performed LUS on pregnant women separately and noted their scores for 14 lung zones. Following a theoretical and hands-on practical course, one experienced obstetrician, two novice obstetric residents, and an experienced radiologist blindly evaluated anonymized and randomized still images and videoclips retrospectively. Weighted Cohen's kappa and Krippendorff's alpha tests were used to assess the interobserver agreement.ResultsFifty-two pregnant women were included, with confirmed COVID-19 diagnosis rate of 82.7%. In total, 336 eligible still images and 115 videoclips were included in the final analysis. The overall weighted Cohen's kappa values ranged from 0.706 to 0.912 for the 14 lung zones. There were only seven instances of major disagreement (>1 point) in the evaluation of 14 lung zones of 52 patients (n=728). The overall agreement between the radiologist and obstetricians for the still images (Krippendorff's α=0.856, 95% confidence interval [CI], 0.797 to 0.915) and videoclips (Krippendorff's α=0.785; 95% CI, 0.709 to 0.861) was good.ConclusionThe interobserver agreement between obstetricians with different levels of experience on still images and videoclips of LUS was good. Following a brief theoretical course, obstetricians' performance of LUS in pregnant women and interpretation of pre-acquired LUS images can be considered consistent.
| S-EPMC7515667 | biostudies-literature
Project description:The aim of this study is to evaluate intra- and interobserver agreement for measurement of intracranial, cerebellar, and thalamic volume with the Virtual Organ Computer-aided AnaLysis (VOCAL) technique in three-dimensional ultrasound images, in comparison to two-dimensional measurements of these brain structures. Three-dimensional ultrasound images of the brains of 80 fetuses at 20-24 weeks' gestational age were obtained from YOUth, a Dutch prospective cohort study. Two observers performed offline measurement of the occipitofrontal diameter, intracranial volume, transcerebellar diameter, cerebellar volume, and thalamic width, area, and volume, independently. VOCAL was used for calculation of the volumes. The two-way random, single measures intraclass correlation coefficient (ICC) was used for analysis of agreement and Bland-Altman plots were configured. Intra- and interobserver agreement was almost perfect for occipitofrontal diameter (intra ICC 0.88, 95% CI 0.82-0.92; inter ICC 0.91, 95% CI 0.85-0.94), intracranial volume (intra ICC 0.96, 95% CI 0.91-0.98; inter ICC 0.97, 95% CI 0.96-0.98) and transcerebellar diameter (intra ICC 0.91, 95% CI 0.86-0.94; inter ICC 0.86, 95% CI 0.78-0.910). For cerebellar volume, the intraobserver agreement was almost perfect (0.85, 95% CI 0.76-0.90), whereas the interobserver agreement was substantial (0.75, 95% CI 0.44-0.88). Agreement was only moderate for thalamic measurements. Bland-Altman plots for the volume measurements are normally distributed with acceptable mean differences and 95% limits of agreement. The intra- and interobserver agreement of the measurement of intracranial and cerebellar volume with VOCAL was almost perfect. These measurements are therefore reliable, and can be used to investigate fetal brain development. Thalamic measurements are not reliable enough.
| S-EPMC6866380 | biostudies-literature
Project description:ObjectiveThe objective was to assess the interobserver agreement rate, progression rates and malignancy rates in the assessment of complex renal cysts (≥ Bosniak IIF) using a population-based database.MethodsA regional database identified 452 complex renal cysts in 415 patients between 2009 and 2019. Each patient was tracked and followed up using a unique identifier and deterministic linkage methodology. The interobserver agreement rate between radiologists was calculated using a weighted kappa statistic. Progression and malignancy rates of cysts (Bosniak ≥IIF) over the 11-year period were calculated.ResultsThe linear-weighted kappa value was 0.69 for all complex cysts. The rate of progression and regression of Bosniak IIF cysts was 4.6% (7/151) and 3.3% (5/151), respectively. All malignant IIF cysts progressed within 16 months of diagnosis. The malignancy rate of surgically resected Bosniak III and IV cysts was 79.3% (23/29) and 84.5% (39/46), respectively. Of all malignant tumours, 73.8% and 93.7% were of low ISUP grade and low stage, respectively.ConclusionsThis study further confirms that there is a good degree of agreement between radiologists in classifying complex renal masses using the Bosniak classification. The progression rate of Bosniak IIF cysts is low, but the malignancy rates of surgically resected Bosniak IIF, III and IV cysts are high. Benign cysts are frequently resected, and a very high proportion of histopathologically confirmed cancers in complex renal cysts are of low grade and stage.Key points• There is a good degree of agreement between radiologists in classifying complex renal masses using the Bosniak classification. • The rate of progression of Bosniak IIF cysts is low, and malignant cysts progress early during surveillance. Although the malignancy rates of resected Bosniak IIF, III and IV cysts are high, the rate of benign cyst resection is significant.
| S-EPMC7813744 | biostudies-literature
Project description:ObjectiveTo determine the interobserver agreement of history and physical examination findings in children undergoing evaluation in the emergency department (ED) for headaches.Study designWe conducted a prospective, cross-sectional study of children aged 2-17 years evaluated at 3 tertiary-care pediatric EDs for non-traumatic headaches. Two clinicians independently completed a standardized assessment of each child and documented the presence or absence of history and physical examination variables. Unweighted κ statistics were determined for 68 history and 24 physical examination variables.ResultsWe analyzed 191 paired observations; median age was 12 years, with 19 (9.9%) children younger than 7 years. Interrater reliability was at least moderate (κ ≥ 0.41) for 41 (60.3%) patient history variables. Eleven (61.1%) of 18 physical examination variables for which κ statistics could be calculated had a κ that was at least moderate.ConclusionsA substantial number of history and physical examination findings demonstrated at least moderate κ statistic values when assessed in children with headaches in the ED. These variables may be generalizable across different types of clinicians for evaluation of children with headaches. If also found to predict the presence or absence of emergent intracranial abnormalities, the more reliable clinical findings may be helpful in the development of clinical prediction rules or risk stratification models that could be used across settings for children with headaches.
| S-EPMC7251971 | biostudies-literature