Dataset Information

Interrater agreement of contouring of the neurovascular bundles and internal pudendal arteries in neurovascular-sparing magnetic resonance-guided radiotherapy for localized prostate cancer

ABSTRACT: Highlights • Interrater DSC of the NVB was 0.60 and 0.61 for the left and right side respectively.• Interrater DSC of the IPA was 0.59 for the left and right side.• Agreement was best for the inferior half (i.e. prostate apex to midgland) of the NVB.• Agreement improved with MRI optimization and rater training.

Background and purpose

Radiation damage to neural and vascular tissue, such as the neurovascular bundles (NVBs) and internal pudendal arteries (IPAs), during radiotherapy for prostate cancer (PCa) may cause erectile dysfunction. Neurovascular-sparing magnetic resonance-guided adaptive radiotherapy (MRgRT) aims to preserve erectile function after treatment. However, the NVBs and IPAs are not routinely contoured in current radiotherapy practice. Before neurovascular-sparing MRgRT for PCa can be implemented, the interrater agreement of the contouring of the NVBs and IPAs on pre-treatment MRI needs to be assessed.

Materials and methods

Four radiation oncologists independently contoured the prostate, NVB, and IPA in an unselected consecutive series of 15 PCa patients, on pre-treatment MRI. Dice similarity coefficients (DSCs) for pairwise interrater agreement of contours were calculated. Additionally, the DCS of a subset of the inferior half of the NVB contours (i.e. approximately prostate midgland to apex level) was calculated.

Results

Median overall interrater DSC for the left and right NVB was 0.60 (IQR: 0.54 – 0.68) and 0.61 (IQR: 0.53 – 0.69) respectively and for the left and right IPA 0.59 (IQR: 0.53 – 0.64) and 0.59 (IQR: 0.52 – 0.64) respectively. Median overall interrater DSC for the inferior half of the left NVB was 0.67 (IQR: 0.58 – 0.74) and 0.67 (IQR: 0.61 – 0.71) for the right NVB.

Conclusion

We found that the interrater agreement for the contouring of the NVB and IPA improved with enhancement of the MRI sequence as well as further training of the raters. The agreement was best in the subset of the inferior half of the NVB, where a good agreement is clinically most relevant for neurovascular-sparing MRgRT for PCa.

SUBMITTER: Teunissen F

PROVIDER: S-EPMC8605225 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:OBJECTIVE:The presence of interictal epileptiform discharges (IED) in the electroencephalogram (EEG) is a key finding in the medical workup of a patient with suspected epilepsy. However, inter-rater agreement (IRA) regarding the presence of IED is imperfect, leading to incorrect and delayed diagnoses. An improved understanding of which IED attributes mediate expert IRA might help in developing automatic methods for IED detection able to emulate the abilities of experts. Therefore, using a set of IED scored by a large number of experts, we set out to determine which attributes of IED predict expert agreement regarding the presence of IED. METHODS:IED were annotated on a 5-point scale by 18 clinical neurophysiologists within 200 30-s EEG segments from recordings of 200 patients. 5538 signal analysis features were extracted from the waveforms, including wavelet coefficients, morphological features, signal energy, nonlinear energy operator response, electrode location, and spectrogram features. Feature selection was performed by applying elastic net regression and support vector regression (SVR) was applied to predict expert opinion, with and without the feature selection procedure and with and without several types of signal normalization. RESULTS:Multiple types of features were useful for predicting expert annotations, but particular types of wavelet features performed best. Local EEG normalization also enhanced best model performance. As the size of the group of EEGers used to train the models was increased, the performance of the models leveled off at a group size of around 11. CONCLUSIONS:The features that best predict inter-rater agreement among experts regarding the presence of IED are wavelet features, using locally standardized EEG. Our models for predicting expert opinion based on EEGer's scores perform best with a large group of EEGers (more than 10). SIGNIFICANCE:By examining a large group of EEG signal analysis features we found that wavelet features with certain wavelet basis functions performed best to identify IEDs. Local normalization also improves predictability, suggesting the importance of IED morphology over amplitude-based features. Although most IED detection studies in the past have used opinion from three or fewer experts, our study suggests a "wisdom of the crowd" effect, such that pooling over a larger number of expert opinions produces a better correlation between expert opinion and objectively quantifiable features of the EEG.

Project description:PURPOSE:Many neonates undergo electroencephalogram (EEG) monitoring to identify and manage acute symptomatic seizures. Information about brain function contained in the EEG background data may also help predict neurobehavioral outcomes. For EEG background features to be useful as prognostic indicators, the interpretation of these features must be standardized across electroencephalographers. We aimed at determining the interrater and intrarater agreement among electroencephalographers interpreting neonatal EEG background patterns. METHODS:Five neonatal electroencephalographers reviewed 5-to-7.5-minute epochs of EEG from full-term neonates who underwent continuous conventional EEG monitoring. The EEG assessment tool used to classify background patterns was based on the American Clinical Neurophysiology Society's guideline for neonatal EEG terminology. Interrater and intrarater agreement were measured using Kappa coefficients. RESULTS:Interrater agreement was consistently highest for voltage (binary: substantial, kappa = 0.783; categorical: moderate, kappa = 0.562), seizure presence (fair-substantial; kappa = 0.375-0.697), continuity (moderate; kappa = 0.481), burst voltage (moderate; kappa = 0.574), suppressed background presence (moderate-substantial; kappa = 0.493-0.643), delta activity presence (fair-moderate; kappa = 0.369-0.432), theta activity presence (fair-moderate; kappa = 0.347-0.600), presence of graphoelements (fair; kappa = 0.381), and overall impression (binary: moderate, kappa = 0.495; categorical: fair-moderate, kappa = 0.347, 0.465). Agreement was poor or inconsistent for all other patterns. Intrarater agreement was variable, with highest average agreement for voltage (binary: substantial, kappa = 0.75; categorical: substantial, kappa = 0.714) and highest consistent agreement for continuity (moderate-substantial; kappa = 0.43-0.67) and overall impression (moderate-substantial; kappa = 0.42-0.68). CONCLUSIONS:This study demonstrates substantial variability in neonatal EEG background interpretation across electroencephalographers, indicating a need for educational and technological strategies aimed at improving performance.

Project description:BackgroundAnterior cruciate ligament (ACL) rupture is the most common ligament injury treated surgically by orthopaedic surgeons. The gold standard for the treatment of the majority of primary ACL tears is ACL reconstruction. However, novel methods of repair, such as bridge-enhanced ACL repair (BEAR), are currently being investigated as alternatives to reconstruction. To assess patients for midsubstance repair suitability, clarify the prognostic implications of injury location and damage, and evaluate the results of a repair technique, it is important to have a baseline classification system or grading scale that is reproducible across surgeons, particularly for multicenter collaboration. Currently, no such system or scale exists.PurposeTo develop an arthroscopic ACL tear classification system and to evaluate its interobserver reliability.Study designCohort study (diagnosis); Level of evidence, 3.MethodsEleven fellowship-trained orthopaedic surgeon investigators reviewed 75 video clips containing arthroscopic evaluation of a torn ACL and then completed the 6-question ACL Pathology Evaluation Form. Agreement statistics including exact agreement, Fleiss ?, Gwet agreement coefficient 1 (AC1), and Gwet AC2 were then calculated to assess interobserver reliability.ResultsIn aggregate, the multiple assessments of observer reproducibility revealed that surgeon participants in this study, when evaluating the same injury, agreed roughly 80% of the time on whether (1) at least 50% of the tibial footprint remained, (2) the remaining tibial stump was ?10 mm, and (3) the injury was therefore reparable using the BEAR procedure. Participants also agreed roughly 60% of the time on exactly how many suturable bundles were available. These characteristics are believed to be most important, among those studied, in determining whether a torn ACL is amenable to midsubstance repair.ConclusionThis study is the first of its kind to demonstrate the interobserver reliability of arthroscopic classification of ACL tears. We have demonstrated that this classification system, though not ideally reproducible, is reliable enough across surgeons at multiple institutions for use in multicenter studies.RegistrationNCT03776162 (ClinicalTrials.gov identifier).