Dataset Information

Inter- and Intra-Observer Variability and the Effect of Experience in Cine-MRI for Adhesion Detection.

ABSTRACT: Cine-MRI for adhesion detection is a promising novel modality that can help the large group of patients developing pain after abdominal surgery. Few studies into its diagnostic accuracy are available, and none address observer variability. This retrospective study explores the inter- and intra-observer variability, diagnostic accuracy, and the effect of experience. A total of 15 observers with a variety of experience reviewed 61 sagittal cine-MRI slices, placing box annotations with a confidence score at locations suspect for adhesions. Five observers reviewed the slices again one year later. Inter- and intra-observer variability are quantified using Fleiss' (inter) and Cohen's (intra) κ and percentage agreement. Diagnostic accuracy is quantified with receiver operating characteristic (ROC) analysis based on a consensus standard. Inter-observer Fleiss' κ values range from 0.04 to 0.34, showing poor to fair agreement. High general and cine-MRI experience led to significantly (p < 0.001) better agreement among observers. The intra-observer results show Cohen's κ values between 0.37 and 0.53 for all observers, except one with a low κ of -0.11. Group AUC scores lie between 0.66 and 0.72, with individual observers reaching 0.78. This study confirms that cine-MRI can diagnose adhesions, with respect to a radiologist consensus panel and shows that experience improves reading cine-MRI. Observers without specific experience adapt to this modality quickly after a short online tutorial. Observer agreement is fair at best and area under the receiver operating characteristic curve (AUC) scores leave room for improvement. Consistently interpreting this novel modality needs further research, for instance, by developing reporting guidelines or artificial intelligence-based methods.

SUBMITTER: de Wilde B

PROVIDER: S-EPMC10054690 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Inter- and Intra-Observer Variability and the Effect of Experience in Cine-MRI for Adhesion Detection.

de Wilde Bram B Joosten Frank F Venderink Wulphert W Davidse Mirjam E J MEJ Geurts Juliëtte J Kruijt Hanneke H Vermeulen Afke A Martens Bibi B Schyns Maxime V P MVP Huige Josephine C B M JCBM de Boer Myrte C MC Tonino Bart A R BAR Zandvoort Herman J A HJA Lammert Kirsti K Parviainen Helka H Vuorinen Aino-Maija AM Syväranta Suvi S Vogels Ruben R M RRM Prins Wiesje W Coppola Andrea A Bossa Nancy N Ten Broek Richard P G RPG Huisman Henkjan H

Journal of imaging 20230223 3

Cine-MRI for adhesion detection is a promising novel modality that can help the large group of patients developing pain after abdominal surgery. Few studies into its diagnostic accuracy are available, and none address observer variability. This retrospective study explores the inter- and intra-observer variability, diagnostic accuracy, and the effect of experience. A total of 15 observers with a variety of experience reviewed 61 sagittal cine-MRI slices, placing box annotations with a confidence ...[more]

PMID: 36976106

Similar Datasets

Project description:BACKGROUND: The Back Pain Cohort of Southern Denmark (BaPa Cohort) was initiated with the aim of evaluating the clinical relevance of magnetic resonance imaging (MRI) in the diagnosis of early spondyloarthritis (SpA). In order to facilitate the collection of MRI data for this study, an electronic evaluation form was developed including both SpA-related and degenerative axial changes. The objective of the current study was to assess the intra- and inter-observer agreement of the MRI changes assessed. METHODS: Three radiologists evaluated 48 MRI scans of the whole spine and the sacroiliac joints from a subsample of the BaPa Cohort, consisting of patients with non-specific low back pain and patients with different stages of SpA features. The spine was evaluated for SpA-related and degenerative MRI changes and the SIJ for SpA-related changes. Inter- and intra-observer agreements were calculated with kappa statistics. In the interpretation of the kappa coefficient, the standards for strength of agreement reported by Landis and Koch were followed. RESULTS: A total of 48 patients, 40% men and mean age of 31 years (range 18-40 years), were evaluated once by all three readers and re-evaluated by two of the readers after 4-12 weeks. For MRI changes in the spine, substantial to almost perfect observer agreement was found for the location and the size of vertebral signal changes and for disc degeneration and disc contour. For the sacroiliac joints, substantial or almost perfect observer agreement was found for the grading of bone marrow oedema and fatty marrow deposition, the depth of bone marrow oedema and for subchondral sclerosis. Global assessment of the SpA diagnosis had substantial to almost perfect observer agreements. CONCLUSION: The acceptable agreement for key MRI changes in the spine and sacroiliac joints makes it possible to use these MRI changes in the BaPa Cohort study and other studies investigating MRI changes in patients with non-specific low back pain and suspected SpA.

Dataset Information

Inter- and Intra-Observer Variability and the Effect of Experience in Cine-MRI for Adhesion Detection.

Publications

Inter- and Intra-Observer Variability and the Effect of Experience in Cine-MRI for Adhesion Detection.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets