Dataset Information

High reproducibility of histological characterization by whole virtual slide quantification; an example using carotid plaque specimens.

ABSTRACT:

Objective

Tissue biobanks are an important source for discovery and validation studies aiming for new proteins that are causally related with disease development. There is an increasing demand for accurate and reproducible histological characterization, especially for subsequent analysis and interpretation of data in association studies. We assessed reproducibility of one semiquantative and two quantitative methods for histological tissue characterization. We introduce a new automated method for whole digital slide quantification. Carotid atherosclerotic plaques were used to test reproducibility.

Methods

50 atherosclerotic plaques that were obtained during carotid endarterectomy were analysed. For the semiquantitative analysis, 6 different plaque characteristics were scored in categories by two independent observers, and Cohen's ? was used to test intra- and interobserver reproducibility. The computer-aided method (assessed by two independent observers) and automated method were tested on CD68 (for macrophages) and ? smooth muscle actin (for smooth muscle cells) stainings. Agreement for these two methods (done on a continuous scale) was assessed by intraclass correlation coefficients (ICCs).

Results

For the semiquantitative analysis, ? values ranged from 0.55 to 0.69 for interobserver variability, and were slightly higher for intraobserver reproducibility in both observers. The computer-aided method yielded intra- and interobserver ICCs between 0.6 and 0.9. The new automated method performed most optimal regarding reproducibility, with ICCs ranging from 0.92 to 0.97.

Conclusions

The analysis of performance of three methods for histological slide characterization on carotid atherosclerotic plaques showed high precision and agreement in repeated measurements for the automated method for whole digital slide quantification. We suggest that this method can fulfill the need for reproducible histological quantification.

SUBMITTER: Vrijenhoek JE

PROVIDER: S-EPMC4277457 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

High reproducibility of histological characterization by whole virtual slide quantification; an example using carotid plaque specimens.

Vrijenhoek Joyce E P JE Nelissen Bastiaan G L BG Velema Evelyn E Vons Kristy K de Vries Jean-Paul P M JP Eijkemans Marinus J C MJ den Ruijter Hester M HM de Borst Gert Jan GJ Moll Frans L FL Pasterkamp Gerard G

PloS one 20141226 12

<h4>Objective</h4>Tissue biobanks are an important source for discovery and validation studies aiming for new proteins that are causally related with disease development. There is an increasing demand for accurate and reproducible histological characterization, especially for subsequent analysis and interpretation of data in association studies. We assessed reproducibility of one semiquantative and two quantitative methods for histological tissue characterization. We introduce a new automated me ...[more]

PMID: 25541691

Similar Datasets

Project description:For a method to be widely adopted in medical research or clinical practice, it needs to be reproducible so that clinicians and regulators can have confidence in its use. Machine learning and deep learning have a particular set of challenges around reproducibility. Small differences in the settings or the data used for training a model can lead to large differences in the outcomes of experiments. In this work, three top-performing algorithms from the Camelyon grand challenges are reproduced using only information presented in the associated papers and the results are then compared to those reported. Seemingly minor details were found to be critical to performance and yet their importance is difficult to appreciate until the actual reproduction is attempted. We observed that authors generally describe the key technical aspects of their models well but fail to maintain the same reporting standards when it comes to data preprocessing which is essential to reproducibility. As an important contribution of the present study and its findings, we introduce a reproducibility checklist that tabulates information that needs to be reported in histopathology ML-based work in order to make it reproducible. Author summary For a method to be used a lot in medical research or clinical practice, it needs to be able to be reproduced so that people can trust it. Machine learning and deep learning have some challenges around this. For example, small changes in how a model is trained can lead to significant changes in the results of experiments. This makes it essential that researchers report how they do things in enough detail for the results of their experiments to be reproducible. In this work, we looked at three different algorithms used for digital pathology image analysis. We tried to reproduce them using only the information reported in their papers. We confirmed that even minor details could be essential. Authors often do not report all the details needed to reproduce their work. We also created a checklist of things that need to be reported to help other researchers make their work reproducible.

Project description:BACKGROUND AND PURPOSE:Techniques to stratify subgroups of patients with asymptomatic carotid artery disease are urgently needed to guide decisions on optimal treatment. Reliance on estimates of % luminal stenosis has not been effective, perhaps because that approach entirely disregards potentially important information on the pathological process in the wall of the artery. METHODS:Since plaque lipid is a key determinant of plaque behaviour we used a newly validated, high-sensitivity T2-mapping MR technique for a systematic survey of the quantity and distribution of plaque lipid in patients undergoing endarterectomy. Lipid percentage was quantified in 50 carotid endarterectomy patients. Lipid distribution was tested, using two imaging indices (contribution of the largest lipid deposit towards total lipid (LLD %) and a newly-developed LAI 'lipid aggregation index'). RESULTS:The bifurcation contained maximal lipid volume. Lipid percentage was higher in symptomatic vs. asymptomatic patients with degree of stenosis (DS ? 50%) and in the total cohort (P = 0.013 and P = 0.005, respectively). Both LLD % and LAI was higher in symptomatic patients (P = 0.028 and P = 0.018, respectively), suggesting that for a given plaque lipid volume, coalesced deposits were more likely to be associated with symptomatic events. There was no correlation between plaque volume or lipid content and degree of luminal stenosis measured on ultrasound duplex (r = -0.09, P = 0.53 and r = -0.05, P = 0.75), respectively. However, there was a strong correlation in lipid between left and right carotid arteries (r = 0.5, P <0.0001, respectively). CONCLUSIONS:Plaque lipid content and distribution is associated with symptomatic status of the carotid plaque. Importantly, plaque lipid content was not related to the degree of luminal stenosis assessed by ultrasound. Determination of plaque lipid content may prove useful for stratification of asymptomatic patients, including selection of optimal invasive treatments.

Dataset Information

High reproducibility of histological characterization by whole virtual slide quantification; an example using carotid plaque specimens.

Objective

Methods

Results

Conclusions

Publications

High reproducibility of histological characterization by whole virtual slide quantification; an example using carotid plaque specimens.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets