Dataset Information

Evaluation of an Artificial Intelligence-Augmented Digital System for Histologic Classification of Colorectal Polyps.

ABSTRACT:

Importance

Colorectal polyps are common, and their histopathologic classification is used in the planning of follow-up surveillance. Substantial variation has been observed in pathologists' classification of colorectal polyps, and improved assessment by pathologists may be associated with reduced subsequent underuse and overuse of colonoscopy.

Objective

To compare standard microscopic assessment with an artificial intelligence (AI)-augmented digital system that annotates regions of interest within digitized polyp tissue and predicts polyp type using a deep learning model to assist pathologists in colorectal polyp classification.

Design, setting, and participants

In this diagnostic study conducted at a tertiary academic medical center and a community hospital in New Hampshire, 100 slides with colorectal polyp samples were read by 15 pathologists using a microscope and an AI-augmented digital system, with a washout period of at least 12 weeks between use of each modality. The study was conducted from February 10 to July 10, 2020.

Main outcomes and measures

Accuracy and time of evaluation were used to compare pathologists' performance when a microscope was used with their performance when the AI-augmented digital system was used. Outcomes were compared using paired t tests and mixed-effects models.

Results

In assessments of 100 slides with colorectal polyp specimens, use of the AI-augmented digital system significantly improved pathologists' classification accuracy compared with microscopic assessment from 73.9% (95% CI, 71.7%-76.2%) to 80.8% (95% CI, 78.8%-82.8%) (P < .001). The overall difference in the evaluation time per slide between the digital system (mean, 21.7 seconds; 95% CI, 20.8-22.7 seconds) and microscopic examination (mean, 13.0 seconds; 95% CI, 12.4-13.5 seconds) was -8.8 seconds (95% CI, -9.8 to -7.7 seconds), but this difference decreased as pathologists became more familiar and experienced with the digital system; the difference between the time of evaluation on the last set of 20 slides for all pathologists when using the microscope and the digital system was 4.8 seconds (95% CI, 3.0-6.5 seconds).

Conclusions and relevance

In this diagnostic study, an AI-augmented digital system significantly improved the accuracy of pathologic interpretation of colorectal polyps compared with microscopic assessment. If applied broadly to clinical practice, this tool may be associated with decreases in subsequent overuse and underuse of colonoscopy and thus with improved patient outcomes and reduced health care costs.

SUBMITTER: Nasir-Moin M

PROVIDER: S-EPMC8603082 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ImportanceAccurate assessment of wound area and percentage of granulation tissue (PGT) are important for optimizing wound care and healing outcomes. Artificial intelligence (AI)-based wound assessment tools have the potential to improve the accuracy and consistency of wound area and PGT measurement, while improving efficiency of wound care workflows.ObjectiveTo develop a quantitative and qualitative method to evaluate AI-based wound assessment tools compared with expert human assessments.Design, setting, and participantsThis diagnostic study was performed across 2 independent wound centers using deidentified wound photographs collected for routine care (site 1, 110 photographs taken between May 1 and 31, 2018; site 2, 89 photographs taken between January 1 and December 31, 2019). Digital wound photographs of patients were selected chronologically from the electronic medical records from the general population of patients visiting the wound centers. For inclusion in the study, the complete wound edge and a ruler were required to be visible; circumferential ulcers were specifically excluded. Four wound specialists (2 per site) and an AI-based wound assessment service independently traced wound area and granulation tissue.Main outcomes and measuresThe quantitative performance of AI tracings was evaluated by statistically comparing error measure distributions between test AI traces and reference human traces (AI vs human) with error distributions between independent traces by 2 humans (human vs human). Quantitative outcomes included statistically significant differences in error measures of false-negative area (FNA), false-positive area (FPA), and absolute relative error (ARE) between AI vs human and human vs human comparisons of wound area and granulation tissue tracings. Six masked attending physician reviewers (3 per site) viewed randomized area tracings for AI and human annotators and qualitatively assessed them. Qualitative outcomes included statistically significant difference in the absolute difference between AI-based PGT measurements and mean reviewer visual PGT estimates compared with PGT estimate variability measures (ie, range, standard deviation) across reviewers.ResultsA total of 199 photographs were selected for the study across both sites; mean (SD) patient age was 64 (18) years (range, 17-95 years) and 127 (63.8%) were women. The comparisons of AI vs human with human vs human for FPA and ARE were not statistically significant. AI vs human FNA was slightly elevated compared with human vs human FNA (median [IQR], 7.7% [2.7%-21.2%] vs 5.7% [1.6%-14.9%]; P < .001), indicating that AI traces tended to slightly underestimate the human reference wound boundaries compared with human test traces. Two of 6 reviewers had a statistically higher frequency in agreement that human tracings met the standard area definition, but overall agreement was moderate (352 yes responses of 583 total responses [60.4%] for AI and 793 yes responses of 1166 total responses [68.0%] for human tracings). AI PGT measurements fell in the typical range of variation in interreviewer visual PGT estimates; however, visual PGT estimates varied considerably (mean range, 34.8%; mean SD, 19.6%).Conclusions and relevanceThis study provides a framework for evaluating AI-based digital wound assessment tools that can be extended to automated measurements of other wound features or adapted to evaluate other AI-based digital image diagnostic tools. As AI-based wound assessment tools become more common across wound care settings, it will be important to rigorously validate their performance in helping clinicians obtain accurate wound assessments to guide clinical care.

Project description:ImportanceHistologic classification of colorectal polyps plays a critical role in screening for colorectal cancer and care of affected patients. An accurate and automated algorithm for the classification of colorectal polyps on digitized histopathologic slides could benefit practitioners and patients.ObjectiveTo evaluate the performance and generalizability of a deep neural network for colorectal polyp classification on histopathologic slide images using a multi-institutional data set.Design, setting, and participantsThis prognostic study used histopathologic slides collected from January 1, 2016, to June 31, 2016, from Dartmouth-Hitchcock Medical Center, Lebanon, New Hampshire, with 326 slides used for training, 157 slides for an internal data set, and 25 for a validation set. For the external data set, 238 slides for 179 distinct patients were obtained from 24 institutions across 13 US states. Data analysis was performed from April 9 to November 23, 2019.Main outcomes and measuresAccuracy, sensitivity, and specificity of the model to classify 4 major colorectal polyp types: tubular adenoma, tubulovillous or villous adenoma, hyperplastic polyp, and sessile serrated adenoma. Performance was compared with that of local pathologists' at the point of care identified from corresponding pathology laboratories.ResultsFor the internal evaluation on the 157 slides with ground truth labels from 5 pathologists, the deep neural network had a mean accuracy of 93.5% (95% CI, 89.6%-97.4%) compared with local pathologists' accuracy of 91.4% (95% CI, 87.0%-95.8%). On the external test set of 238 slides with ground truth labels from 5 pathologists, the deep neural network achieved an accuracy of 87.0% (95% CI, 82.7%-91.3%), which was comparable with local pathologists' accuracy of 86.6% (95% CI, 82.3%-90.9%).Conclusions and relevanceThe findings suggest that this model may assist pathologists by improving the diagnostic efficiency, reproducibility, and accuracy of colorectal cancer screenings.

Project description:Backgroundartificial intelligence (AI) for cellular phenotyping diagnosis of nasal polyps by whole-slide imaging (WSI) is lacking. We aim to establish an AI chronic rhinosinusitis evaluation platform 2.0 (AICEP 2.0) to obtain the proportion of inflammatory cells for cellular phenotyping diagnosis of nasal polyps and to explore the clinical significance of different phenotypes of nasal polyps on the WSI.Methodsa total of 453 patients were enrolled in our study. For the development of AICEP 2.0, 179 patients (WSIs) were obtained from the Third Affiliated Hospital of Sun Yat-Sen University (3HSYSU) from January 2008 to December 2018. A total of 24,625 patches were automatically extracted from the regions of interest under a 400× HPF by Openslide and the number of inflammatory cells in these patches was counted by two pathologists. For the application of AICEP 2.0 in a prospective cohort, 158 patients aged 14-70 years old with chronic rhinosinusitis with nasal polyps (CRSwNP) who had undergone endoscopic sinus surgery at 3HSYSU from June 2020 to December 2020 were included for preoperative demographic characteristics. For the application of AICEP 2.0 in a retrospective cohort, 116 patients with CRSwNP who had undergone endoscopic sinus surgery from May 2016 to June 2017 were enrolled for the recurrence rate. The proportion of inflammatory cells of these patients on WSI was calculated by our AICEP 2.0.Findingsfor AICEP 2.0, the mean absolute errors of the ratios of eosinophils, lymphocytes, neutrophils, and plasma cells were 1.64%, 2.13%, 1.06%, and 1.22%, respectively. The four phenotypes of nasal polyps were significantly different in clinical characteristics (including asthma, itching, sneezing, total IgE, peripheral eosinophils%, tissue eosinophils%, tissue neutrophils%, tissue lymphocytes%, tissue plasma cells%, and recurrence rate; P <0.05), but there were no significant differences in age distribution, onset time, total VAS score, Lund-Kennedy score, or Lund-Mackay score. The percentage of peripheral eosinophils was positively correlated with the percentage of tissue eosinophils (r = 0.560, P <0.001) and negatively correlated with tissue lymphocytes% (r = -0.489, P <0.001), tissue neutrophils% (r = -0.225, P = 0.005), and tissue plasma cells% (r = -0.266, P = 0.001) in WSIs.

Project description:BackgroundTo investigate machine-learning (ML) algorithms to differentiate corneal biomechanical properties between different topographical stages of keratoconus (KC) by dynamic Scheimpflug tonometry (CST, Corvis ST, Oculus, Wetzlar, Germany). In the following, ML models were used to predict the severity in a training and validation dataset.MethodsThree hundred and eighteen keratoconic and one hundred sixteen healthy eyes were included in this monocentric and cross-sectional pilot study. Dynamic corneal response (DCR) and corneal thickness related (pachymetric) parameters from CST were chosen by appropriated selection techniques to develop a ML algorithm. The stage of KC was determined by the topographical keratoconus classification system (TKC, Pentacam, Oculus). Patients who were classified as TKC 1, TKC 2 and TKC 3 were assigned to subgroup mild, moderate, and advanced KC. If patients were classified as TKC 1-2, TKC 2-3 or TKC 3-4, they were assigned to subgroups according to the normative range of further corneal indices (index of surface variance, keratoconus index and minimum radius). Patients classified as TKC 4 were not included in this study due to the limited amount of cases. Linear discriminant analysis (LDA) and random forest (RF) algorithms were used to develop the classification models. Data were divided into training (70% of cases) and validation (30% of cases) datasets.ResultsLDA model predicted healthy, mild, moderate, and advanced KC eyes with a sensitivity (Sn)/specificity (Sp) of 82%/97%, 73%/81%, 62%/83% and 68%/95% from a validation dataset, respectively. For the RF model, a Sn/Sp of 91%/94%, 80%/90%, 63%/87%, 72%/95% could be reached for predicting healthy, mild, moderate, and advanced KC eyes, respectively. The overall accuracy of LDA and RF was 71% and 78%, respectively. The accuracy for KC detection including all subgroups of KC severity was 93% in both models.ConclusionThe RF model showed good accuracy in predicting healthy eyes and various stages of KC. The accuracy was superior with respect to the LDA model. The clinical importance of the models is that the standalone dynamic Scheimpflug tonometry is able to predict the severity of KC without having the keratometric data.Trial registrationNCT04251143 at Clinicaltrials.gov, registered at 12 March 2018 (Retrospectively registered).